Thread: Getting a move on for 8.2 beta
(I've already bounced this off the core committee, but it's time for
wider discussion.)
September is upon us and it doesn't seem like we are ready to ship
a beta. I think it's time to start making some hard choices.
In the main code I see these outstanding features/patches:
* bitmap indexes
* updatable views
* GUC variable reload + refactoring (previously applied and reverted)
* plpython improvements (needs review by someone who knows plpython)
* plpgsql improvements for returning record types
* patch to build on VC
* make libpq default client_encoding setting from locale?
In contrib we've got:
* new ISBN/etc module
* hstore (finally proposed for inclusion)
* new sslinfo module
* pgstattuple changes
* removing the deadwood
These are not all the open issues, for sure, but these are what I think
have to be resolved before we can go beta. Everything else is in the
category of "bug fixes", and none of it seems like a showstopper (in
fact a lot of the small stuff on my todo list is bugs reported against
8.1, so those certainly are not beta-stoppers).
My feeling is that we ought to bounce bitmap indexes and updatable views
as not being ready, accept all the contrib stuff, and try to get the
other items done in time for a beta at, say, the end of next week.
I'm not thrilled about postponing those two large items till 8.3, but
we are a month past feature freeze and we do not have credible patches
in hand for either. The alternative is to be willing to slip the
schedule until whenever they are ready, and we have learned the hard way
that that's not good project management.
Comments?
regards, tom lane
> My feeling is that we ought to bounce bitmap indexes and updatable views > as not being ready, accept all the contrib stuff, and try to get the > other items done in time for a beta at, say, the end of next week. > > I'm not thrilled about postponing those two large items till 8.3, but > we are a month past feature freeze and we do not have credible patches > in hand for either. I know there is a lot of "backend" work that has been done for 8.2 but the two features you are suggesting we bounce, are probably the most visible of marketable features we will have for this release. Especially updateable views. > The alternative is to be willing to slip the > schedule until whenever they are ready, and we have learned the hard way > that that's not good project management. Well I don't like the idea either, but if it isn't done, it isn't done. How often do we hear the complaint about software that is shipped before it is done? Outside of feature freeze, we really haven't had AFAICS a "solid" release schedule. Is it going to be push Beta for a month? Does that push us till the end of the year? I don't know which is the better solution, but you asked for comments :) Sincerely, Joshua D. Drake > > Comments? > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 3: Have you checked our extensive FAQ? > > http://www.postgresql.org/docs/faq > -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
Tom Lane wrote: > September is upon us and it doesn't seem like we are ready to ship > a beta. I think it's time to start making some hard choices. > > In the main code I see these outstanding features/patches: > > * bitmap indexes > * updatable views IMHO these two should be bounced. They are big features that still seem to need a lot of work and will probably take long before they are ready for inclusion. Interested parties should continue working on them with an eye to be included *at the start* of the 8.3 development cycle. > * GUC variable reload + refactoring (previously applied and reverted) > * plpython improvements (needs review by someone who knows plpython) > * plpgsql improvements for returning record types > * patch to build on VC > * make libpq default client_encoding setting from locale? > In contrib we've got: > > * new ISBN/etc module > * hstore (finally proposed for inclusion) > * new sslinfo module > * pgstattuple changes > * removing the deadwood These all seem to be manageable within reasonable time. I can help with the contrib stuff. > My feeling is that we ought to bounce bitmap indexes and updatable views > as not being ready, accept all the contrib stuff, and try to get the > other items done in time for a beta at, say, the end of next week. +1. -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Tom, > September is upon us and it doesn't seem like we are ready to ship > a beta. I think it's time to start making some hard choices. > > In the main code I see these outstanding features/patches: > > * bitmap indexes > * updatable views > * GUC variable reload + refactoring (previously applied and reverted) > * plpython improvements (needs review by someone who knows plpython) > * plpgsql improvements for returning record types > * patch to build on VC What's VC? > * make libpq default client_encoding setting from locale? > > In contrib we've got: > > * new ISBN/etc module > * hstore (finally proposed for inclusion) > * new sslinfo module > * pgstattuple changes > * removing the deadwood This is just a delete. I've fixed the pgFoundry permissions issues and will be loading the last CVS snapshot today. At that point, Bruce can delete stuff. Can you be a little clearer on which things are non-working patches, and which things are pending due to lack of review? I'm all for bouncing stuff that's not ready to go, but bouncing stuff because we haven't recruited enough code reviewers is just bad practice. -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco
* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> My feeling is that we ought to bounce bitmap indexes and updatable views
> as not being ready, accept all the contrib stuff, and try to get the
> other items done in time for a beta at, say, the end of next week.
I agree with bouncing these to get the beta out, though I hate having
to wait another year for them. If things change in some way and we have
the opportunity to fit one in this time, I'd rather see updatable views
than bitmap indexes, personally.
Overall, I really think 8.2 is going to be an excellent release. I wish
autovacuum could have been enabled by default and I'd just like to ask,
now and I'll try to remember again once 8.2 is out, please let's turn it
on by default for 8.3 (and early on so we get some good testing of it).
Thanks,
Stephen
Josh Berkus <josh@agliodbs.com> writes:
> Can you be a little clearer on which things are non-working patches, and
> which things are pending due to lack of review?
The only things I currently want to reject as not having provided a
timely patch are bitmap indexes and updatable views. The other stuff
I mentioned is in need of review, but I don't currently have a reason
to think it can't get in.
> I'm all for bouncing
> stuff that's not ready to go, but bouncing stuff because we haven't
> recruited enough code reviewers is just bad practice.
Well, it's taken us the full month to get this far through the queue, so
I'd sure like to have more people on board reviewing next time. Alvaro
helped but I miss Neil Conway, and some other people have helped in the
past. However --- the present situation has nothing to do with lack of
reviewers, it's lack of time to finish the patches.
regards, tom lane
"Joshua D. Drake" <jd@commandprompt.com> writes:
>> My feeling is that we ought to bounce bitmap indexes and updatable views
>> as not being ready, accept all the contrib stuff, and try to get the
>> other items done in time for a beta at, say, the end of next week.
> I know there is a lot of "backend" work that has been done for 8.2 but
> the two features you are suggesting we bounce, are probably the most
> visible of marketable features we will have for this release. Especially
> updateable views.
Josh said nearly the same in core's discussion, but come now. The
updateable-view patch (in its current form) does not offer *one single
thing* you can't do today, indeed for many PG releases past; it just
saves writing out some tedious rules. If that's the most marketable
feature in 8.2 we're already in trouble. I don't think it is anyway.
Looking at the CVS logs I see
INSERT/UPDATE/DELETE RETURNINGVALUES listsCREATE INDEX CONCURRENTLYpl/pgsql debugger (OK, not in core, but
available)restartableWAL recoveryconstraint exclusion works on UPDATE/DELETE queriesmultiple-argument aggregate
functionsSQL2003-standardstatistical aggregatesCOPY (SELECT ...) TO ...pg_dump selectivity optionscustomizable timezone
namesGINindexesFILLFACTOR for indexes and tables
and that's just back to the beginning of July, so it probably represents
about a quarter of the work that's gone into 8.2, but I got tired of
reading the logs at that point.
We have this discussion every single release cycle: "hey, if we wait X
amount more time then we can have cool features Y and Z". But there's
always another X, Y, and Z. More importantly, this view ignores the
very real disadvantages of not getting already-finished cool features
A through W out to our users sooner.
regards, tom lane
> > Josh said nearly the same in core's discussion, but come now. The > updateable-view patch (in its current form) does not offer *one single > thing* you can't do today, indeed for many PG releases past; it just > saves writing out some tedious rules. Sure, I can write the rules, you can write the rules but we can do rollup queries using unions too, but we still don't have rollup :) Perception is everything and we don't have Updateable views until we don't have to write those rules (yes, its stupid)... > If that's the most marketable > feature in 8.2 we're already in trouble. I don't think it is anyway. > Looking at the CVS logs I see > > INSERT/UPDATE/DELETE RETURNING > VALUES lists > CREATE INDEX CONCURRENTLY > constraint exclusion works on UPDATE/DELETE queries > multiple-argument aggregate functions > SQL2003-standard statistical aggregates > COPY (SELECT ...) TO ... > pg_dump selectivity options > customizable timezone names > GIN indexes > FILLFACTOR for indexes and tables Let's take multiple-argument aggregate functions for example... The marketable portion of that is limited. 85% of the people we are marketing too don't even know what that means. Heck, I haven't even run into the pervious limitation :) Versus SQL2003-standard statistical aggregates. That wins hands down just because of the SQL2003-standard part of the string :) COPY (SELECT...) TO, is only marketable to current PostgreSQL users. Those who are not users don't know the limitation and it is likely not something they would look for when evaluating PostgreSQL. For my current customers the things that are the most marketable (note current customers) is probably: * CREATE INDEX CONCURRENTLY (this is a huge in production boon) * GIN indexes (after explanation) * constraint exclusion works on UPDATE/DELETE queries (We are using alot of tp now, so this is great) * Bitmap Indexes It does not mean all those features are useful, they definitely are. I am just trying to look at it from at: WHIZ* BANG* POW* perspective. What makes the analyst (and I know this isn't the communities problem) say, "Holy crap batman, we need to be talking to more people about this database". I think those would be Updateable views, Bitmap Indexes and the SQL2003-standard statistical aggregates. Sincerely, Joshua D. Drake > > and that's just back to the beginning of July, so it probably represents > about a quarter of the work that's gone into 8.2, but I got tired of > reading the logs at that point. > > We have this discussion every single release cycle: "hey, if we wait X > amount more time then we can have cool features Y and Z". But there's > always another X, Y, and Z. More importantly, this view ignores the > very real disadvantages of not getting already-finished cool features > A through W out to our users sooner. > > regards, tom lane > -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
Josh Berkus wrote: > Tom, > > > September is upon us and it doesn't seem like we are ready to ship > > a beta. I think it's time to start making some hard choices. > > > > In the main code I see these outstanding features/patches: > > > > * bitmap indexes > > * updatable views > > * GUC variable reload + refactoring (previously applied and reverted) > > * plpython improvements (needs review by someone who knows plpython) > > * plpgsql improvements for returning record types > > * patch to build on VC > > What's VC? MS Visual C++ -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Tom Lane wrote: > > I'm all for bouncing > > stuff that's not ready to go, but bouncing stuff because we haven't > > recruited enough code reviewers is just bad practice. > > Well, it's taken us the full month to get this far through the queue, so > I'd sure like to have more people on board reviewing next time. Alvaro > helped but I miss Neil Conway, and some other people have helped in the > past. However --- the present situation has nothing to do with lack of > reviewers, it's lack of time to finish the patches. Also, for next time it would be good to have the reviewers got through the patches *before* feature freeze, so that developers get early feedback. (Which in turn means more time to finish the patches). -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Bruce, > > What's VC? > > MS Visual C++ Given that this could lead to us recruiting more developers out of our Windows user base, I'd prioritize it. Overall, I think this whole process is pointing up that there's a problem with our historic Feature Freeze+1 month|beta|RCs cycle. Given the number of developers and companies involved and the increasing size of the code, we need to develop a new schedule that allows us to get draft patches (or at least full specifications) eariler than the month before feature freeze. In short,the issue is that Feature Freeze isn't just Feature Freeze, it's "final patch submission". If you look at the two "incomplete" patches, and the "misfired" one (Bitmaps, Updatable Views, and WITH RECURSIVE) all of them are patches where the submitter had been working on them months ago, and might have made the release (or let us know they weren't on schedule) if we'd held them to an earlier deadline. As I recall, it was the same situation last year: getting large patches dropped on us out of the blue the week before feature freeze, which then didnt' make it in. Therefore I propose that we adopt the following schedule for 8.3, assuming an September 1 Beta date: June 1: Specification Freeze: specifications for all new features due July 1: Feature Freeze: Draft patches and any minor tweaks due August 1: Final (completed, mostly debugged) patches due September 1: First Beta -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco
Joshua D. Drake wrote: > It does not mean all those features are useful, they definitely are. I > am just trying to look at it from at: > > WHIZ* BANG* POW* perspective. Holy crap, Batman! This database can do INSERT INTO foo VALUES (1,1, 'so long'), (42, 2, 'and thanks'), (142857, 3, 'for all the fish') now! We should be talking to more people about that! That will make some MySQL users happy at least a bit :-) -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Tom Lane wrote: > Josh Berkus <josh@agliodbs.com> writes: > > Can you be a little clearer on which things are non-working patches, and > > which things are pending due to lack of review? > > The only things I currently want to reject as not having provided a > timely patch are bitmap indexes and updatable views. The other stuff > I mentioned is in need of review, but I don't currently have a reason > to think it can't get in. > > > I'm all for bouncing > > stuff that's not ready to go, but bouncing stuff because we haven't > > recruited enough code reviewers is just bad practice. > > Well, it's taken us the full month to get this far through the queue, so > I'd sure like to have more people on board reviewing next time. Alvaro > helped but I miss Neil Conway, and some other people have helped in the > past. However --- the present situation has nothing to do with lack of > reviewers, it's lack of time to finish the patches. I did try to get us additional help in reviewing. Neil was unavailable, and Alvaro could only give part of his time. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Josh Berkus wrote: > Bruce, > > > > What's VC? > > > > MS Visual C++ > > Given that this could lead to us recruiting more developers out of our > Windows user base, I'd prioritize it. > > Overall, I think this whole process is pointing up that there's a problem > with our historic Feature Freeze+1 month|beta|RCs cycle. Given the number > of developers and companies involved and the increasing size of the code, > we need to develop a new schedule that allows us to get draft patches (or > at least full specifications) eariler than the month before feature > freeze. In short,the issue is that Feature Freeze isn't just Feature > Freeze, it's "final patch submission". > > If you look at the two "incomplete" patches, and the "misfired" one > (Bitmaps, Updatable Views, and WITH RECURSIVE) all of them are patches > where the submitter had been working on them months ago, and might have > made the release (or let us know they weren't on schedule) if we'd held > them to an earlier deadline. As I recall, it was the same situation last > year: getting large patches dropped on us out of the blue the week before > feature freeze, which then didnt' make it in. > > Therefore I propose that we adopt the following schedule for 8.3, assuming > an September 1 Beta date: > > June 1: Specification Freeze: specifications for all new features due > July 1: Feature Freeze: Draft patches and any minor tweaks due > August 1: Final (completed, mostly debugged) patches due > September 1: First Beta No rejiggering is going to get people to complete things they didn't complete under the old system. The plan you list above is what we did for this release. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Tom Lane wrote: > Well, it's taken us the full month to get this far through the queue, > so I'd sure like to have more people on board reviewing next time. > Alvaro helped but I miss Neil Conway, and some other people have > helped in the past. However --- the present situation has nothing to > do with lack of reviewers, it's lack of time to finish the patches. Not that I've had particularly plenty of time to give, but what's concerning me is that while we're moving toward an increasingly rigid process, we don't really have the process support to go with it. How would anyone even know what patches are pending review? Something to think about ... -- Peter Eisentraut http://developer.postgresql.org/~petere/
Joshua D. Drake wrote: > Let's take multiple-argument aggregate functions for example... The > marketable portion of that is limited. 85% of the people we are > marketing too don't even know what that means. Heck, I haven't even > run into the pervious limitation :) As far as market analysis goes, I'd guess that 100% of the people who already use PostgreSQL by definition don't need any new features and would rather see incremental improvements across the board, which is what this or any release would give them. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Bruce Momjian <bruce@momjian.us> writes:
> Tom Lane wrote:
>> Well, it's taken us the full month to get this far through the queue, so
>> I'd sure like to have more people on board reviewing next time. Alvaro
>> helped but I miss Neil Conway, and some other people have helped in the
>> past. However --- the present situation has nothing to do with lack of
>> reviewers, it's lack of time to finish the patches.
> I did try to get us additional help in reviewing. Neil was unavailable,
> and Alvaro could only give part of his time.
It strikes me that setting feature freeze in midsummer might not be the
best strategy for having manpower available to review --- people tend to
be on vacation in August. Maybe the answer is just to move the dates a
bit one way or the other.
regards, tom lane
> Therefore I propose that we adopt the following schedule for 8.3, assuming > an September 1 Beta date: > > June 1: Specification Freeze: specifications for all new features due I would suggest the above be: June 1: Specification Freeze: specifications for all new features due & approved. The last thing we want it to spend all of june arguing about how something should be implemented and thus no real coding gets done. > July 1: Feature Freeze: Draft patches and any minor tweaks due > August 1: Final (completed, mostly debugged) patches due > September 1: First Beta other then that w00t! Joshua D. Drake > -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
Josh Berkus <josh@agliodbs.com> writes:
> Bruce,
>>> What's VC?
>>
>> MS Visual C++
> Given that this could lead to us recruiting more developers out of our
> Windows user base, I'd prioritize it.
Yeah, that's why I listed it as a major item. I haven't had a chance to
look at the patch, but if it's not too ugly I would like to get it in
this time rather than next --- that could lead directly to having more
people available next time.
> If you look at the two "incomplete" patches, and the "misfired" one
> (Bitmaps, Updatable Views, and WITH RECURSIVE) all of them are patches
> where the submitter had been working on them months ago, and might have
> made the release (or let us know they weren't on schedule) if we'd held
> them to an earlier deadline.
Perhaps, but I'm not sure what we can or should do about it. Moving
deadlines up will either create a dead zone where we all sit around
twiddling our thumbs, or people will keep on coding till the last minute
anyway.
I think having a few patches that don't make the deadline isn't a bad
thing: it means we didn't have people sitting idle. It's not like the
work will go to waste --- those things can still get in in the next
devel cycle.
regards, tom lane
Alvaro Herrera wrote: > Joshua D. Drake wrote: > >> It does not mean all those features are useful, they definitely are. I >> am just trying to look at it from at: >> >> WHIZ* BANG* POW* perspective. > > Holy crap, Batman! This database can do > > INSERT INTO foo VALUES (1,1, 'so long'), (42, 2, 'and thanks'), > (142857, 3, 'for all the fish') > > now! We should be talking to more people about that! > > > That will make some MySQL users happy at least a bit :-) To be fair, it will make some of our customers happy to.. they can now get rid of the weird union hack they had to do instead :) Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
On Fri, 1 Sep 2006, Tom Lane wrote: > My feeling is that we ought to bounce bitmap indexes and updatable views > as not being ready, accept all the contrib stuff, and try to get the > other items done in time for a beta at, say, the end of next week. For what it's worth, Jie and I hope to have finished the bitmap streaming this weekend which is the only outstanding issue with the code as it currently stands. I now realise (nothing like hindsight) that we should have posted a patch before we looked at streaming bitmaps because they've proved more fiddly to implement than I thought :-(. At that point, we'd more or less addressed the issues I'd raised when I posted at feature freeze time. Thanks, Gavin
Bruce, Tom, all: > No rejiggering is going to get people to complete things they didn't > complete under the old system. It'll help the new people. A lot of people -- if not most -- submitting their first major patch to PostgreSQL dramatically underestimate the amount of fix-up that's going to be required, and assume that there won't be a spec discussion, which there often is. By getting them to submit a little at a time, *earlier*, we can avoid doing those things at the last minute. Alternately, we can just make sure that first-time patchers have mentors who check progress well before feature freeze. > The plan you list above is what we did > for this release. No, it's not. There's a bunch of patches which we had nothing on -- not spec, not design draft, not anything -- until we got them on July 20th. Our current system is to have only one deadline, at which point you're expected to have 85% of the patch done and up to PostgreSQL standards. That's quite a bit of "jumping in with both feet" for a newbie. > I did try to get us additional help in reviewing. Neil was unavailable, > and Alvaro could only give part of his time Asking two people is not exactly an all-out effort to get reviewers. > It strikes me that setting feature freeze in midsummer might not be the > best strategy for having manpower available to review --- people tend to > be on vacation in August. Maybe the answer is just to move the dates a > bit one way or the other. We've discussed that issue before, yes. Since we're proposing a new roadmap process for 8.3, and will likely be dealing with a lot of major patches, maybe that's the release to delay? However, as PR maven I do want to point out that doing the final release in December would be a bad idea. Hard to get news coverage. Also, we'd have the same issue with people being gone. -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco
Gavin Sherry <swm@linuxworld.com.au> writes:
> On Fri, 1 Sep 2006, Tom Lane wrote:
>> My feeling is that we ought to bounce bitmap indexes and updatable views
>> as not being ready, accept all the contrib stuff, and try to get the
>> other items done in time for a beta at, say, the end of next week.
> For what it's worth, Jie and I hope to have finished the bitmap streaming
> this weekend which is the only outstanding issue with the code as it
> currently stands.
AFAIR, you told me "it'll be done this weekend" last week. And the week
before.
In any case, it's going to be a large complicated patch, and if I spend
all next week reviewing it then I won't make any progress on the other
items on my to-do list. At some point we have to say "this has slipped
too far, we have to hold it over for 8.3".
regards, tom lane
Tom Lane <tgl@sss.pgh.pa.us> writes: >> If you look at the two "incomplete" patches, and the "misfired" one >> (Bitmaps, Updatable Views, and WITH RECURSIVE) all of them are patches >> where the submitter had been working on them months ago, and might have >> made the release (or let us know they weren't on schedule) if we'd held >> them to an earlier deadline. > > Perhaps, but I'm not sure what we can or should do about it. Moving > deadlines up will either create a dead zone where we all sit around > twiddling our thumbs, or people will keep on coding till the last minute > anyway. I don't think that's what would happen at all. In fact what I think would happen is that it would free up people like you to work on stuff you're interested in instead of reviewing patches. As an extreme example consider the new Linux release cycle. They have a non-freeze period of a couple weeks, followed by months of frozen time. Users who want to try out new features on different hardware can do so and often turn up problems developers reviewing the patches missed. Developers spend the months of frozen time working on new patches which, if they're ready on time, all go into the queue in the first few weeks of a new release. If they miss the window they'll make the next one. As a result they don't get things like what we saw with 8.0 where major new features like PITR, nested transactions and so on all hit the tree in the final days before the freeze. Instead they go in early and if they turn out to be immature they get pulled long before the actual release. I would love to see the bitmap indexes and updatable views get merged into the tree just weeks after the release comes out rather than disappear again until just before the next release. If I were a user depending on these new features it would give me a hell of a lot more confidence if people could say they've been in the CVS version for months and nobody's had problems with them than to be told they were reviewed by one person three weeks ago and nobody else has had much of a chance to use it. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com
Josh Berkus <josh@agliodbs.com> writes:
> No, it's not. There's a bunch of patches which we had nothing on -- not
> spec, not design draft, not anything -- until we got them on July 20th.
> Our current system is to have only one deadline,
Well, no, it's not. We have told people till we're blue in the face
"post early, post often". Now I will plead guilty to not always having
spent as much time giving feedback on draft patches as I should've, but
the process is pretty clear. As I see it the main problem is people
undertaking patches off in corners somewhere rather than discussing
their work on the mailing lists while they do it.
If you want more process in this we could institute rules like "feature
patches will be rejected out of hand if there wasn't previously a spec
proposal posted to pghackers". I don't think this is really a good
idea, but I'm not sure what else to do.
regards, tom lane
Tom Lane wrote: > Well, no, it's not. We have told people till we're blue in the face > "post early, post often". Now I will plead guilty to not always > having spent as much time giving feedback on draft patches as I > should've, but the process is pretty clear. As I see it the main > problem is people undertaking patches off in corners somewhere rather > than discussing their work on the mailing lists while they do it. Again, process support. If all we can offer people is to post multi-megabyte patches to the mailing list every month, that totally doesn't help. We'd need ways to track the progress on these things: what was the specification for that patch, where was the discussion on it, what has changed in the patch since the last time, since the time before last time, what is left to be done, who has worked on it, etc. Figuring out the answer to those questions from a mailing list archive is tedious to the point that no one wants to do it. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Peter Eisentraut wrote: > Tom Lane wrote: > > Well, it's taken us the full month to get this far through the queue, > > so I'd sure like to have more people on board reviewing next time. > > Alvaro helped but I miss Neil Conway, and some other people have > > helped in the past. However --- the present situation has nothing to > > do with lack of reviewers, it's lack of time to finish the patches. > > Not that I've had particularly plenty of time to give, but what's > concerning me is that while we're moving toward an increasingly rigid > process, we don't really have the process support to go with it. How > would anyone even know what patches are pending review? Something to > think about ... They are in the patches queue? http://momjian.postgresql.org/cgi-bin/pgpatches -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Josh Berkus wrote: > Bruce, Tom, all: > > > No rejiggering is going to get people to complete things they didn't > > complete under the old system. > > It'll help the new people. A lot of people -- if not most -- submitting > their first major patch to PostgreSQL dramatically underestimate the > amount of fix-up that's going to be required, and assume that there won't > be a spec discussion, which there often is. By getting them to submit a > little at a time, *earlier*, we can avoid doing those things at the last > minute. That is in the developer's FAQ. > Alternately, we can just make sure that first-time patchers have mentors > who check progress well before feature freeze. > > > The plan you list above is what we did > > for this release. > > No, it's not. There's a bunch of patches which we had nothing on -- not > spec, not design draft, not anything -- until we got them on July 20th. > Our current system is to have only one deadline, at which point you're > expected to have 85% of the patch done and up to PostgreSQL standards. > That's quite a bit of "jumping in with both feet" for a newbie. Right. The developer's FAQ says they should follow a process. Making another process doesn't mean they will follow that either. > > > I did try to get us additional help in reviewing. Neil was unavailable, > > and Alvaro could only give part of his time > > Asking two people is not exactly an all-out effort to get reviewers. Well, not sure what else I can do. Those are the people who used to help out a lot. > > It strikes me that setting feature freeze in midsummer might not be the > > best strategy for having manpower available to review --- people tend to > > be on vacation in August. Maybe the answer is just to move the dates a > > bit one way or the other. > > We've discussed that issue before, yes. Since we're proposing a new > roadmap process for 8.3, and will likely be dealing with a lot of major > patches, maybe that's the release to delay? Moving it away from summer might help, yea. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Tom Lane wrote: > Josh Berkus <josh@agliodbs.com> writes: > > No, it's not. There's a bunch of patches which we had nothing on -- not > > spec, not design draft, not anything -- until we got them on July 20th. > > Our current system is to have only one deadline, > > Well, no, it's not. We have told people till we're blue in the face > "post early, post often". Now I will plead guilty to not always having > spent as much time giving feedback on draft patches as I should've, but > the process is pretty clear. As I see it the main problem is people > undertaking patches off in corners somewhere rather than discussing > their work on the mailing lists while they do it. > > If you want more process in this we could institute rules like "feature > patches will be rejected out of hand if there wasn't previously a spec > proposal posted to pghackers". I don't think this is really a good > idea, but I'm not sure what else to do. Well stated. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > Peter Eisentraut wrote: >> Tom Lane wrote: >>> Well, it's taken us the full month to get this far through the queue, >>> so I'd sure like to have more people on board reviewing next time. >>> Alvaro helped but I miss Neil Conway, and some other people have >>> helped in the past. However --- the present situation has nothing to >>> do with lack of reviewers, it's lack of time to finish the patches. >> Not that I've had particularly plenty of time to give, but what's >> concerning me is that while we're moving toward an increasingly rigid >> process, we don't really have the process support to go with it. How >> would anyone even know what patches are pending review? Something to >> think about ... > > They are in the patches queue? > > http://momjian.postgresql.org/cgi-bin/pgpatches Side note:: Perhaps this would be better served from the main website under the developer and community sections. Joshua D. Drake > -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
Peter Eisentraut wrote: > Tom Lane wrote: > > Well, no, it's not. We have told people till we're blue in the face > > "post early, post often". Now I will plead guilty to not always > > having spent as much time giving feedback on draft patches as I > > should've, but the process is pretty clear. As I see it the main > > problem is people undertaking patches off in corners somewhere rather > > than discussing their work on the mailing lists while they do it. > > Again, process support. If all we can offer people is to post > multi-megabyte patches to the mailing list every month, that totally > doesn't help. We'd need ways to track the progress on these things: > what was the specification for that patch, where was the discussion on > it, what has changed in the patch since the last time, since the time > before last time, what is left to be done, who has worked on it, etc. > Figuring out the answer to those questions from a mailing list archive > is tedious to the point that no one wants to do it. Uh, Tom has been tracking Gavin on the bitmap patch every week for weeks, and I pummelled EnterpriseDB/Jonah over the recursive query patch. Neither effort was very fruitful, but tracking wasn't what made them fail. I am not saying tracking is wrong, but rather tracking would not have helped make these things happen faster. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Joshua D. Drake wrote: > Bruce Momjian wrote: > > Peter Eisentraut wrote: > >> Tom Lane wrote: > >>> Well, it's taken us the full month to get this far through the queue, > >>> so I'd sure like to have more people on board reviewing next time. > >>> Alvaro helped but I miss Neil Conway, and some other people have > >>> helped in the past. However --- the present situation has nothing to > >>> do with lack of reviewers, it's lack of time to finish the patches. > >> Not that I've had particularly plenty of time to give, but what's > >> concerning me is that while we're moving toward an increasingly rigid > >> process, we don't really have the process support to go with it. How > >> would anyone even know what patches are pending review? Something to > >> think about ... > > > > They are in the patches queue? > > > > http://momjian.postgresql.org/cgi-bin/pgpatches > > Side note:: > > Perhaps this would be better served from the main website under the > developer and community sections. It pulls my a mailbox file I use, and it does instant updates as soon as I change it. It is a URL. Why do people care where it is? -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
>>> They are in the patches queue? >>> >>> http://momjian.postgresql.org/cgi-bin/pgpatches >> Side note:: >> >> Perhaps this would be better served from the main website under the >> developer and community sections. > > It pulls my a mailbox file I use, and it does instant updates as soon as > I change it. It is a URL. Why do people care where it is? I would think that it would be easier to find :) Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
Bruce Momjian wrote: > Uh, Tom has been tracking Gavin on the bitmap patch every week for > weeks, and I pummelled EnterpriseDB/Jonah over the recursive query > patch. Neither effort was very fruitful, but tracking wasn't what made > them fail. I am not saying tracking is wrong, but rather tracking would > not have helped make these things happen faster. > > It would make the process more transparent, which is something several people have expressed a desire for. cheers andrew
Gregory Stark <stark@enterprisedb.com> writes:
> As an extreme example consider the new Linux release cycle. They have a
> non-freeze period of a couple weeks, followed by months of frozen time. Users
> who want to try out new features on different hardware can do so and often
> turn up problems developers reviewing the patches missed. Developers spend the
> months of frozen time working on new patches which, if they're ready on time,
> all go into the queue in the first few weeks of a new release. If they miss
> the window they'll make the next one.
That doesn't seem particularly appealing for our purposes. What it
sounds like is that all the development happens somewhere outside the
main tree, and every so often everybody tries to land a bunch of large
patches at once. That's going to be a mess if the patches interact at
all, and it certainly isn't going to address my primary concern of
encouraging a publicly-visible development process instead of having
people hiding in a corner until they can drop a large patch on us.
> I would love to see the bitmap indexes and updatable views get merged
> into the tree just weeks after the release comes out rather than
> disappear again until just before the next release.
Here I agree - partly. With some continuing effort these patches could
be in fine shape to apply by the time we branch CVS for 8.3 development.
However, our traditional approach to beta period is that it's supposed
to be a "quiet time" where people focus on testing, debugging, and
documentation. Shepherding people who are doing major development
during that period is likely to be a serious distraction from making the
release ready and high-quality.
regards, tom lane
Bruce Momjian wrote: > It pulls my a mailbox file I use, and it does instant updates as soon > as I change it. It is a URL. Why do people care where it is? The complaint has been that not enough people help with reviewing patches. That would indicate that the problem is not location but scalability. If everything has to go through your private mailbox, then it's not a very obvious process to outsiders. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Joshua D. Drake wrote: > > >>> They are in the patches queue? > >>> > >>> http://momjian.postgresql.org/cgi-bin/pgpatches > >> Side note:: > >> > >> Perhaps this would be better served from the main website under the > >> developer and community sections. > > > > It pulls my a mailbox file I use, and it does instant updates as soon as > > I change it. It is a URL. Why do people care where it is? > > I would think that it would be easier to find :) We have a link from the developer's page: http://www.postgresql.org/developer/roadmap -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Peter Eisentraut wrote: > Bruce Momjian wrote: > > It pulls my a mailbox file I use, and it does instant updates as soon > > as I change it. It is a URL. Why do people care where it is? > > The complaint has been that not enough people help with reviewing > patches. That would indicate that the problem is not location but > scalability. If everything has to go through your private mailbox, > then it's not a very obvious process to outsiders. Well, you can grab items from there and apply them. I will remove them from the mailbox when I see the commit. You have another system that will not be significantly more work? Any you can apply them right from the email lists so they never get to the queue. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > Peter Eisentraut wrote: > > Bruce Momjian wrote: > > > It pulls my a mailbox file I use, and it does instant updates as soon > > > as I change it. It is a URL. Why do people care where it is? > > > > The complaint has been that not enough people help with reviewing > > patches. That would indicate that the problem is not location but > > scalability. If everything has to go through your private mailbox, > > then it's not a very obvious process to outsiders. > > Well, you can grab items from there and apply them. I will remove them > from the mailbox when I see the commit. You have another system that > will not be significantly more work? Any you can apply them right from > the email lists so they never get to the queue. Another novel idea: have a "bugtracker" of sorts where selected people can add "feature requests" and append patches to them. There, people could add "fake" feature requests describing the stuff they are working on, and publish the patches they have for them. -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Bruce Momjian wrote: > Uh, Tom has been tracking Gavin on the bitmap patch every week for > weeks, and I pummelled EnterpriseDB/Jonah over the recursive query > patch. Great, but where is this documented, so others know about this? > Neither effort was very fruitful, but tracking wasn't what > made them fail. I am not saying tracking is wrong, but rather > tracking would not have helped make these things happen faster. The fallacy here is assuming that all these things should be single-person tasks. As long as we only have one coder and one "manager", we don't need much process support, but then we're pretty nearly at the point we're now, where two or three people review patches while the rest just sits around and wonders what this feature freeze thing is supposed to be about. I can tell you plenty of stories about the updatable views patch. One month after feature freeze, we notice that we didn't even have an accepted design specification. I'm sure it was posted sometime, but how do we find it now? People complain unjustly that the patch was posted at the last minute, but in fact updated patches and information have been posted regularly for more than one year. But it's impossible to tie these things together unless you are mailing list crawling software with artificial intelligence capabilities. And during the last two weeks, no make that six months, Bernd has spent half his time analyzing and reverting breakage that well-meaning reviewers had injected into his patch, with the other half possibly spent keeping the patch up to date with the moving development tree. There is, of course, no silver bullet. But more successful involvement of people who are not in the inner circle needs more support in many ways. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Bruce Momjian wrote: > Well, you can grab items from there and apply them. I will remove > them from the mailbox when I see the commit. You have another > system that will not be significantly more work? Any you can apply > them right from the email lists so they never get to the queue. What the current system lacks is documentation, which makes the process behind those lists known to others, and the ability to interact with the system. If I can't see any feedback in the system for the actions I do, then the system is not very useful to me. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Alvaro Herrera wrote: > I've been bitten by having stuff rejected because there was no > immediate use for them; so I had to maintain them in a private tree, > where they were subsequently broken by someone else's parallel > development. A better version control system would surely seem to help here and in other instances. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Peter Eisentraut wrote: > Bruce Momjian wrote: > > Uh, Tom has been tracking Gavin on the bitmap patch every week for > > weeks, and I pummelled EnterpriseDB/Jonah over the recursive query > > patch. > > Great, but where is this documented, so others know about this? I see no value in documenting it. > > Neither effort was very fruitful, but tracking wasn't what > > made them fail. I am not saying tracking is wrong, but rather > > tracking would not have helped make these things happen faster. > > The fallacy here is assuming that all these things should be > single-person tasks. As long as we only have one coder and > one "manager", we don't need much process support, but then we're > pretty nearly at the point we're now, where two or three people review > patches while the rest just sits around and wonders what this feature > freeze thing is supposed to be about. > > I can tell you plenty of stories about the updatable views patch. One > month after feature freeze, we notice that we didn't even have an > accepted design specification. I'm sure it was posted sometime, but > how do we find it now? People complain unjustly that the patch was > posted at the last minute, but in fact updated patches and information > have been posted regularly for more than one year. But it's impossible > to tie these things together unless you are mailing list crawling > software with artificial intelligence capabilities. And during the > last two weeks, no make that six months, Bernd has spent half his time > analyzing and reverting breakage that well-meaning reviewers had > injected into his patch, with the other half possibly spent keeping the > patch up to date with the moving development tree. > > There is, of course, no silver bullet. But more successful involvement > of people who are not in the inner circle needs more support in many > ways. I do things only if others do not. If committers applied patches as they came in, the patch queue would be empty, and if others tracked open issues, I wouldn't have to. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Alvaro Herrera wrote: > Another novel idea: have a "bugtracker" of sorts where selected > people can add "feature requests" and append patches to them. There, > people could add "fake" feature requests describing the stuff they > are working on, and publish the patches they have for them. Certainly, much of this discussion is trying to yell "tracking software". Track patches, track comments, track open items, and so on. This doesn't have to be the same as a bug tracker, though. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Peter Eisentraut wrote: > Bruce Momjian wrote: > > Well, you can grab items from there and apply them. I will remove > > them from the mailbox when I see the commit. You have another > > system that will not be significantly more work? Any you can apply > > them right from the email lists so they never get to the queue. > > What the current system lacks is documentation, which makes the process > behind those lists known to others, and the ability to interact with > the system. If I can't see any feedback in the system for the actions > I do, then the system is not very useful to me. What do you want to happen? Just apply the patch? You want to log into a system to say you applied it? If people are not doing applications when they have no overhead, I doubt they will do it when there are additional steps. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Peter Eisentraut wrote: > Alvaro Herrera wrote: > > Another novel idea: have a "bugtracker" of sorts where selected > > people can add "feature requests" and append patches to them. There, > > people could add "fake" feature requests describing the stuff they > > are working on, and publish the patches they have for them. > > Certainly, much of this discussion is trying to yell "tracking > software". Track patches, track comments, track open items, and so on. > This doesn't have to be the same as a bug tracker, though. OK, why do we use email then? Move it all into a tracker? If not, who is going to move info from email to the tracking system? -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
On 9/1/06, Bruce Momjian <bruce@momjian.us> wrote: > I pummelled Jonah over the recursive query patch. He did. Trust me on this... think I still have some bruises too :) > Neither effort was very fruitful, but tracking wasn't what made > them fail. I am not saying tracking is wrong, but rather tracking would > not have helped make these things happen faster. This is correct. I just had too much stuff to do and there wasn't enough time to get the hierarchical query patch worthy of someone spending their time on a review. IMHO, tracking occasionally is alright. However, as a developer, being "pummelled" does sometimes get irritating; even to the point that I will sometimes discontinue working on something because it's too much of a hassle for me to spend my free-time on. Though, this was not the case for hierarchical queries at all; that really was due to a lack of time. There's got to be a "happy medium" in which we can keep our status updated without it becoming an irritation. Has anyone looked at something like dotProject? It may be something we could use for development. Of course, there's lots of other tools... but it would be nice if we had a central location for each task's status so that we don't have to resort to searching email and/or archives. -- Jonah H. Harris, Software Architect | phone: 732.331.1300 EnterpriseDB Corporation | fax: 732.331.1301 33 Wood Ave S, 2nd Floor | jharris@enterprisedb.com Iselin, New Jersey 08830 | http://www.enterprisedb.com/
Bruce Momjian wrote: > I see no value in documenting it. The value in documenting it is that the general awareness about the progress of development is raised, so more people know where and how to get involved, as opposed to only three people knowing and the rest finding out one month after feature freeze that the patch isn't ready. > I do things only if others do not. If committers applied patches as > they came in, the patch queue would be empty, and if others tracked > open issues, I wouldn't have to. Fair enough. Perhaps we should find others with ideas about better tracking and the willingness to implement them. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Peter Eisentraut <peter_e@gmx.net> writes: > Tom Lane wrote: > > Well, no, it's not. We have told people till we're blue in the face > > "post early, post often". Now I will plead guilty to not always > > having spent as much time giving feedback on draft patches as I > > should've, but the process is pretty clear. As I see it the main > > problem is people undertaking patches off in corners somewhere rather > > than discussing their work on the mailing lists while they do it. > > Again, process support. If all we can offer people is to post > multi-megabyte patches to the mailing list every month, that totally > doesn't help. You're right, it's a pain for developers to have to keep shipping around patches over and over. It means anyone else who wants to look at your code is probably looking at an old version, there's no convenient way to convert from incremental patches to cumulative patches, etc. There are tools to solve this but they're not "process support", they're revision control systems. If CVS didn't suck so hard I would suggest that the natural thing to do would be to grant CVS access far and wide and have people create branches for their proposed changes. The existing set of committers would become the set of people allowed to commit to HEAD. Sadly CVS makes this about as much work as the current system and the risks are much higher. > We'd need ways to track the progress on these things: > what was the specification for that patch, where was the discussion on > it, what has changed in the patch since the last time, since the time > before last time, what is left to be done, who has worked on it, etc. > Figuring out the answer to those questions from a mailing list archive > is tedious to the point that no one wants to do it. Frankly these things all sound pretty trivial. -- greg
Tom Lane <tgl@sss.pgh.pa.us> writes: > Gregory Stark <stark@enterprisedb.com> writes: > > As an extreme example consider the new Linux release cycle. They have a > > non-freeze period of a couple weeks, followed by months of frozen time. Users > > who want to try out new features on different hardware can do so and often > > turn up problems developers reviewing the patches missed. Developers spend the > > months of frozen time working on new patches which, if they're ready on time, > > all go into the queue in the first few weeks of a new release. If they miss > > the window they'll make the next one. > > That doesn't seem particularly appealing for our purposes. What it > sounds like is that all the development happens somewhere outside the > main tree, and every so often everybody tries to land a bunch of large > patches at once. That's going to be a mess if the patches interact at > all, and it certainly isn't going to address my primary concern of > encouraging a publicly-visible development process instead of having > people hiding in a corner until they can drop a large patch on us. Well that's basically what's been happening except they drop it on us a day before feature freeze giving you a frantic month of trying to make it work. The reason people go off on their own and develop without help are manifold but I'm not sure the release process is one of them. They're not going to be able to commit their incomplete code to the main CVS regardless of whether there's a freeze or not anyways. They can work openly on new features during the feature freeze and beta period, they just won't have the expectation that they'll squeeze it into this release. Instead they'll know that if it's ready for the patch landing period it'll be in the next release. > Here I agree - partly. With some continuing effort these patches could > be in fine shape to apply by the time we branch CVS for 8.3 development. > However, our traditional approach to beta period is that it's supposed > to be a "quiet time" where people focus on testing, debugging, and > documentation. Shepherding people who are doing major development > during that period is likely to be a serious distraction from making the > release ready and high-quality. Well on the flip side you have months to do that work instead of having pressure to rework whole patches in short order. Perhaps it helps that at least in the case of Linux it's mostly different people handling the releases and the raw development. I'm not sure we have enough manpower for that. -- greg
Andrew Dunstan wrote: > It would make the process more transparent, which is something several > people have expressed a desire for. Yes, the processes seems to work by having two of the most important people waste time on getting information anyone else could collect, or that the developer himself could quickly provide and keep up to date. Furthermore the process expects these two people to know who to ask, what to look for etc. Wouldn't it be great if someone could just decide: "hey I know postgres, I have some unexpected spare time, I want to do some code reviewing on patch x, y and z" If this all works through some issue tracker, a wiki or a combination of both, the end result is transparency, the opportunity to take load off from people that have important other things to do and a chance at getting unexpected help. For example I have no expertise in coding on Postgres, but I think I would be able to collect information from this mailinglist (like specs, url's etc.) and put them in some issue tracker or wiki. I have done exactly the same for PHP [1] (though there are rarely specs thrown around in PHP, so my PHP todo list is not much more than a simple bullet list of todo's with a name and occasional URL's to additional information). regards, Lukas [1] http://oss.backendmedia.com/PHPTODO/
Lukas Kahwe Smith wrote: > For example I have no expertise in coding on Postgres, but I think I > would be able to collect information from this mailinglist (like specs, > url's etc.) and put them in some issue tracker or wiki. I have done > exactly the same for PHP [1] (though there are rarely specs thrown > around in PHP, so my PHP todo list is not much more than a simple bullet > list of todo's with a name and occasional URL's to additional information). Actually I should add that I went ahead and created the PHP todo list on my own, without any official blessing and one by one internals developer have joined. Now its actively used in the entire release process. This is probably the best approach to go about doing this for PostgreSQL as well. That way there is no additional work, and instead it can show if it actually helps people. And if it does more people will start to contribute to it. If not then someone who did not contribute to the code anyways wasted a bit of time to get to know the PostgreSQL todo items really well. ;) regards, Lukas
On Fri, Sep 01, 2006 at 10:13:07PM -0400, Jonah H. Harris wrote: > On 9/1/06, Bruce Momjian <bruce@momjian.us> wrote: > >I pummelled Jonah over the recursive query patch. > > He did. Trust me on this... think I still have some bruises too :) That wasn't productive. Getting it out in public that your available time didn't match the size of the project in, say, February or even April, would have been. > >Neither effort was very fruitful, but tracking wasn't what made > >them fail. I am not saying tracking is wrong, but rather tracking > >would not have helped make these things happen faster. > > This is correct. I just had too much stuff to do and there wasn't > enough time to get the hierarchical query patch worthy of someone > spending their time on a review. I know it's hard to judge from the inside, which is why this process needs more public visibility, early and often, to the rest of the community. It does nobody any good to have Bruce bawl you out repeatedly in private, and then spring it on the rest of us in June that you weren't even close. > IMHO, tracking occasionally is alright. However, as a developer, > being "pummelled" does sometimes get irritating; even to the point > that I will sometimes discontinue working on something because it's > too much of a hassle for me to spend my free-time on. That's an excellent outcome for the project, by the way. When somebody is feeling "pummeled," it's frequently a signal that they really don't have the time they thought they did, and the sooner and more widely such a situation is known, the better. > Though, this was not the case for hierarchical queries at all; that > really was due to a lack of time. > > There's got to be a "happy medium" in which we can keep our status > updated without it becoming an irritation. Has anyone looked at > something like dotProject? It may be something we could use for > development. Of course, there's lots of other tools... but it would > be nice if we had a central location for each task's status so that > we don't have to resort to searching email and/or archives. ...and we're back to the infamous tarbaby of bug/issue trackers. It's time we took that head-on instead of dancing around it. As expressed many times earlier, such a system must be accessible, both for read and write, by email. What other things must it do? Should it do? Must it avoid? Cheers, D -- David Fetter <david@fetter.org> http://fetter.org/ phone: +1 415 235 3778 AIM: dfetter666 Skype: davidfetter Remember to vote!
On Fri, Sep 01, 2006 at 09:46:02PM -0400, Bruce Momjian wrote: > Peter Eisentraut wrote: > > Bruce Momjian wrote: > > > It pulls my a mailbox file I use, and it does instant updates as > > > soon as I change it. It is a URL. Why do people care where it > > > is? > > > > The complaint has been that not enough people help with reviewing > > patches. That would indicate that the problem is not location but > > scalability. If everything has to go through your private > > mailbox, then it's not a very obvious process to outsiders. > > Well, you can grab items from there and apply them. I will remove > them from the mailbox when I see the commit. This, in essence, is the problem. You're using the first person singular here, and it's become obvious that one person can't keep up with this any more. The project, thanks in part to your hard work, has grown past that stage. > You have another system that will not be significantly more work? If it's significantly more work spread among many people, that is a perfectly viable solution, just as long as it doesn't cause any one person to be overloaded. We need to get this project more bus-proof, whether it's the patch queue or a server people depend on that's silently down for a week with no ETAs in sight for getting any piece of it back on line. Bruce, are you ready to help create a more bus-proof situation? If so, what specific steps would you like to take in that direction? Cheers, D -- David Fetter <david@fetter.org> http://fetter.org/ phone: +1 415 235 3778 AIM: dfetter666 Skype: davidfetter Remember to vote!
On Sat, Sep 02, 2006 at 09:00:35AM +0200, Lukas Kahwe Smith wrote: > Actually I should add that I went ahead and created the PHP todo list on > my own, without any official blessing and one by one internals developer > have joined. Now its actively used in the entire release process. That is the approach we should be taking. I see too many people wait for an "official" solution when in fact the first person to come up with a todo list that people like better than the current one will win. I remember something about setting up a wiki for a todo list and pie in the sky list. I thought it had promise, but until the wiki is there we won't know... Have a nice day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > From each according to his ability. To each according to his ability to litigate.
Martijn van Oosterhout wrote: > I remember something about setting up a wiki for a todo list and pie > in the sky list. I thought it had promise, but until the wiki is > there we won't know... I think the wiki is the prerequisite for many ideas about alternative tracking and documentation mechanisms. I just wonder what the hold up is on it. -- Peter Eisentraut http://developer.postgresql.org/~petere/
On Saturday 02 September 2006 07:14, David Fetter wrote:
> On Fri, Sep 01, 2006 at 09:46:02PM -0400, Bruce Momjian wrote:
> > Peter Eisentraut wrote:
> > > Bruce Momjian wrote:
> > > > It pulls my a mailbox file I use, and it does instant updates as
> > > > soon as I change it. It is a URL. Why do people care where it
> > > > is?
> > >
> > > The complaint has been that not enough people help with reviewing
> > > patches. That would indicate that the problem is not location but
> > > scalability. If everything has to go through your private
> > > mailbox, then it's not a very obvious process to outsiders.
> >
> > Well, you can grab items from there and apply them. I will remove
> > them from the mailbox when I see the commit.
>
> This, in essence, is the problem. You're using the first person
> singular here, and it's become obvious that one person can't keep up
> with this any more. The project, thanks in part to your hard work,
> has grown past that stage.
>
AFAICT Bruce is not the bottleneck and our problem is not that multiple people
are reviewing the same item and duplicating effort, so I think your making
conclusions based not on the evidence but instead on a desired outcome.
No offense, a whole lot of this thread seems to be positioned that way, but
the real problem seems to be we do not have enough patch reviewers. ISTM the
questions we should be asking are who can actually help out with patch review
and then ask those people why they haven't done it. If folks like Peter,
Andrew, Magnus, Simon, Joe, and Niel all say that they are not reviewing
patches because they can't find the patches that need review, they can't
figure out who is reviewing what, or they don't think anyone is paying
attention when they do review something, then I think we have a serious
problem and we certainly need to change processes. What I think you'll find
is that they are all just busy working on other things, which in that case I
think we need to figure out how to motivate them to focus on the patch queue
rather than other items. IMHO
--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL
Lukas Kahwe Smith <smith@pooteeweet.org> writes:
>> For example I have no expertise in coding on Postgres, but I think I
>> would be able to collect information from this mailinglist (like specs,
>> url's etc.) and put them in some issue tracker or wiki. I have done
>> exactly the same for PHP [1] (though there are rarely specs thrown
>> around in PHP, so my PHP todo list is not much more than a simple bullet
>> list of todo's with a name and occasional URL's to additional information).
> Actually I should add that I went ahead and created the PHP todo list on
> my own, without any official blessing and one by one internals developer
> have joined. Now its actively used in the entire release process.
> This is probably the best approach to go about doing this for PostgreSQL
> as well.
I agree. Look at the most successful recent process change around here:
the buildfarm. Andrew Dunstan took it upon himself to make that happen.
He built it, and they came.
No bug/issue tracker, or anything else, is going to be successful unless
somebody commits enough time to make it so. I've noted a whole lot of
enthusiasm for having a tracker in these recent discussions, but a
remarkable shortage of individuals stepping up to do the work.
regards, tom lane
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tom Lane sagely noted: > No bug/issue tracker, or anything else, is going to be successful unless > somebody commits enough time to make it so. I've noted a whole lot of > enthusiasm for having a tracker in these recent discussions, but a > remarkable shortage of individuals stepping up to do the work. There are a number of reasons for this, not least of which is the enormous and ever-changing requirements such a system would have to have. My company has even offered resources to get something going, and I think I convinced Bruce at one point of the usefulness of a PG bug/feature tracking system. However, the problem is that existing software is never going to be good enough. The buildfarm is an excellent example of this. If Andrew had tried to create it with something "off-the-shelf", it probably would have taken longer to get started, been more complicated than it needed to be, and would not work as well as it now does. However, it's been a smashing success, so much so that MySQL is now envious[1] and is trying to make their own version of it[2]. [1] http://www.jpipes.com/index.php?/archives/93-MySQL-Build-Farm-Part-1.html [2] http://forge.mysql.com/wiki/MySQL_Build_Farm_Initiative I've been thinking about this a lot since before the Summit, and the only solution I see is to design something specifically for us. Rather than get bogged down in details about how it will work and what technologies it will be using, I'd like to share my ideas on how it will work from an end-user perspective. There's a simple web interface. You can use it to get the answer to questions like this: What are all the open bugs for the current release? Who is working on the updateable views patch? Which bugs were fixed in 8.1.3? What's the feature history of psql since 2003? What are some easy TODO items that need getting done? Just when does Tom Lane sleep? What features were added in 8.1? What are the current unapplied patches? What bugs were fixed in tsearch2 from version 7.4.5 to 7.4.12? Yes, maintaining it will be a royal pain in the butt. But my theory has been "if you build it, they will come". It will require a lot of human interaction, as automation only takes you so far, especially when trying to parse mailing list messages. But if we eventually get a decent-size group of people who each put in a little work on it, it should be quite possible to keep it up to date. (Also, this would be a great way for people to help the project who don't have the time and/or skills to do coding). The idea is to allow everything to happen as it already does, e.g. no extra burdens for Tom, no extra processes anywhere. However, all the existing information is gathered into one centralized and searchable place. The primary sources will include the code, the docs, and the mailing lists. Especially the mailing lists. So, rather than go off in a corner and start coding, how about I start a pgfoundry project for this and document my existing ideas? Any objections to giving this a try? - -- Greg Sabino Mullane greg@endpoint.com greg@turnstep.com End Point Corporation 610-983-9073 PGP Key: 0x14964AC8 200609021107 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -----BEGIN PGP SIGNATURE----- iD8DBQFE+Z/1vJuQZxSWSsgRAu7rAKCJcIOiw4jty0slWuEd6fNsCV8xcgCfQpEj MBp00ZxG9B6UVAoHnSDq0W0= =zlQm -----END PGP SIGNATURE-----
Alvaro Herrera <alvherre@commandprompt.com> writes:
> Here's a completely novel idea: accept incremental patches.
I don't think it's as novel as all that --- personally I've always
preferred to tackle large projects incrementally.
> I've been bitten by having stuff rejected because there was no immediate
> use for them;
Maybe you failed to make the larger picture clear. If you'd gotten the
pghackers list to agree to some plan like "I want to accomplish X, and
to get there I want to do A then B then C", I doubt we'd have rejected a
subsequent patch to do A (assuming it doesn't break anything else, of
course).
regards, tom lane
On Sep 2, 2006, at 11:28 AM, Tom Lane wrote: > Alvaro Herrera <alvherre@commandprompt.com> writes: >> Here's a completely novel idea: accept incremental patches. > > I don't think it's as novel as all that --- personally I've always > preferred to tackle large projects incrementally. I think that accepting incremental patches to a mainline is _bad_ and that accepting them in general is good (as you have to go through that process outside of version control sometimes anyway). Sticking them in CVS however can get a bit messy. This is where other version control systems that have a bit better branching and merging support has an advantage as people can work in the repository on their project in separate branches and pulling them all back together again (once an overall satisfaction metric has been reached) is not excruciatingly painful. Where am I going with this? From my experience accepting incremental patches is a _bad_ idea unless you have a VCS that has really makes it _easy_ to manage them. Sounds like more work than worth on the postgres project as it is now. Additionally, what problem is accepting incremental patches supposed to solve? // Theo Schlossnagle // CTO -- http://www.omniti.com/~jesus/ // OmniTI Computer Consulting, Inc. -- http://www.omniti.com/
"Greg Sabino Mullane" <greg@turnstep.com> writes:
> I've been thinking about this a lot since before the Summit, and the
> only solution I see is to design something specifically for us.
Well, nobody's going to accuse you of thinking too small ;-). Sounds
great to me, though, if you think you can pull it off. Have at it!
> Rather than get bogged down in details about how it will work and
> what technologies it will be using, I'd like to share my ideas on
> how it will work from an end-user perspective. There's a simple web
> interface. You can use it to get the answer to questions like this:
> What are all the open bugs for the current release?
> Who is working on the updateable views patch?
> Which bugs were fixed in 8.1.3?
> [etc]
That sounds fine for the "output" part of things, but where does the
"input" come from? ISTM that it's been the input side that's been the
real bone of contention in all the discussions of specific tracking
software.
BTW, another "output" thing you might consider is "having draft release
notes ready-to-go on demand". Currently, Bruce prepares the release
notes on the basis of a very tedious scan of the CVS commit logs.
If this sort of stuff were being dropped into a tracker as it went
into the CVS tree, at least the research part of making the notes would
be zero-effort (or perhaps better to say that the work would be spread
out instead of concentrated).
regards, tom lane
On Saturday 02 September 2006 11:42, Tom Lane wrote:
> BTW, another "output" thing you might consider is "having draft release
> notes ready-to-go on demand". Currently, Bruce prepares the release
> notes on the basis of a very tedious scan of the CVS commit logs.
> If this sort of stuff were being dropped into a tracker as it went
> into the CVS tree, at least the research part of making the notes would
> be zero-effort (or perhaps better to say that the work would be spread
> out instead of concentrated).
FWIW I have never understood why we don't require patch submitters/committers
to update the release notes when they do the patch. I know Bruce says that
it's not a problem for him to do it, but it sure would help out some of the
other folks.
--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL
Theo Schlossnagle <jesus@omniti.com> writes:
> Additionally, what problem is accepting incremental patches supposed
> to solve?
Keeping the individual patches reviewable is one useful goal.
We may be talking at cross-purposes here. The sort of thing I think
Alvaro is imagining is something like what I did a year or two back when
I wanted to make the executor treat plan trees as read-only --- if
memory serves, I did that in three or four commits spread over a week or
two. I could have done it in one massive patch but it would have been
unreadable to me or anyone else. And for that matter, the reason for
doing it at all was as part of a much larger concept that's still
unfinished (better caching and invalidation of plans). When you're
talking about tasks of that magnitude, stepwise improvement is the only
way you are going to get there at all.
I really don't have any faith in the concept of doing very large tasks
on a development branch and expecting to land them in one huge merge.
For a nearby counterexample look at Postgres-R, which never has and
probably never will manage to produce a patch that could apply to the
core project's CVS head. At least not without thinking of some
incremental way to do it, because by the time they bring all their
changes up to HEAD, the tree has always drifted under them. There is
no magic version control system that will fix that.
regards, tom lane
Tom Lane wrote: > "Greg Sabino Mullane" <greg@turnstep.com> writes: > > I've been thinking about this a lot since before the Summit, and the > > only solution I see is to design something specifically for us. > > Well, nobody's going to accuse you of thinking too small ;-). Sounds > great to me, though, if you think you can pull it off. Have at it! > > > Rather than get bogged down in details about how it will work and > > what technologies it will be using, I'd like to share my ideas on > > how it will work from an end-user perspective. There's a simple web > > interface. You can use it to get the answer to questions like this: > > > What are all the open bugs for the current release? > > Who is working on the updateable views patch? > > Which bugs were fixed in 8.1.3? > > [etc] > > That sounds fine for the "output" part of things, but where does the > "input" come from? ISTM that it's been the input side that's been the > real bone of contention in all the discussions of specific tracking > software. > > BTW, another "output" thing you might consider is "having draft release > notes ready-to-go on demand". Currently, Bruce prepares the release > notes on the basis of a very tedious scan of the CVS commit logs. Having an intermediate release note list, beyond the dashed TODO items, might be helpful. So the idea is to track work in progress, and completed items. The question is always whether the effort is worth the benefit. We can try and see how it works. > If this sort of stuff were being dropped into a tracker as it went > into the CVS tree, at least the research part of making the notes would > be zero-effort (or perhaps better to say that the work would be spread > out instead of concentrated). Well, multiple commits often end up as a single entry (and many commits don't need to show up in the release notes), so the tracker would just give me a backup when creating the release notes. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Robert Treat wrote: > On Saturday 02 September 2006 11:42, Tom Lane wrote: > > BTW, another "output" thing you might consider is "having draft release > > notes ready-to-go on demand". Currently, Bruce prepares the release > > notes on the basis of a very tedious scan of the CVS commit logs. > > If this sort of stuff were being dropped into a tracker as it went > > into the CVS tree, at least the research part of making the notes would > > be zero-effort (or perhaps better to say that the work would be spread > > out instead of concentrated). > > FWIW I have never understood why we don't require patch submitters/committers > to update the release notes when they do the patch. I know Bruce says that > it's not a problem for him to do it, but it sure would help out some of the > other folks. I can't even get documentation for many patches. I am hesitant to add even more burden. I would prefer they concentrate on documentation. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
On Friday 01 September 2006 19:42, Tom Lane wrote:
> Gavin Sherry <swm@linuxworld.com.au> writes:
> > On Fri, 1 Sep 2006, Tom Lane wrote:
> >> My feeling is that we ought to bounce bitmap indexes and updatable views
> >> as not being ready, accept all the contrib stuff, and try to get the
> >> other items done in time for a beta at, say, the end of next week.
> >
> > For what it's worth, Jie and I hope to have finished the bitmap streaming
> > this weekend which is the only outstanding issue with the code as it
> > currently stands.
>
> AFAIR, you told me "it'll be done this weekend" last week. And the week
> before.
>
> In any case, it's going to be a large complicated patch, and if I spend
> all next week reviewing it then I won't make any progress on the other
> items on my to-do list. At some point we have to say "this has slipped
> too far, we have to hold it over for 8.3".
>
Perhaps if some other committers would take ownership of other items on
your "to be done list", that would spread the workload enough to keep
everything going? (Yes, this does require some of the other folks to step
up... responses to your original email with "I will commit to finishing this
item this week" would probably be enough)
--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL
Tom Lane wrote: > Theo Schlossnagle <jesus@omniti.com> writes: >> Additionally, what problem is accepting incremental patches supposed >> to solve? > > Keeping the individual patches reviewable is one useful goal. > > We may be talking at cross-purposes here. The sort of thing I think > Alvaro is imagining is something like what I did a year or two back when > I wanted to make the executor treat plan trees as read-only --- if > memory serves, I did that in three or four commits spread over a week or > two. To second this, Alvaro is constantly beating our (cmd) other developers to do this, so I would guess that you are correct :). I find also that this method allows someone like me, who can read C and understand good parts of it to get the gist of what is going on without trying to grok the whole thing. Large patches make it very difficult. Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
Greg Sabino Mullane wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > Tom Lane sagely noted: >> No bug/issue tracker, or anything else, is going to be successful unless >> somebody commits enough time to make it so. I've noted a whole lot of >> enthusiasm for having a tracker in these recent discussions, but a >> remarkable shortage of individuals stepping up to do the work. Although I applaud Greg's efforts, I think it makes more sense to take something and make it our own instead of trying to build something from scratch. I think that Bugzilla will probably serve our purpose for most things. Greg, how about you and I (I am talking with Dunstan on this as well) set up Bugzilla and just doing an small group audit of how it could (or couldn't) work and report back? Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
Robert Treat wrote: > On Saturday 02 September 2006 07:14, David Fetter wrote: > > On Fri, Sep 01, 2006 at 09:46:02PM -0400, Bruce Momjian wrote: > > > Peter Eisentraut wrote: > > > > Bruce Momjian wrote: > > > > > It pulls my a mailbox file I use, and it does instant updates as > > > > > soon as I change it. It is a URL. Why do people care where it > > > > > is? > > > > > > > > The complaint has been that not enough people help with reviewing > > > > patches. That would indicate that the problem is not location but > > > > scalability. If everything has to go through your private > > > > mailbox, then it's not a very obvious process to outsiders. > > > > > > Well, you can grab items from there and apply them. I will remove > > > them from the mailbox when I see the commit. > > > > This, in essence, is the problem. You're using the first person > > singular here, and it's become obvious that one person can't keep up > > with this any more. The project, thanks in part to your hard work, > > has grown past that stage. > > > > AFAICT Bruce is not the bottleneck and our problem is not that multiple people > are reviewing the same item and duplicating effort, so I think your making > conclusions based not on the evidence but instead on a desired outcome. > > No offense, a whole lot of this thread seems to be positioned that way, but > the real problem seems to be we do not have enough patch reviewers. ISTM the > questions we should be asking are who can actually help out with patch review Let me explain what I do. I subscribe to most PostgreSQL email lists. I basically try to make sure every bug, patch, or idea is processed. If it is a bug via user error, did they get correct information. If it is valid PostgreSQL bug, was it fixed or added to the TODO list. If it was a patch, was it applied or rejected. If it is a feature request, was it added to the TODO list or rejected. That's about it, folks. Not a lot of fancy magic. As other people take up these items, my workload will be reduced. I do not take ownership of any of these things. I merely jump on them when they are _not_ dealt with by others. What I am unclear about is how all of this traffic is going to be pushed into a system that cleanly categorizes it so others can understand it. It can be done, but what is the cost/benefit to it? Perhaps if email could be pushed into the system, and somehow categorized as bug, patch, or feature request, and somehow removed only when it is dealt with. That is what I do now via my mailbox. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Robert Treat wrote: > No offense, a whole lot of this thread seems to be positioned that way, but > the real problem seems to be we do not have enough patch reviewers. ISTM the > questions we should be asking are who can actually help out with patch review > and then ask those people why they haven't done it. If folks like Peter, > Andrew, Magnus, Simon, Joe, and Niel all say that they are not reviewing > patches because they can't find the patches that need review, they can't > figure out who is reviewing what, or they don't think anyone is paying > attention when they do review something, then I think we have a serious > problem and we certainly need to change processes. What I think you'll find > is that they are all just busy working on other things, which in that case I > think we need to figure out how to motivate them to focus on the patch queue > rather than other items. IMHO I think that if all the patches are listed with all relevant context information on a webpage, then people can more easily jump in when they unexpectedly have time (or prefer to procrastinate some other real world thing they should rather work on). Right now if you have a few hours to spare you do not have all the information readily available. Even worse by inquiring for the information you might feel you are commiting more than you really wanted to. This kind of information needs to be right there without any person having to actively provide it upon inquiry. regards, Lukas
Peter Eisentraut wrote: > Martijn van Oosterhout wrote: >> I remember something about setting up a wiki for a todo list and pie >> in the sky list. I thought it had promise, but until the wiki is >> there we won't know... > > I think the wiki is the prerequisite for many ideas about alternative > tracking and documentation mechanisms. I just wonder what the hold up > is on it. We have a wiki already: http://wiki.postgresql.org/ It could be a bit faster, but its there. I even already started on a little documentation effort: http://wiki.postgresql.org/index.php/pgwiki:Variations regards, Lukas
Greg Sabino Mullane wrote: > Yes, maintaining it will be a royal pain in the butt. But my theory has > been "if you build it, they will come". It will require a lot of human > interaction, as automation only takes you so far, especially when trying > to parse mailing list messages. But if we eventually get a decent-size > group of people who each put in a little work on it, it should be > quite possible to keep it up to date. (Also, this would be a great way > for people to help the project who don't have the time and/or skills > to do coding). I think we should start with the wiki and then add in whatever we notice can be automated. This way we may not initially be able to cover all the bases, but it will mean that whatever we do automate actually matches real world requirements. > The idea is to allow everything to happen as it already does, e.g. no > extra burdens for Tom, no extra processes anywhere. However, all the > existing information is gathered into one centralized and searchable place. > The primary sources will include the code, the docs, and the mailing lists. > Especially the mailing lists. Yeah, let the ones work on this that do not do any of the pgsql coding. regards, Lukas
Greg Sabino Mullane wrote: > There are a number of reasons for this, not least of which is the > enormous and ever-changing requirements such a system would have to > have. > The buildfarm is an excellent example of this. The build farm is not an example of this. There isn't any build-farm software out there. There is, however, plenty of issue tracking software out there. Just pointing that out. Have at it. I for one would rather set up existing issue trackers and work with them. If I could just find hardware to host it. Not that I've looked, though. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Lukas Kahwe Smith wrote: > Robert Treat wrote: > > > No offense, a whole lot of this thread seems to be positioned that way, but > > the real problem seems to be we do not have enough patch reviewers. ISTM the > > questions we should be asking are who can actually help out with patch review > > and then ask those people why they haven't done it. If folks like Peter, > > Andrew, Magnus, Simon, Joe, and Niel all say that they are not reviewing > > patches because they can't find the patches that need review, they can't > > figure out who is reviewing what, or they don't think anyone is paying > > attention when they do review something, then I think we have a serious > > problem and we certainly need to change processes. What I think you'll find > > is that they are all just busy working on other things, which in that case I > > think we need to figure out how to motivate them to focus on the patch queue > > rather than other items. IMHO > > I think that if all the patches are listed with all relevant context > information on a webpage, then people can more easily jump in when they > unexpectedly have time (or prefer to procrastinate some other real world > thing they should rather work on). Right now if you have a few hours to > spare you do not have all the information readily available. Even worse > by inquiring for the information you might feel you are commiting more > than you really wanted to. This kind of information needs to be right > there without any person having to actively provide it upon inquiry. OK, how does that happen without a lot of work, or moving all discussion on to a web-based system? -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Greg Sabino Mullane wrote: > I've been thinking about this a lot since before the Summit, and the > only solution I see is to design something specifically for us. > Rather than get bogged down in details about how it will work and > what technologies it will be using, I'd like to share my ideas on > how it will work from an end-user perspective. There's a simple web > interface. You can use it to get the answer to questions like this: > > What are all the open bugs for the current release? > Who is working on the updateable views patch? > Which bugs were fixed in 8.1.3? > What's the feature history of psql since 2003? > What are some easy TODO items that need getting done? > Just when does Tom Lane sleep? > What features were added in 8.1? > What are the current unapplied patches? > What bugs were fixed in tsearch2 from version 7.4.5 to 7.4.12? > > Yes, maintaining it will be a royal pain in the butt. But my theory has > been "if you build it, they will come". It will require a lot of human > interaction, as automation only takes you so far, especially when trying > to parse mailing list messages. But if we eventually get a decent-size > group of people who each put in a little work on it, it should be > quite possible to keep it up to date. (Also, this would be a great way > for people to help the project who don't have the time and/or skills > to do coding). > > The idea is to allow everything to happen as it already does, e.g. no > extra burdens for Tom, no extra processes anywhere. However, all the > existing information is gathered into one centralized and searchable place. > The primary sources will include the code, the docs, and the mailing lists. > Especially the mailing lists. > > So, rather than go off in a corner and start coding, how about I start a > pgfoundry project for this and document my existing ideas? Any objections > to giving this a try? Good approach. Let me tell you what doesn't work well currently: What is fixed in current CVS that was not on the TODO list already, and marked with a dash?What fixes/bugs are being workedon but not yet in the patch queue?What are all the features/fixes in each release? Right now, the release notes show a list of all the significant items in each release, but it isn't available until the release, and it isn't complete (because it would be unreadable by ordinary users). And there is no tracking of individual items in progress except by individual developers. Magnus, for example, tracks Win32 items, and Tom tracks backend stuff. I track whatever no one else tracks. As far as tracking, right now, +50% of items come in and are discussed via email, not a bug form. How does that information get into a tracking system without huge extra work? We can do it, and it would give people nice feedback, but is the effort worth the benefit? -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > Holy crap, Batman! This database can do > > INSERT INTO foo VALUES (1,1, 'so long'), (42, 2, 'and thanks'), > (142857, 3, 'for all the fish') > > now! We should be talking to more people about that! > > That will make some MySQL users happy at least a bit :-) It will certainly make porting applications from MySQL to Postgres a lot easier, and such porting is where a lot more effort needs to be put, IMO. (I mean more people working on porting applications, not adding mysqlisms to the code :) - -- Greg Sabino Mullane greg@turnstep.com End Point Corporation PGP Key: 0x14964AC8 200609021200 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -----BEGIN PGP SIGNATURE----- iD8DBQFE+as8vJuQZxSWSsgRAq9RAJwJzY9uTBmMjafX/96tyRgviN7TQQCaAt2S bgXGiusM1R7RD8FoZsiYizo= =3GI5 -----END PGP SIGNATURE-----
Lukas Kahwe Smith wrote: > Peter Eisentraut wrote: >> Martijn van Oosterhout wrote: >>> I remember something about setting up a wiki for a todo list and pie >>> in the sky list. I thought it had promise, but until the wiki is >>> there we won't know... >> >> I think the wiki is the prerequisite for many ideas about alternative >> tracking and documentation mechanisms. I just wonder what the hold up >> is on it. > > We have a wiki already: > http://wiki.postgresql.org/ > > It could be a bit faster, but its there. That wiki is wrong. :) It was set up wrong and configured wrong. It was supposed to be for developers only. There is also another wiki that is a trac based that was set up at dave pages request (for slaves_to_www). > > I even already started on a little documentation effort: > http://wiki.postgresql.org/index.php/pgwiki:Variations > And note it has already been mentioned that, that wiki was the wrong place to put it. Not that I am slamming your efforts, I think it was really good that you did the docs :). They are just in the wrong place. Sincerely, Joshua D. Drake > regards, > Lukas > > > ---------------------------(end of broadcast)--------------------------- > TIP 9: In versions below 8.0, the planner will ignore your desire to > choose an index scan if your joining column's datatypes do not > match > -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
Joshua D. Drake wrote: > That wiki is wrong. :) It was set up wrong and configured wrong. It was > supposed to be for developers only. > > There is also another wiki that is a trac based that was set up at dave > pages request (for slaves_to_www). Setup something better, until then we can start using it to generate content. >> I even already started on a little documentation effort: >> http://wiki.postgresql.org/index.php/pgwiki:Variations >> > > And note it has already been mentioned that, that wiki was the wrong > place to put it. Not that I am slamming your efforts, I think it was > really good that you did the docs :). They are just in the wrong place. No problem. Its easy to move content, its much harder to generate it. regards, Lukas
Lukas Kahwe Smith wrote: > Bruce Momjian wrote: > > Lukas Kahwe Smith wrote: > >> Robert Treat wrote: > >> > >>> No offense, a whole lot of this thread seems to be positioned that way, but > >>> the real problem seems to be we do not have enough patch reviewers. ISTM the > >>> questions we should be asking are who can actually help out with patch review > >>> and then ask those people why they haven't done it. If folks like Peter, > >>> Andrew, Magnus, Simon, Joe, and Niel all say that they are not reviewing > >>> patches because they can't find the patches that need review, they can't > >>> figure out who is reviewing what, or they don't think anyone is paying > >>> attention when they do review something, then I think we have a serious > >>> problem and we certainly need to change processes. What I think you'll find > >>> is that they are all just busy working on other things, which in that case I > >>> think we need to figure out how to motivate them to focus on the patch queue > >>> rather than other items. IMHO > >> I think that if all the patches are listed with all relevant context > >> information on a webpage, then people can more easily jump in when they > >> unexpectedly have time (or prefer to procrastinate some other real world > >> thing they should rather work on). Right now if you have a few hours to > >> spare you do not have all the information readily available. Even worse > >> by inquiring for the information you might feel you are commiting more > >> than you really wanted to. This kind of information needs to be right > >> there without any person having to actively provide it upon inquiry. > > > > OK, how does that happen without a lot of work, or moving all discussion > > on to a web-based system? > > No discussion takes place on the lists and IRC and busy bees make sure > that the progress/decisions etc get updated on the static wiki. Wiki's > suck at discussions, but they are great to store the decisions made so > that anyone can get himself upto speed and things do not get lost over time. > > It seems that you have been the only busy bee so far, and we definitely > need more for this to work. Yea, I was afraid that was the answer. :-( -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > Lukas Kahwe Smith wrote: >> Robert Treat wrote: >> >>> No offense, a whole lot of this thread seems to be positioned that way, but >>> the real problem seems to be we do not have enough patch reviewers. ISTM the >>> questions we should be asking are who can actually help out with patch review >>> and then ask those people why they haven't done it. If folks like Peter, >>> Andrew, Magnus, Simon, Joe, and Niel all say that they are not reviewing >>> patches because they can't find the patches that need review, they can't >>> figure out who is reviewing what, or they don't think anyone is paying >>> attention when they do review something, then I think we have a serious >>> problem and we certainly need to change processes. What I think you'll find >>> is that they are all just busy working on other things, which in that case I >>> think we need to figure out how to motivate them to focus on the patch queue >>> rather than other items. IMHO >> I think that if all the patches are listed with all relevant context >> information on a webpage, then people can more easily jump in when they >> unexpectedly have time (or prefer to procrastinate some other real world >> thing they should rather work on). Right now if you have a few hours to >> spare you do not have all the information readily available. Even worse >> by inquiring for the information you might feel you are commiting more >> than you really wanted to. This kind of information needs to be right >> there without any person having to actively provide it upon inquiry. > > OK, how does that happen without a lot of work, or moving all discussion > on to a web-based system? No discussion takes place on the lists and IRC and busy bees make sure that the progress/decisions etc get updated on the static wiki. Wiki's suck at discussions, but they are great to store the decisions made so that anyone can get himself upto speed and things do not get lost over time. It seems that you have been the only busy bee so far, and we definitely need more for this to work. regards, Lukas
On Sat, Sep 02, 2006 at 06:18:13PM +0200, Lukas Kahwe Smith wrote: > We have a wiki already: > http://wiki.postgresql.org/ I must have missed the annoucement, oh well... Now I'm only familiar with twiki so maybe this sounds silly but: Does it support sections? Like can you have a developer section, a patches section, a TODO section, etc. And then have differing permissions per section? So I was thinking a TODO section with the list as front page and each tem having a page within that section. Create a parallel page Wannabe-todo-tiems that can be edited by anyone and lock the main page to just a few. Then all some developer has to do is scan the wannabe list and copy the links to the main list. Is this kind of setup ossible? Have a nice day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > From each according to his ability. To each according to his ability to litigate.
Bruce Momjian wrote: >> It seems that you have been the only busy bee so far, and we definitely >> need more for this to work. > > Yea, I was afraid that was the answer. :-( But we have a few volunteers, like me for example. Now don't say "I was afraid that was the answer" again or I might feel insulted :) regards, Lukas
On 2/9/06 16:42, "Tom Lane" <tgl@sss.pgh.pa.us> wrote: > BTW, another "output" thing you might consider is "having draft release > notes ready-to-go on demand". Currently, Bruce prepares the release > notes on the basis of a very tedious scan of the CVS commit logs. > If this sort of stuff were being dropped into a tracker as it went > into the CVS tree, at least the research part of making the notes would > be zero-effort (or perhaps better to say that the work would be spread > out instead of concentrated). We have the developers update the CHANGELOG file with each non-trivial change in pgAdmin. It works very well for us and producing release announcements etc. is a trivial task. The overhead on each developer is virtually zero as well as they have to compose some text for the commit message anyway. http://www.pgadmin.org/development/changelog.php Regards, Dave.
On 2/9/06 17:18, "Lukas Kahwe Smith" <smith@pooteeweet.org> wrote: > Peter Eisentraut wrote: >> Martijn van Oosterhout wrote: >>> I remember something about setting up a wiki for a todo list and pie >>> in the sky list. I thought it had promise, but until the wiki is >>> there we won't know... >> >> I think the wiki is the prerequisite for many ideas about alternative >> tracking and documentation mechanisms. I just wonder what the hold up >> is on it. > > We have a wiki already: > http://wiki.postgresql.org/ > > It could be a bit faster, but its there. > > I even already started on a little documentation effort: > http://wiki.postgresql.org/index.php/pgwiki:Variations The wiki will be going shortly as it's not been setup in the way that was agreed, nor is it being used for it's intended purpose. I'll put it right when I get time from dealing with releases of everything I seem to be involved in. Regards, Dave.
Martijn van Oosterhout wrote: > On Sat, Sep 02, 2006 at 06:18:13PM +0200, Lukas Kahwe Smith wrote: >> We have a wiki already: >> http://wiki.postgresql.org/ > > I must have missed the annoucement, oh well... > > Now I'm only familiar with twiki so maybe this sounds silly but: wiki.postgresql.org is dead. :). It wasn't supposed to be an open wiki like that. Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
On Sat, Sep 02, 2006 at 06:38:30PM +0100, Dave Page wrote: > The wiki will be going shortly as it's not been setup in the way that was > agreed, nor is it being used for it's intended purpose. I'll put it right > when I get time from dealing with releases of everything I seem to be > involved in. I beleive the idea was to not make the site editable by the public? Does't that kind of defeat the purpose of getting more people to takeover work? Have a nice day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > From each according to his ability. To each according to his ability to litigate.
On 2/9/06 20:16, "Martijn van Oosterhout" <kleptog@svana.org> wrote: > On Sat, Sep 02, 2006 at 06:38:30PM +0100, Dave Page wrote: >> The wiki will be going shortly as it's not been setup in the way that was >> agreed, nor is it being used for it's intended purpose. I'll put it right >> when I get time from dealing with releases of everything I seem to be >> involved in. > > I beleive the idea was to not make the site editable by the public? > Does't that kind of defeat the purpose of getting more people to > takeover work? That was one of the more flexible points as far as I'm concerned. What was more important was to make it clear it was a *developer* resource and not an alternative techdocs site or in fact a site intended in any way for end users. See my next email for more... Regards, Dave.
Peter Eisentraut wrote: > Greg Sabino Mullane wrote: > >> There are a number of reasons for this, not least of which is the >> enormous and ever-changing requirements such a system would have to >> have. >> > > >> The buildfarm is an excellent example of this. >> > > The build farm is not an example of this. There isn't any build-farm > software out there. There is, however, plenty of issue tracking > software out there. > > Just pointing that out. Have at it. I for one would rather set up > existing issue trackers and work with them. If I could just find > hardware to host it. Not that I've looked, though. > > What is more, a good bug tracking system is orders of magnitude more complex that the buildfarm. If we want something sooner rather than later, let's use something off the shelf and tailor it. cheers andrew
Andrew Dunstan wrote: > > > Peter Eisentraut wrote: >> Greg Sabino Mullane wrote: >> >>> There are a number of reasons for this, not least of which is the >>> enormous and ever-changing requirements such a system would have to >>> have. >>> >> >> >>> The buildfarm is an excellent example of this. >>> >> >> The build farm is not an example of this. There isn't any build-farm >> software out there. There is, however, plenty of issue tracking >> software out there. >> >> Just pointing that out. Have at it. I for one would rather set up >> existing issue trackers and work with them. If I could just find >> hardware to host it. Not that I've looked, though. >> >> > > What is more, a good bug tracking system is orders of magnitude more > complex that the buildfarm. > > If we want something sooner rather than later, let's use something off > the shelf and tailor it. http://pgbugs.commandprompt.com (still need to configure email). Joshua D. Drake > > cheers > > andrew > > ---------------------------(end of broadcast)--------------------------- > TIP 5: don't forget to increase your free space map settings > -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
Bruce Momjian <bruce@momjian.us> writes:
> ... The GUC comment/default patch had tons of
> emails, but no other committers got involved to review or commit the
> patch. Peter, who knows GUC well, looked at it, but said he didn't
> review it enough.
Peter has made it pretty clear that he didn't care for the
refactorization aspect of that patch.
> I just spent 1/2 hour fixing the multi-value UPDATE
> patch for the code drift caused by UPDATE/RETURNING. The patch is a
> simple grammar macro. Any coder could have taken that, reviewed it, and
> applied it, but no one did.
Perhaps that's because nobody but you wanted it to go in.
Some amount of the issue here is that people won't work on patches they
don't approve of; that's certainly the case for me. I have more than
enough to do working on patches I do think should go in, and I get tired
of having to repeatedly object to the same bad patch. Do you remember
Sturgeon's Law? It applies to patches too.
regards, tom lane
Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > ... The GUC comment/default patch had tons of > > emails, but no other committers got involved to review or commit the > > patch. Peter, who knows GUC well, looked at it, but said he didn't > > review it enough. > > Peter has made it pretty clear that he didn't care for the > refactorization aspect of that patch. Peter asked why it was done, a good answer was given, and Peter did not reply. > > I just spent 1/2 hour fixing the multi-value UPDATE > > patch for the code drift caused by UPDATE/RETURNING. The patch is a > > simple grammar macro. Any coder could have taken that, reviewed it, and > > applied it, but no one did. > > Perhaps that's because nobody but you wanted it to go in. We got tons of people who wanted that. > Some amount of the issue here is that people won't work on patches they > don't approve of; that's certainly the case for me. I have more than > enough to do working on patches I do think should go in, and I get tired > of having to repeatedly object to the same bad patch. Do you remember > Sturgeon's Law? It applies to patches too. Sure, you have to want the patch to be in to be motivated to work on it. I think I am more willing to work with imperfection. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
>>> I just spent 1/2 hour fixing the multi-value UPDATE >>> patch for the code drift caused by UPDATE/RETURNING. The patch is a >>> simple grammar macro. Any coder could have taken that, reviewed it, and >>> applied it, but no one did. >> Perhaps that's because nobody but you wanted it to go in. > > We got tons of people who wanted that. To further this, I didn't even know it was an issue. If it was only half an hour and it needed to be done, why wasn't it put out there? Thanks goes to Bruce for fixing it but I didn't know it was an issue, I have 5 C developers, if any of them could have done it -- then some communication is needed and I would have allocated someone to it. I am sure that is the case with others as well. I am not saying it is anyone's responsibility to speak up, but I can say had I known some help was needed (especially something so trivial) I would have gladly donated some time. Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
Bruce Momjian <bruce@momjian.us> writes:
> Tom Lane wrote:
>> Peter has made it pretty clear that he didn't care for the
>> refactorization aspect of that patch.
> Peter asked why it was done, a good answer was given, and Peter did not
> reply.
Au contraire, he's reiterated since then that he didn't like it.
>> Perhaps that's because nobody but you wanted it to go in.
> We got tons of people who wanted that.
But no committers, else it would've got done.
There was some remark upthread about reviewers getting discouraged
because their comments seem to fall on deaf ears. I know exactly
what is meant by that. I'm getting tired of arguing with you about
bad patches, because it's obvious that you put no weight on my
objections --- and it looks like Peter has got the same problem.
How is it that you are willing to apply submissions from newbie
developers over the objections of core developers?
regards, tom lane
Bruce Momjian wrote: > Why aren't more people involved? Is this such a thankless task? I > am starting to think so. I can tell you exactly what my problem is. The tools are terrible. In projects with a useful issue tracking system I can take small chunks out of my day to contribute to specific items that I feel interested in or competent about. With PostgreSQL, you need to read all the mailing lists all the time to even know where things are, let alone do anything about them. Moreover, it doesn't help that you need to have a high-speed internet connection or a superfast disk to do any effective work with the version control system. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Joshua D. Drake wrote: > http://pgbugs.commandprompt.com (still need to configure email). Thank you for that. I think an issue tracking system for patches and such may need to be distinct from a bug-tracking system such as bugzilla, but let's get one thing after another up. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Peter Eisentraut wrote: > Joshua D. Drake wrote: >> http://pgbugs.commandprompt.com (still need to configure email). > > Thank you for that. > > I think an issue tracking system for patches and such may need to be > distinct from a bug-tracking system such as bugzilla, but let's get one > thing after another up. I made a mistake on this install and installed the dev version. I will fix this weekend. I was also thinking the domain: tracker.postgresql.org? Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
Joshua D. Drake wrote: > I was also thinking the domain: tracker.postgresql.org? Certainly not for bugzilla. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Tom Lane wrote: > Lukas Kahwe Smith <smith@pooteeweet.org> writes: > >>> For example I have no expertise in coding on Postgres, but I think I >>> would be able to collect information from this mailinglist (like specs, >>> url's etc.) and put them in some issue tracker or wiki. I have done >>> exactly the same for PHP [1] (though there are rarely specs thrown >>> around in PHP, so my PHP todo list is not much more than a simple bullet >>> list of todo's with a name and occasional URL's to additional information). >>> > > >> Actually I should add that I went ahead and created the PHP todo list on >> my own, without any official blessing and one by one internals developer >> have joined. Now its actively used in the entire release process. >> > > >> This is probably the best approach to go about doing this for PostgreSQL >> as well. >> > > I agree. Look at the most successful recent process change around here: > the buildfarm. Andrew Dunstan took it upon himself to make that happen. > He built it, and they came. > :-) The difference is that the buildfarm could get going with effort only from me and a handful of early adopters, while a tracker probably needs higher level of initial community buyin. Nevertheless, I take your point. > No bug/issue tracker, or anything else, is going to be successful unless > somebody commits enough time to make it so. I've noted a whole lot of > enthusiasm for having a tracker in these recent discussions, but a > remarkable shortage of individuals stepping up to do the work. > > > You are right that it will need ongoing effort. There are discussions happening about resources. Stay tuned. cheers andrew
Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > Tom Lane wrote: > >> Peter has made it pretty clear that he didn't care for the > >> refactorization aspect of that patch. > > > Peter asked why it was done, a good answer was given, and Peter did not > > reply. > > Au contraire, he's reiterated since then that he didn't like it. The thread order was: patch, Peter comments, submitter gives reasons, patch put in the queue, Peter comments again, I reply that the change is not just "refactoring" but is needed based on submitters comments, and no reply from Peter: http://archives.postgresql.org/pgsql-patches/2006-08/msg00334.php Without a reply from Peter, I have to assume the patch is valid. > >> Perhaps that's because nobody but you wanted it to go in. > > > We got tons of people who wanted that. > > But no committers, else it would've got done. True that it was of more interest to non-committers than committers, probably because it is the committers who are going to have to remove it when full SQL functionality is added. :-} > There was some remark upthread about reviewers getting discouraged > because their comments seem to fall on deaf ears. I know exactly > what is meant by that. I'm getting tired of arguing with you about > bad patches, because it's obvious that you put no weight on my > objections --- and it looks like Peter has got the same problem. > How is it that you are willing to apply submissions from newbie > developers over the objections of core developers? Well, as I see it, core developers have no special weight in votes, except by their ability to influence the community by their posted opinions. If we want core people to have a larger weight in making decisions, I think the community should decide that. I have never heard that stated by anyone in this group. I think the UPDATE ... SET (val, ...) patch is illustrative. Some committers didn't want it because it didn't add new functionality, and because it has to be remove later when full functionality is added. But a lot of users wanted it because it implemented part of the spec, and because it will make porting easier. Based on how many users wanted it, and because the patch is very localized (gram.y), I thought it should be in 8.2. The Win32 port is an extreme case of this dichotomy --- headache for developers, boon for the user community. We all started as newbie patch submitters (some better than others, of course). I think as leaders if we don't like a patch for some reason, we have to give feedback on how the submitter can fix it, or community-agreed reasons why the patch will never be accepted. Another example is the FETCH/MOVE int64 patch. If there is a performance concern, should we just ask the submitters to do testing. You mentioned there is a way to do MOVE >2gig already, but did not supply the method. I can't figure out what it is, and I am sure others don't know either. If we can do it, that should be stated so we can document it and all understand why FETCH/MOVE int64 isn't a good addition. Basically, the people who have been around longer have to spend time to educate patch submitters to improve. If we don't, we don't grow our pool of experienced developers. The fundamental problem is if someone objects to a patch, and then someone reasonably replies to that objection, lack of a reply means to me that the last reply was valid. I realize this places a burden on people to discuss patches, but I see no better way to do it. I don't think we want to go down the road where we have individuals who object to patches, but are not required to engage in discussion about the patch. I think many dysfunctional open source projects have this problem. On a personal note, Tom, I do place great weight on your opinions, and try to adjust things to make sure you, the submitter, and everyone else is satisified with the result. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Joshua D. Drake wrote: > > >>> I just spent 1/2 hour fixing the multi-value UPDATE > >>> patch for the code drift caused by UPDATE/RETURNING. The patch is a > >>> simple grammar macro. Any coder could have taken that, reviewed it, and > >>> applied it, but no one did. > >> Perhaps that's because nobody but you wanted it to go in. > > > > We got tons of people who wanted that. > > To further this, I didn't even know it was an issue. If it was only half > an hour and it needed to be done, why wasn't it put out there? I did, sort of: http://archives.postgresql.org/pgsql-hackers/2006-08/msg00869.php The author replied to me privately, and I said I would adjust it. > Thanks goes to Bruce for fixing it but I didn't know it was an issue, I > have 5 C developers, if any of them could have done it -- then some > communication is needed and I would have allocated someone to it. > > I am sure that is the case with others as well. > > I am not saying it is anyone's responsibility to speak up, but I can say > had I known some help was needed (especially something so trivial) I > would have gladly donated some time. The patch also required a little adjustment, as I posted, so it wasn't a trival issue you could have just thrown at someone. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
bruce wrote: > Tom Lane wrote: > > Bruce Momjian <bruce@momjian.us> writes: > > > Tom Lane wrote: > > >> Peter has made it pretty clear that he didn't care for the > > >> refactorization aspect of that patch. > > > > > Peter asked why it was done, a good answer was given, and Peter did not > > > reply. > > > > Au contraire, he's reiterated since then that he didn't like it. > > The thread order was: patch, Peter comments, submitter gives reasons, > patch put in the queue, Peter comments again, I reply that the change is > not just "refactoring" but is needed based on submitters comments, and > no reply from Peter: > > http://archives.postgresql.org/pgsql-patches/2006-08/msg00334.php > > Without a reply from Peter, I have to assume the patch is valid. This is also an interesting example for a tracker. If we had one, all discussion on the patch would be in one place, but I am thinking that would require all posting to happen in a browser, or somehow have emails tagged to attach to each item. Is that something that can happen easily? I don't know. Would the repost of a patch be attached to the original submission? -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > Without a reply from Peter, I have to assume the patch is valid. To make it more explicit: I think the patch is stupid, but if someone wants to review it, go ahead. But I am not comfortable with the "if no one objects, I'll just commit it" mode that is sometimes going on. Has anyone actually tested the patch? -- Peter Eisentraut http://developer.postgresql.org/~petere/
On Sun, Sep 03, 2006 at 03:40:40PM -0400, Bruce Momjian wrote: > This is also an interesting example for a tracker. If we had one, all > discussion on the patch would be in one place, but I am thinking that > would require all posting to happen in a browser, or somehow have emails > tagged to attach to each item. Is that something that can happen > easily? I don't know. Would the repost of a patch be attached to the > original submission? In debbugs for example, the bug gets an email address <bugnumber>@bugs.project.site. Anything sent to that address gets recorded in the bug. So as long as people hit "reply-to-all" the whole thread will be archived in the bug. I assume other bug trackers have a similar feature... Have a nice day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > From each according to his ability. To each according to his ability to litigate.
Martijn van Oosterhout <kleptog@svana.org> writes: > I assume other bug trackers have a similar feature... Sadly no. That's precisely why I was pushing debbugs so hard earlier. The weakest of them will send a content-free email saying *something* happened to your issue and you have to click a link to find out what. Bugzilla is one step above that: it includes the latest comment though it doesn't thread comments and it can't handle replies. RT is intended to be email based but it requires a fair amount of work to get the transparent behaviour you want. Generally they make you click a link the email and fill in your comments in a text widget in a browser. That might be acceptable (not to me personally but...) when you're talking about requests made in some structured environment where requests are supposed to go through specific workflow. It'll never fly if you want it to track all the development discussions. I don't see old school unix hackers used to discussing things in email switching over to some web based interface to replace development mailing lists. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com
Gregory Stark wrote: > Martijn van Oosterhout <kleptog@svana.org> writes: > > >> I assume other bug trackers have a similar feature... >> > > Sadly no. That's precisely why I was pushing debbugs so hard earlier. > > The weakest of them will send a content-free email saying *something* happened > to your issue and you have to click a link to find out what. Bugzilla is one > step above that: it includes the latest comment though it doesn't thread > comments and it can't handle replies. RT is intended to be email based but it > requires a fair amount of work to get the transparent behaviour you want. > > Generally they make you click a link the email and fill in your comments in a > text widget in a browser. That might be acceptable (not to me personally > but...) when you're talking about requests made in some structured environment > where requests are supposed to go through specific workflow. It'll never fly > if you want it to track all the development discussions. I don't see old > school unix hackers used to discussing things in email switching over to some > web based interface to replace development mailing lists. > > Bz has a contrib module which I believe is supposed to handle replies. I have not used it and don't know how good it is, but there is nothing to say we can't make it work if we want to. Maybe you need to take a new look at bz, instead of retailing out of date info. As for remarks about old school unix hackers not liking web interfaces, I think Tom's recent remarks relating to the 21st century were more than apposite. cheers andrew
Andrew Dunstan <andrew@dunslane.net> writes: > As for remarks about old school unix hackers not liking web interfaces, I > think Tom's recent remarks relating to the 21st century were more than > apposite. I like web interfaces well enough for the things they're good at. Replacing e-mail is not one of them. -- greg
Andrew Dunstan <andrew@dunslane.net> writes:
> As for remarks about old school unix hackers not liking web interfaces,
> I think Tom's recent remarks relating to the 21st century were more than
> apposite.
I don't see a big problem with using a web interface to search for
information --- they're good at that. If you're thinking that the
hackers community is willing to abandon email as a discussion medium
for a web forum, however, you're going to be sadly disappointed ... and
I don't see how it would help anyway: a pile of text is a pile of text,
no matter where it came from.
The hard part of this problem is finding a convenient way to capture
status data out of the community's conversations. I think when you find
a solution to that, you'll notice that email is not the problem.
regards, tom lane
Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: >> As for remarks about old school unix hackers not liking web interfaces, >> I think Tom's recent remarks relating to the 21st century were more than >> apposite. > > I don't see a big problem with using a web interface to search for > information --- they're good at that. If you're thinking that the > hackers community is willing to abandon email as a discussion medium > for a web forum, however, you're going to be sadly disappointed ... and > I don't see how it would help anyway: a pile of text is a pile of text, > no matter where it came from. I don't think anyone would expect that (at least I wouldn't). Heck, even I wouldn't use a web forum. Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: > >> As for remarks about old school unix hackers not liking web interfaces, >> I think Tom's recent remarks relating to the 21st century were more than >> apposite. >> > > I don't see a big problem with using a web interface to search for > information --- they're good at that. If you're thinking that the > hackers community is willing to abandon email as a discussion medium > for a web forum, however, you're going to be sadly disappointed ... and > I don't see how it would help anyway: a pile of text is a pile of text, > no matter where it came from. > > The hard part of this problem is finding a convenient way to capture > status data out of the community's conversations. I think when you find > a solution to that, you'll notice that email is not the problem. > > > You have put your finger on the central problem. Email is a wonderful way of conducting an ongoing conversation and a horrid way of recording it (mail archives and search engines notwithstanding). I live by email, but I am also (sometimes painfully) aware of its limitations. Anyway, the larger point in my email was that we might well be able to have an email conversational mode with bugzilla, if indeed it doesn't already exist. cheers andrew
Peter Eisentraut wrote: > Bruce Momjian wrote: > > Without a reply from Peter, I have to assume the patch is valid. > > To make it more explicit: I think the patch is stupid, but if someone > wants to review it, go ahead. But I am not comfortable with the "if no > one objects, I'll just commit it" mode that is sometimes going on. Has > anyone actually tested the patch? Are you saying you don't like the patch, or the goal of resetting the value to their defaults when they are commented out? If the first, do you have a suggestion, if the later, it is something confuses many users. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Martijn van Oosterhout wrote: -- Start of PGP signed section. > On Sun, Sep 03, 2006 at 03:40:40PM -0400, Bruce Momjian wrote: > > This is also an interesting example for a tracker. If we had one, all > > discussion on the patch would be in one place, but I am thinking that > > would require all posting to happen in a browser, or somehow have emails > > tagged to attach to each item. Is that something that can happen > > easily? I don't know. Would the repost of a patch be attached to the > > original submission? > > In debbugs for example, the bug gets an email address > <bugnumber>@bugs.project.site. Anything sent to that address gets > recorded in the bug. So as long as people hit "reply-to-all" the whole > thread will be archived in the bug. > > I assume other bug trackers have a similar feature... Oh, so the bug is tracked by being part of the email reply list. That is a good idea. Now, how does that get assigned for non-bugs, like patches? Does any email sent to the lists that doesn't already have a bug number get one? That might be really valuable. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Gregory Stark wrote: > > Martijn van Oosterhout <kleptog@svana.org> writes: > > > I assume other bug trackers have a similar feature... > > Sadly no. That's precisely why I was pushing debbugs so hard earlier. Oh. That is bad. > The weakest of them will send a content-free email saying *something* happened > to your issue and you have to click a link to find out what. Bugzilla is one > step above that: it includes the latest comment though it doesn't thread > comments and it can't handle replies. RT is intended to be email based but it > requires a fair amount of work to get the transparent behaviour you want. > > Generally they make you click a link the email and fill in your comments in a > text widget in a browser. That might be acceptable (not to me personally > but...) when you're talking about requests made in some structured environment > where requests are supposed to go through specific workflow. It'll never fly > if you want it to track all the development discussions. I don't see old > school unix hackers used to discussing things in email switching over to some > web based interface to replace development mailing lists. Double-ouch. No threads, reply has to be via web browser. I can see that really killing communication. That bugs email address seemed very interesting. Could we create something ourselves, and have emails that don't have a bug id that are sent to the list assigned on? One problem I have now is that threads are not tracked across months, and if a new email is started, it doesn't attached to the old threads. The bug-id might not fix that, but perhaps it would. Of course one easier fix would be to get our threads to track across months, but it still doesn't handle cases where new email threads are started that apply to earlier threads. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: > > As for remarks about old school unix hackers not liking web interfaces, > > I think Tom's recent remarks relating to the 21st century were more than > > apposite. > > I don't see a big problem with using a web interface to search for > information --- they're good at that. If you're thinking that the > hackers community is willing to abandon email as a discussion medium > for a web forum, however, you're going to be sadly disappointed ... and > I don't see how it would help anyway: a pile of text is a pile of text, > no matter where it came from. > > The hard part of this problem is finding a convenient way to capture > status data out of the community's conversations. I think when you find > a solution to that, you'll notice that email is not the problem. And somehow binding threads of email together into bundles. That is also something we can't do now. The only clean fix seems to require manual work to attach them, and I doubt that is every going to happen reliably. And our email threads wander around quite a bit, with patches, ideas, and bugs sometimes all thrown in --- see the interval multiplication/division thread as a good example. How do you capture that? -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian <bruce@momjian.us> writes:
> Oh, so the bug is tracked by being part of the email reply list. That
> is a good idea. Now, how does that get assigned for non-bugs, like
> patches? Does any email sent to the lists that doesn't already have a
> bug number get one? That might be really valuable.
*Any* email? I hope not ... otherwise it's hard to see what you've got
that you don't get with a full-text search on a mailing list archive.
AFAICS the bottom line here is that we need some intelligent filtering.
In the short run I doubt that we can have that except through human
gruntwork to filter the mail traffic and update a tracker database.
Maybe after we see such a system in operation for awhile, we can start
to automate some obvious bits. But if we start with the assumption that
it's going to be mostly automated on day zero, I predict a resounding
failure.
It strikes me that the CERT CVE database might be a useful analogy.
AFAIK there is little or no automated entry into that database ---
every change has a human reviewer in front. Of course, they have some
darn good security reasons for wanting strong filters in front of their
database, but still it's a case worth thinking about. They have the
same problem of pulling status information from a lot of not-very-
well-standardized input sources.
regards, tom lane
Bruce Momjian wrote: > Oh, so the bug is tracked by being part of the email reply list. That > is a good idea. Now, how does that get assigned for non-bugs, like > patches? Does any email sent to the lists that doesn't already have a > bug number get one? That might be really valuable. > > It would just clog the bug tracker and make it unusable. I spent months on a working party on these and similar issues a few years back, in a commercial setting. One of the big issues is when you start tracking something. Bugs are a pretty simple case. Features are much harder to handle. Someone comes up with an idea. There is a lot of discussion. a consensus is arrived at to go forward. I think that's the point at which we start tracking, but it's a judgement call. What is we decide not to go ahead? Do we capture that in the tracker (with a resolution of "rejected")? One useful idea, not implemented by any of the OSS tracking / SCM systems that I know of, is to tie the two together, i.e every commit relates to a tracker item which captures the major discussions surrounding it. (If there isn't a bug, it prompts you to start one, which you can immediately mark as fixed if necessary). That way you can go backwards and forwards between the code and the discussion that led to it easily. Trust me, once you have worked with such a system you appreciate the benefits. Anyway, let me reiterate: bz has at least the start of an email reply facility, and making that work if it doesn't already should not be beyond us. cheers andrew
Andrew Dunstan <andrew@dunslane.net> writes:
> Anyway, let me reiterate: bz has at least the start of an email reply
> facility, and making that work if it doesn't already should not be
> beyond us.
I agree that starting from scratch seems like a good way to waste a
lot of time. I don't have a preference for BZ over debbugs or the
others that have been mentioned, but I'd suggest starting with something
that exists and trying to adapt it. If you end up throwing it away and
starting over, well, you'll have spent a lot less time on the trial
version than if you insist on coding from scratch.
regards, tom lane
Tom Lane wrote: > > AFAICS the bottom line here is that we need some intelligent filtering. > In the short run I doubt that we can have that except through human > gruntwork to filter the mail traffic and update a tracker database. > Maybe after we see such a system in operation for awhile, we can start > to automate some obvious bits. But if we start with the assumption that > it's going to be mostly automated on day zero, I predict a resounding > failure. > > > +1 cheers andrew
Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > Oh, so the bug is tracked by being part of the email reply list. That > > is a good idea. Now, how does that get assigned for non-bugs, like > > patches? Does any email sent to the lists that doesn't already have a > > bug number get one? That might be really valuable. > > *Any* email? I hope not ... otherwise it's hard to see what you've got > that you don't get with a full-text search on a mailing list archive. Yes, I was thinking any email. +50% of features/bugs/patches don't come in via the bug form. If we don't capture everything, will it be useful? > AFAICS the bottom line here is that we need some intelligent filtering. > In the short run I doubt that we can have that except through human > gruntwork to filter the mail traffic and update a tracker database. > Maybe after we see such a system in operation for awhile, we can start > to automate some obvious bits. But if we start with the assumption that > it's going to be mostly automated on day zero, I predict a resounding > failure. > > It strikes me that the CERT CVE database might be a useful analogy. > AFAIK there is little or no automated entry into that database --- > every change has a human reviewer in front. Of course, they have some > darn good security reasons for wanting strong filters in front of their > database, but still it's a case worth thinking about. They have the > same problem of pulling status information from a lot of not-very- > well-standardized input sources. Oh, lots of grunt work. I can see that working, but at a high cost. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > > Oh, lots of grunt work. I can see that working, but at a high cost. > > I doubt it. Let's just start with bugs, since that's the easy case anyway. Our real volume is pretty low, so the cost of maintaining it should not be high. I am assuming we would not be including HEAD, but only stable branches. With HEAD the volume would be quite a bit higher, but not impossibly so. cheers andrew
Andrew Dunstan wrote: > > > Bruce Momjian wrote: >> >> Oh, lots of grunt work. I can see that working, but at a high cost. >> >> > > I doubt it. Let's just start with bugs, since that's the easy case > anyway. Our real volume is pretty low, so the cost of maintaining it > should not be high. I am assuming we would not be including HEAD, but > only stable branches. With HEAD the volume would be quite a bit higher, > but not impossibly so. The email interface to bugzilla has potential. I *think* with a nominal amount of work, we should be able to modify it, so that a person could reply to the bug and it would correct associate as a bug. Joshua D. Drake > > cheers > > andrew > > ---------------------------(end of broadcast)--------------------------- > TIP 9: In versions below 8.0, the planner will ignore your desire to > choose an index scan if your joining column's datatypes do not > match > -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
Andrew Dunstan <andrew@dunslane.net> writes:
> Bruce Momjian wrote:
>> Oh, lots of grunt work. I can see that working, but at a high cost.
> I doubt it. Let's just start with bugs, since that's the easy case
> anyway. Our real volume is pretty low, so the cost of maintaining it
> should not be high. I am assuming we would not be including HEAD, but
> only stable branches. With HEAD the volume would be quite a bit higher,
> but not impossibly so.
Maybe we're working from different assumptions, but I thought the entire
basis for this discussion is that the project has grown to the point
where we have more resources than we used to --- and in particular,
we can find people who don't feel able to fix deep backend bugs, but are
ready and willing to track bug details and status with a goodly amount
of cluefulness. If that resource doesn't actually exist, then I fear
this whole discussion will come to naught. If it does exist, we should
call upon it.
This is not that far different from the premise upon which you built
the buildfarm: that there were people out there able to provide machine
resources and a certain amount of admin time. The resources this
project requires are not those exactly, but why shouldn't we expect
that some people will answer the call?
regards, tom lane
> This is not that far different from the premise upon which you built > the buildfarm: that there were people out there able to provide machine > resources and a certain amount of admin time. The resources this > project requires are not those exactly, but why shouldn't we expect > that some people will answer the call? I concur, more then one person has complained on this thread and others about wanting these tools. It is time that those people step up. Sincerely, Joshua D. Drake > > regards, tom lane > -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
Tom Lane wrote: > AFAICS the bottom line here is that we need some intelligent filtering. > In the short run I doubt that we can have that except through human > gruntwork to filter the mail traffic and update a tracker database. > Maybe after we see such a system in operation for awhile, we can start > to automate some obvious bits. But if we start with the assumption that > it's going to be mostly automated on day zero, I predict a resounding > failure. I agree 100%. Lets start off with grunt workers doing their magic in parallel with whatever systems we currently have. They will one by one figure out what to automate, what cannot be automated, and maybe provide value that is promising enough for people to slightly modify their modus operanti for those aspects that cannot be automated. However there will probably always be a great deal of grunt work. Again Email's are great for discussions and I think its great to link up discussions with a bug or issue tracker id. However Email discussions also often go in circles, are side tracked by IRC discussions etc. So its really hard to figure out what decisions have been made if you look things up later on. So the task of the grunt workers is to make sure that there is a summary of the relevant information available, even if all they do is flag the important decision emails. regards, Lukas
Andrew Dunstan wrote: > I spent months on a working party on these and similar issues a few > years back, in a commercial setting. One of the big issues is when you > start tracking something. Bugs are a pretty simple case. Features are > much harder to handle. Someone comes up with an idea. There is a lot of > discussion. a consensus is arrived at to go forward. I think that's the > point at which we start tracking, but it's a judgement call. What is we > decide not to go ahead? Do we capture that in the tracker (with a > resolution of "rejected")? Exactly its a judgement call. The idea would be to try and pick up each of the proposals in the discussion, summarize them in the issue tracker or via a link to the wiki. This way people do not argue in circles all that much (hopefully) and there is something to vote on. More importantly there is something to point to if the topic ever comes up again and the previous discussion did not lead to a decision. The classic "read the archives" is just very suboptimal. regards, Lukas
> Right now, the release notes show a list of all the significant > items in each release, but it isn't available until the release, > and it isn't complete (because it would be unreadable by ordinary > users). And there is no tracking of individual items in progress > except by individual developers. Magnus, for example, tracks Win32 > items, and Tom tracks backend stuff. I track whatever no one else > tracks. Well, I *try*. I find myself not managing quite as well as I'd like ;-) So I for one would definitely appreciate a tool that would help with that - provided that the tool fits the development model, not the other way around. (most other things seems to have been said already this time around, so I won't repeat arguments others have made) //Magnus
Am Montag, 4. September 2006 04:10 schrieb Bruce Momjian: > Are you saying you don't like the patch, That's it. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Am Montag, 4. September 2006 04:19 schrieb Bruce Momjian: > And our email threads wander around quite a bit, with patches, ideas, > and bugs sometimes all thrown in --- see the interval > multiplication/division thread as a good example. How do you capture > that? It's easy: Those who put in the care to capture all that have a better chance of getting their issue resolved. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: > >> Bruce Momjian wrote: >> >>> Oh, lots of grunt work. I can see that working, but at a high cost. >>> > > >> I doubt it. Let's just start with bugs, since that's the easy case >> anyway. Our real volume is pretty low, so the cost of maintaining it >> should not be high. I am assuming we would not be including HEAD, but >> only stable branches. With HEAD the volume would be quite a bit higher, >> but not impossibly so. >> > > Maybe we're working from different assumptions, but I thought the entire > basis for this discussion is that the project has grown to the point > where we have more resources than we used to --- and in particular, > we can find people who don't feel able to fix deep backend bugs, but are > ready and willing to track bug details and status with a goodly amount > of cluefulness. If that resource doesn't actually exist, then I fear > this whole discussion will come to naught. If it does exist, we should > call upon it. > > This is not that far different from the premise upon which you built > the buildfarm: that there were people out there able to provide machine > resources and a certain amount of admin time. The resources this > project requires are not those exactly, but why shouldn't we expect > that some people will answer the call? > > > I am not saying there is no work involved. In fact, throughout this thread I have agreed with you that a tracker that is not given regular maintenance effort is doomed to fail. But I don't think the level of effort required is undoable, nor that the cost is too high. Unlike running buildfarm, this is not largely automatable. Incidentally, the buildfarm has taken far more of my time and energy than I originally expected. It has probably been worth it - I doubt I could have delivered anything else as valuable in its place, but in that sense the cost to me has been quite high. I hope that by spreading the load a bit maintenance of a tracker will not be as burdensome to anybody. cheers andrew
Peter Eisentraut <peter_e@gmx.net> writes:
> Bruce Momjian wrote:
>> Without a reply from Peter, I have to assume the patch is valid.
> To make it more explicit: I think the patch is stupid, but if someone
> wants to review it, go ahead. But I am not comfortable with the "if no
> one objects, I'll just commit it" mode that is sometimes going on. Has
> anyone actually tested the patch?
Perhaps more to the point: a refactorization patch is all about beauty
in the eye of the beholder. If Peter, the original author of the guc
code, thinks that it's a disimprovement, I think it's a hard argument
to make that the patch should go in anyway.
regards, tom lane
Tom Lane wrote: > Peter Eisentraut <peter_e@gmx.net> writes: > > Bruce Momjian wrote: > >> Without a reply from Peter, I have to assume the patch is valid. > > > To make it more explicit: I think the patch is stupid, but if someone > > wants to review it, go ahead. But I am not comfortable with the "if no > > one objects, I'll just commit it" mode that is sometimes going on. Has > > anyone actually tested the patch? > > Perhaps more to the point: a refactorization patch is all about beauty > in the eye of the beholder. If Peter, the original author of the guc > code, thinks that it's a disimprovement, I think it's a hard argument > to make that the patch should go in anyway. How many times do I have to say this: IT IS NOT A REFACTOR PATCH AS REPORTED BY THE AUTHOR, AND PETER HAS NOT REFUTED THAT. It fixes a bug reported by the author, and Peter's inability to reply to the comments the author made is exactly the behavior I am talking about. If Peter does not want to engage in a technical discussion about the patch, I don't think we can consider his opinion valid. Seems I will have to call for a vote on this patch. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > How many times do I have to say this: IT IS NOT A REFACTOR PATCH AS > REPORTED BY THE AUTHOR, AND PETER HAS NOT REFUTED THAT. The initial patch was the feature plus some code refactoring included. That was what the author said. I asked him to submit the refactoring and the feature as two separate patches. What I got was a refactoring subpatch that actually made the code longer in terms of lines, which must be the very first code refactoring ever to achieve that. I did not get a satisfying answer on why that has to be, so I sort of lost interest in working with that patch. That does not mean that the patch is bad, and I certainly support the feature change. But I can't efficiently review the patch. If someone else wants to do it, go ahead. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Peter Eisentraut wrote: > Bruce Momjian wrote: > > How many times do I have to say this: IT IS NOT A REFACTOR PATCH AS > > REPORTED BY THE AUTHOR, AND PETER HAS NOT REFUTED THAT. > > The initial patch was the feature plus some code refactoring included. > That was what the author said. I asked him to submit the refactoring > and the feature as two separate patches. What I got was a refactoring > subpatch that actually made the code longer in terms of lines, which > must be the very first code refactoring ever to achieve that. I did > not get a satisfying answer on why that has to be, so I sort of lost > interest in working with that patch. Sure, thanks. Here is his reply from the patch author as to why the patch isn't just refactoring: http://archives.postgresql.org/pgsql-patches/2006-08/msg00103.php The next email in the thread is two weeks later from the patch author asking about the status of the patch. If we don't need the refactoring part, great, but I want to be sure. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
On Sun, Sep 03, 2006 at 07:42:02PM -0400, Tom Lane wrote: > The hard part of this problem is finding a convenient way to capture > status data out of the community's conversations. I think when you find > a solution to that, you'll notice that email is not the problem. In private groups (like companies) that do this well, that sort of "convenient way" turns out to be someone who is willing to do the summarisation and post it. Perhaps what is needed is a small group of people who would like to contribute, who can't contribute code, but who can spend some time doing summaries, documents, and that sort of thing? (Yes, I'll put my money where my mouth is.) A -- Andrew Sullivan | ajs@crankycanuck.ca Users never remark, "Wow, this software may be buggy and hard to use, but at least there is a lot of code underneath." --Damien Katz
Andrew Sullivan wrote: > On Sun, Sep 03, 2006 at 07:42:02PM -0400, Tom Lane wrote: > >> The hard part of this problem is finding a convenient way to capture >> status data out of the community's conversations. I think when you find >> a solution to that, you'll notice that email is not the problem. >> > > In private groups (like companies) that do this well, that sort of > "convenient way" turns out to be someone who is willing to do the > summarisation and post it. Perhaps what is needed is a small group > of people who would like to contribute, who can't contribute code, > but who can spend some time doing summaries, documents, and that sort > of thing? (Yes, I'll put my money where my mouth is.) > > Excellent! You are just the sort of person for this task, I think. cheers andrew
Bruce Momjian wrote: > Robert Treat wrote: >> FWIW I have never understood why we don't require patch submitters/committers >> to update the release notes when they do the patch. I've suggested this more than once in the past -- I think it would be a clear improvement over the status quo. Updating the release notes incrementally would lead to more accurate and complete release notes: more accurate because the description for a feature would be written at the same time as the feature itself, and more complete because it would be harder to unintentionally omit discussion of a new feature. It would also help communicate to users what features will be in the next release of Postgres, which is certainly good from a PR point of view (a certain Swedish software company is very fond of talking about the features it will be adding in future releases, for example...) Finally, it would remove the need for a sequential scan of the CVS history, which I'm sure is pretty time-consuming, and delays the beta process. > I can't even get documentation for many patches. I am hesitant to add > even more burden. I would prefer they concentrate on documentation. The first revision of a patch often doesn't include documentation updates, but in that case the submitter should be promptly told what they need to fix; I think the same would apply here. In practice, if you're committing a patch, you *should* understand it well enough to write a release note entry for it, so the burden might end up falling on committers, anyway. -Neil
Alvaro Herrera wrote: > Joshua D. Drake wrote: > > >> It does not mean all those features are useful, they definitely are. I >> am just trying to look at it from at: >> >> WHIZ* BANG* POW* perspective. >> > > Holy crap, Batman! This database can do > > INSERT INTO foo VALUES (1,1, 'so long'), (42, 2, 'and thanks'), > (142857, 3, 'for all the fish') > I just lurk here at pgsql-hackers. That function would be very cool. Thanks! Best Regards, Carlo Florendo Astra Philippines Inc. www.astra.ph
Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > Tom Lane wrote: > >> Well, it's taken us the full month to get this far through the queue, so > >> I'd sure like to have more people on board reviewing next time. Alvaro > >> helped but I miss Neil Conway, and some other people have helped in the > >> past. However --- the present situation has nothing to do with lack of > >> reviewers, it's lack of time to finish the patches. > > > I did try to get us additional help in reviewing. Neil was unavailable, > > and Alvaro could only give part of his time. > > It strikes me that setting feature freeze in midsummer might not be the > best strategy for having manpower available to review --- people tend to > be on vacation in August. Maybe the answer is just to move the dates a > bit one way or the other. Hmm...but if you're going to do that, why not do that now: push the beta date back by, say, a month (or however long you had in mind) for this cycle. That way, the two major patches that are likely to be dropped for this cycle stand a chance to make it into this release, and you accomplish your goal of moving the dates a bit all at the same time. -- Kevin Brown kevin@sysexperts.com
On Sat, Sep 02, 2006 at 06:31:51PM +0100, Dave Page wrote: > > BTW, another "output" thing you might consider is "having draft release > > notes ready-to-go on demand". Currently, Bruce prepares the release > > notes on the basis of a very tedious scan of the CVS commit logs. > > If this sort of stuff were being dropped into a tracker as it went > > into the CVS tree, at least the research part of making the notes would > > be zero-effort (or perhaps better to say that the work would be spread > > out instead of concentrated). > > We have the developers update the CHANGELOG file with each non-trivial > change in pgAdmin. It works very well for us and producing release > announcements etc. is a trivial task. The overhead on each developer is > virtually zero as well as they have to compose some text for the commit > message anyway. Another possibility would be to annotate commit messages that should get added to the release notes and have something automagically cull those out of CVS. -- Jim C. Nasby, Database Architect jim@nasby.net 512.569.9461 (cell) http://jim.nasby.net
On Sun, Sep 03, 2006 at 09:18:00AM +0200, Peter Eisentraut wrote: > Joshua D. Drake wrote: > > http://pgbugs.commandprompt.com (still need to configure email). > > Thank you for that. > > I think an issue tracking system for patches and such may need to be > distinct from a bug-tracking system such as bugzilla, but let's get one > thing after another up. Actually, I've generally found bugzilla to be a decent tool for general-purpose tracking. Feature requests and what-not need a lot of the same capabilities of a bug tracker, and having everything in one tool is often very handy. -- Jim C. Nasby, Database Architect jim@nasby.net 512.569.9461 (cell) http://jim.nasby.net
On Fri, Sep 01, 2006 at 03:28:36PM -0400, Stephen Frost wrote: > Overall, I really think 8.2 is going to be an excellent release. I wish > autovacuum could have been enabled by default and I'd just like to ask, > now and I'll try to remember again once 8.2 is out, please let's turn it > on by default for 8.3 (and early on so we get some good testing of it). Can someone put this on the TODO, just so we (hopefully) don't forget about it? -- Jim C. Nasby, Database Architect jim@nasby.net 512.569.9461 (cell) http://jim.nasby.net
On Sat, Sep 02, 2006 at 09:26:36PM -0700, Joshua D. Drake wrote: > > >>>I just spent 1/2 hour fixing the multi-value UPDATE > >>>patch for the code drift caused by UPDATE/RETURNING. The patch is a > >>>simple grammar macro. Any coder could have taken that, reviewed it, and > >>>applied it, but no one did. > >>Perhaps that's because nobody but you wanted it to go in. > > > >We got tons of people who wanted that. > > To further this, I didn't even know it was an issue. If it was only half > an hour and it needed to be done, why wasn't it put out there? > > Thanks goes to Bruce for fixing it but I didn't know it was an issue, I > have 5 C developers, if any of them could have done it -- then some > communication is needed and I would have allocated someone to it. > > I am sure that is the case with others as well. > > I am not saying it is anyone's responsibility to speak up, but I can say > had I known some help was needed (especially something so trivial) I > would have gladly donated some time. There's been talk in the past of having some kind of system that automatically attempts to build things that are in the patch queue, both as an initial sanity-check and as a means to detect when something bit-rots... perhaps it's becoming worthwhile to set that up. -- Jim Nasby jim@nasby.net EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
Jim C. Nasby wrote:> There's been talk in the past of having some kind of system that> automatically attempts to build
thingsthat are in the patch queue, both> as an initial sanity-check and as a means to detect when something>
bit-rots...perhaps it's becoming worthwhile to set that up.
After writing the enum patch, I hacked the buildfarm client code to
apply a patch to the checked out code before building. You could then
run it thusly:
./run_build.pl --nosend --nostatus --verbose \ --patch=/home/tom/src/enums-v1.patch --patch-level=1
The idea was that patch authors could either run it manually or stick it
in a cron so they could get emailed when the patch no longer cleanly
applied, or when the patched source failed in make, make check etc.
Obviously my motivation was to keep the enum patch up to date until we
hit 8.3 and someone looks at it. To that end it might also be useful for
it to die if duplicate_oids finds anything.
I submitted a patch to Andrew, but it needed a couple of tweaks
(disabling patching on vpath builds, for example) and I don't think I
ever got around to resubmitting it, but if there's more general interest
I'll do so.
Note that it was intended for patch authors to run themselves rather
than any kind of central mechanism to test the patch queue. While it
would obviously be nice to know what the current status of any given
patch in the queue is, the thing about the patch queue is that it
contains patches that we haven't had time to review yet. It'll only take
one patch to get into the queue containing a security vulnerability, or
worse, a trojan, for it to seem unfortunate.
I had thoughts of hacking the buildfarm server to allow the posting of a
patch along with results, so that authors could report results for their
own patches, but ran out of time. Is there interest in doing that?
Obviously it'd be a different server to the existing buildfarm.
Cheers
Tom
Tom Dunstan wrote: > The idea was that patch authors could either run it manually or stick it > in a cron so they could get emailed when the patch no longer cleanly > applied, or when the patched source failed in make, make check etc. > Obviously my motivation was to keep the enum patch up to date until we > hit 8.3 and someone looks at it. To that end it might also be useful for > it to die if duplicate_oids finds anything. > > I submitted a patch to Andrew, but it needed a couple of tweaks > (disabling patching on vpath builds, for example) and I don't think I > ever got around to resubmitting it, but if there's more general interest > I'll do so. Huh, why would you disable patching on vpath builds? > Note that it was intended for patch authors to run themselves rather > than any kind of central mechanism to test the patch queue. Well, I'd think that one important benefit of passing patches through the buildfarm is detecting possible portability problems in it. > While it would obviously be nice to know what the current status of > any given patch in the queue is, the thing about the patch queue is > that it contains patches that we haven't had time to review yet. It'll > only take one patch to get into the queue containing a security > vulnerability, or worse, a trojan, for it to seem unfortunate. We could have a register of developers allowed to upload patches to the "patched build queue", and have a policy that says that you should only upload a patch if it has already passed some review. -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Alvaro Herrera wrote: >> I submitted a patch to Andrew, but it needed a couple of tweaks >> (disabling patching on vpath builds, for example) and I don't think I >> ever got around to resubmitting it, but if there's more general interest >> I'll do so. > > Huh, why would you disable patching on vpath builds? As I understand it, vpath builds only do a checkout to a single place, and then build against that (from a different directory). Non-vpath builds take a copy of the checked out source and build in the copy. If we patched the source in vpath builds, the patch would stick around when updating the source tree from CVS, and the next patch attempt would fail etc. Non-vpath builds will get a clean copy to patch in subsequent runs. I suppose I could have made vpath builds take a copy when doing a patch, but it hardly seemed worth it. > Well, I'd think that one important benefit of passing patches through > the buildfarm is detecting possible portability problems in it. Absolutely. As long as they're blessed as mention below... > We could have a register of developers allowed to upload patches to the > "patched build queue", and have a policy that says that you should only > upload a patch if it has already passed some review. This was where I was originally heading when I first thought about this functionality. I didn't go that far with my fairly modest patch since a) it wasn't clear that there was manpower available to do the initial review to bless the patches, and b) what I did do solved my immediate problem. If there is support for doing something like this, there are all kinds of things that could be done. For example, you could check which patches break which other ones. An even more way-out idea might be to have performance patches run pgbench or some other set of benchmarks. You have a performance related patch? Let's stick it in the queue and see if it really improves things or not. Note that the existing patch queue mechanism wouldn't suffice, since there's no standard way to pull patches through via http or whatever. We'd probably want to back it with a small database and webapp, which might just be a hacked buildfarm server. Cheers Tom
Tom Dunstan wrote: > Alvaro Herrera wrote: > >>I submitted a patch to Andrew, but it needed a couple of tweaks > >>(disabling patching on vpath builds, for example) and I don't think I > >>ever got around to resubmitting it, but if there's more general interest > >>I'll do so. > > > >Huh, why would you disable patching on vpath builds? > > As I understand it, vpath builds only do a checkout to a single place, > and then build against that (from a different directory). Non-vpath > builds take a copy of the checked out source and build in the copy. If > we patched the source in vpath builds, the patch would stick around when > updating the source tree from CVS, and the next patch attempt would fail > etc. Non-vpath builds will get a clean copy to patch in subsequent runs. > I suppose I could have made vpath builds take a copy when doing a patch, > but it hardly seemed worth it. Huh, but the patch can be applied with -R to revert it after the buildfarm run ... the one problem I can see is if the patch fails for some reason; for which I'd suggest running a patch --dry-run as a first step, checking that it applies cleanly, and only continue in that case. [nice ideas snipped (not sniped)] > Note that the existing patch queue mechanism wouldn't suffice, since > there's no standard way to pull patches through via http or whatever. > We'd probably want to back it with a small database and webapp, which > might just be a hacked buildfarm server. Oh, sure. I'd say there is no "existing patch queue", just a Mhonarc archive which is just useful as a reminder of what patches are there outstanding. -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Alvaro Herrera wrote: > > Huh, but the patch can be applied with -R to revert it after the > buildfarm run ... the one problem I can see is if the patch fails for > some reason; for which I'd suggest running a patch --dry-run as a first > step, checking that it applies cleanly, and only continue in that case. > > Unfortunately, this won't fly as we rely on file timestamps to tell us which files have changed. Tom's idea of making a temp copy of the repo and patching that would work, but if you're going to do that why do a vpath build anyway? Regarding the idea of a list of approved patch authorisers, don't we have such a group now? i.e. "committers". cheers andrew
Andrew Dunstan wrote: > Tom's idea of making a temp copy of the repo and patching that would work, > but if you're going to do that why do a vpath build anyway? In this case, the answer is to make sure that your patch works when *someone else* does a vpath build. > Regarding the idea of a list of approved patch authorisers, don't we have > such a group now? i.e. "committers". Right, and if committers or others are willing to put in the time required to verify that patches aren't nasty before going onto the blessed patch queue, the idea could quite possibly work and provide some value. Note that all we really need to test for here is that the patch isn't malicious; patches that are bad design or whatever are unlikely to open security holes or fry your box. A major point of the queue is that the appropriate committer often doesn't have the time to review the patch right now. There might be some benefit in allowing a wider set of contributors to bless patches as non-nasty for testing purposes, rather than waste the limited time of qualified committers. Maybe such an army doesn't exist, though. Cheers Tom
Jim C. Nasby wrote:
> There's been talk in the past of having some kind of system that
> automatically attempts to build things that are in the patch queue, both
> as an initial sanity-check and as a means to detect when something
> bit-rots... perhaps it's becoming worthwhile to set that up.
After writing the enum patch, I hacked the buildfarm client code to
apply a patch to the checked out code before building. You could then
run it thusly:
./run_build.pl --nosend --nostatus --verbose \ --patch=/home/tom/src/enums-v1.patch --patch-level=1
The idea was that patch authors could either run it manually or stick it
in a cron so they could get emailed when the patch no longer cleanly
applied, or when the patched source failed in make, make check etc.
Obviously my motivation was to keep the enum patch up to date until we
hit 8.3 and someone looks at it. To that end it might also be useful for
it to die if duplicate_oids finds anything.
I submitted a patch to Andrew, but it needed a couple of tweaks
(disabling patching on vpath builds, for example) and I don't think I
ever got around to resubmitting it, but if there's more general interest
I'll do so.
Note that it was intended for patch authors to run themselves rather
than any kind of central mechanism to test the patch queue. While it
would obviously be nice to know what the current status of any given
patch in the queue is, the thing about the patch queue is that it
contains patches that we haven't had time to review yet. It'll only take
one patch to get into the queue containing a security vulnerability, or
worse, a trojan, for it to seem unfortunate.
I had thoughts of hacking the buildfarm server to allow the posting of a
patch along with results, so that authors could report results for their
own patches, but ran out of time. Is there interest in doing that?
Obviously it'd be a different server to the existing buildfarm.
Cheers
Tom
On Sep 13, 2006, at 6:56 PM, Tom Dunstan wrote: >> Regarding the idea of a list of approved patch authorisers, don't >> we have >> such a group now? i.e. "committers". > > Right, and if committers or others are willing to put in the time > required to verify that patches aren't nasty before going onto the > blessed patch queue, the idea could quite possibly work and provide > some value. Note that all we really need to test for here is that > the patch isn't malicious; patches that are bad design or whatever > are unlikely to open security holes or fry your box. A major point > of the queue is that the appropriate committer often doesn't have > the time to review the patch right now. There might be some benefit > in allowing a wider set of contributors to bless patches as non- > nasty for testing purposes, rather than waste the limited time of > qualified committers. Maybe such an army doesn't exist, though. That's something I'd be willing to do. And for many people that aren't committers but are still trusted in the community, we could probably bypass the checking. Another possibility would be to test these patches in some kind of virtual machine that gets blown away every X days, so that even if someone did get something malicious in there it wouldn't last long. -- Jim Nasby jim@nasby.net EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
Jim Nasby wrote: > That's something I'd be willing to do. And for many people that aren't > committers but are still trusted in the community, we could probably > bypass the checking. That's a worthwhile point. How many patches come from the general community vs out of the blue? Patches from regulars could probably get a free pass, which might cut down the review burden substantially. > Another possibility would be to test these patches in some kind of > virtual machine that gets blown away every X days, so that even if > someone did get something malicious in there it wouldn't last long. Yeah, nasties could be roughly separated into two categories: stuff which affects your box, or stuff which uses your box to affect someone else. A VM fixes the first, and a firewall blocking outgoing connections (with exceptions for the CVS server and patch buildfarm or whatever it is server) largely fixes the second. I was under the impression that most VM products are x86 centric, which wouldn't lead to huge amounts of diversity in the buildfarm results. At least, not as far as architecture goes. Cheers Tom
On Wed, 13 Sep 2006, Tom Dunstan wrote: > > Another possibility would be to test these patches in some kind of virtual > > machine that gets blown away every X days, so that even if someone did get > > something malicious in there it wouldn't last long. Or just have a snapshot which is reverted after each run, and read-only access to files used to do the build. I know vmware supports this, probably others too... > Yeah, nasties could be roughly separated into two categories: stuff which > affects your box, or stuff which uses your box to affect someone else. A VM > fixes the first, and a firewall blocking outgoing connections (with exceptions > for the CVS server and patch buildfarm or whatever it is server) largely fixes > the second. > > I was under the impression that most VM products are x86 centric, which > wouldn't lead to huge amounts of diversity in the buildfarm results. At least, > not as far as architecture goes. I have played with QEmu (www.qemu.org) which is open source and supports multiple target architectures. I'm not sure how stable all of the different targets are, I know that sparc64 is not quite done yet. -- The problem with engineers is that they tend to cheat in order to get results. The problem with mathematicians is that they tend to work on toy problems in order to get results. The problem with program verifiers is that they tend to cheat at toy problems in order to get results.
Jeremy Drake wrote: > On Wed, 13 Sep 2006, Tom Dunstan wrote: > >> I was under the impression that most VM products are x86 centric, which >> wouldn't lead to huge amounts of diversity in the buildfarm results. At least, >> not as far as architecture goes. > > I have played with QEmu (www.qemu.org) which is open source and supports > multiple target architectures. I'm not sure how stable all of the > different targets are, I know that sparc64 is not quite done yet. Oh, I didn't realize Qemu did non-x86 architectures. Is it considered good enough at emulating e.g. a sparc for it to be useful to us? PearPC was a PowerPC emulator that got some press a while ago, although it appears that the project has stagnated a bit (probably because people who wanted to run OSX on intel hardware have a legit way to do it now :) ) The problem with these things is if something goes wrong, was it the patch that failed or the not-quite-perfect VM product? To cut down on those sorts of problems, I suppose we could have it do a clean, non-patched run first, and then only do the patched version if the clean version passed. We'd have to be reasonably unlucky to have a patch trigger a VM bug under those circumstance, I would think. Cheers Tom
Hi, Jeremy, Jeremy Drake wrote: >>> Another possibility would be to test these patches in some kind of virtual >>> machine that gets blown away every X days, so that even if someone did get >>> something malicious in there it wouldn't last long. > > Or just have a snapshot which is reverted after each run, and read-only > access to files used to do the build. I know vmware supports this, > probably others too... A chroot / fakeroot combined with unionfs should do the same, probably with less effort. There are other user-mode jail projects that also block networking. Markus -- Markus Schaber | Logical Tracking&Tracing International AG Dipl. Inf. | Software Development GIS Fight against software patents in Europe! www.ffii.org www.nosoftwarepatents.org
Peter Eisentraut <peter_e@gmx.net> writes:
> That does not mean that the patch is bad, and I certainly support the
> feature change. But I can't efficiently review the patch. If someone
> else wants to do it, go ahead.
I've finally taken a close look at this patch, and I don't like it any
more than Peter does. The refactoring might or might not be good at its
core, but as presented it is horrid. As just one example, it replaces one
reasonably well-commented function with three misnamed, poorly commented
functions. In place of
/*
! * Sets option `name' to given value. The value should be a string
! * which is going to be parsed and converted to the appropriate data
! * type. The context and source parameters indicate in which context this
! * function is being called so it can apply the access restrictions
! * properly.
! *
! * If value is NULL, set the option to its default value. If the
! * parameter changeVal is false then don't really set the option but do all
! * the checks to see if it would work.
! *
! * If there is an error (non-existing option, invalid value) then an
! * ereport(ERROR) is thrown *unless* this is called in a context where we
! * don't want to ereport (currently, startup or SIGHUP config file reread).
! * In that case we write a suitable error message via ereport(DEBUG) and
! * return false. This is working around the deficiencies in the ereport
! * mechanism, so don't blame me. In all other cases, the function
! * returns true, including cases where the input is valid but we chose
! * not to apply it because of context or source-priority considerations.
! *
! * See also SetConfigOption for an external interface. */
! bool
! set_config_option(const char *name, const char *value,
! GucContext context, GucSource source,
! bool isLocal, bool changeVal)
we find
/*
! * Try to parse value. Determine what is type and call related
! * parsing function or if newval is equal to NULL, reset value
! * to default or bootval. If the value parsed okay return true,
! * else false. */
! static bool
! parse_value(int elevel, const struct config_generic *record,
! const char *value, GucSource *source, bool changeVal,
! union config_var_value *retval)
which doesn't tell you quite what the parameters do, but more
fundamentally is misnamed because one would expect "parse_value"
returning bool to merely check whether the value is syntactically
correct. Well, it doesn't do that: it applies the value too.
Another broken-out routine is
! /*
! * Check if the option can be set at this time. See guc.h for the precise
! * rules.
! */
! static bool
! checkContext(int elevel, struct config_generic *record, GucContext context)
which is again a misleading description because it doesn't bother to
explain that control may not come back if the option is rejected
(depending on elevel). One might also think, given that description,
that the caller is supposed to emit an error message on false result.
Lastly we have
+ /*
+ * Verify if option exists and value is valid.
+ * It is primary used for validation of items in configuration file.
+ */
+ bool
+ verify_config_option(const char *name, const char *value,
+ GucContext context, GucSource source,
+ bool *isNewEqual, bool *isContextOK)
which again is far south of my ideas for adequate documentation of a
function with a fairly complicated API. And guess what, this one has
side effects too, which it surely should not (and that leads directly
to a bug: GUC_IN_CONFFILE could remain set in a variable after a
parsing failure).
It's possible that a refactoring along these lines could be an
improvement if it were well coded and well documented, but this patch
is not it.
The comment-reversion part of the patch is not any better. It's poorly
factored (what the heck is guc-file.l doing patching up the source
settings after calling set_config_option?), which is surprising
considering the whole point of the refactoring was to support this.
And the handling of reversion to a PGC_S_ENV_VAR setting is just a kluge
involving duplicated code (what was that about refactoring again?).
In short, whether or not it has any remaining undetected bugs, this
patch is a severe disimprovement from the standpoint of source code
quality, and I recommend rejecting it.
regards, tom lane
OK, patch sent to the 8.3 hold queue for later work, open item removed. --------------------------------------------------------------------------- Tom Lane wrote: > Peter Eisentraut <peter_e@gmx.net> writes: > > That does not mean that the patch is bad, and I certainly support the > > feature change. But I can't efficiently review the patch. If someone > > else wants to do it, go ahead. > > I've finally taken a close look at this patch, and I don't like it any > more than Peter does. The refactoring might or might not be good at its > core, but as presented it is horrid. As just one example, it replaces one > reasonably well-commented function with three misnamed, poorly commented > functions. In place of > > /* > ! * Sets option `name' to given value. The value should be a string > ! * which is going to be parsed and converted to the appropriate data > ! * type. The context and source parameters indicate in which context this > ! * function is being called so it can apply the access restrictions > ! * properly. > ! * > ! * If value is NULL, set the option to its default value. If the > ! * parameter changeVal is false then don't really set the option but do all > ! * the checks to see if it would work. > ! * > ! * If there is an error (non-existing option, invalid value) then an > ! * ereport(ERROR) is thrown *unless* this is called in a context where we > ! * don't want to ereport (currently, startup or SIGHUP config file reread). > ! * In that case we write a suitable error message via ereport(DEBUG) and > ! * return false. This is working around the deficiencies in the ereport > ! * mechanism, so don't blame me. In all other cases, the function > ! * returns true, including cases where the input is valid but we chose > ! * not to apply it because of context or source-priority considerations. > ! * > ! * See also SetConfigOption for an external interface. > */ > ! bool > ! set_config_option(const char *name, const char *value, > ! GucContext context, GucSource source, > ! bool isLocal, bool changeVal) > > we find > > /* > ! * Try to parse value. Determine what is type and call related > ! * parsing function or if newval is equal to NULL, reset value > ! * to default or bootval. If the value parsed okay return true, > ! * else false. > */ > ! static bool > ! parse_value(int elevel, const struct config_generic *record, > ! const char *value, GucSource *source, bool changeVal, > ! union config_var_value *retval) > > which doesn't tell you quite what the parameters do, but more > fundamentally is misnamed because one would expect "parse_value" > returning bool to merely check whether the value is syntactically > correct. Well, it doesn't do that: it applies the value too. > Another broken-out routine is > > ! /* > ! * Check if the option can be set at this time. See guc.h for the precise > ! * rules. > ! */ > ! static bool > ! checkContext(int elevel, struct config_generic *record, GucContext context) > > which is again a misleading description because it doesn't bother to > explain that control may not come back if the option is rejected > (depending on elevel). One might also think, given that description, > that the caller is supposed to emit an error message on false result. > Lastly we have > > + /* > + * Verify if option exists and value is valid. > + * It is primary used for validation of items in configuration file. > + */ > + bool > + verify_config_option(const char *name, const char *value, > + GucContext context, GucSource source, > + bool *isNewEqual, bool *isContextOK) > > which again is far south of my ideas for adequate documentation of a > function with a fairly complicated API. And guess what, this one has > side effects too, which it surely should not (and that leads directly > to a bug: GUC_IN_CONFFILE could remain set in a variable after a > parsing failure). > > It's possible that a refactoring along these lines could be an > improvement if it were well coded and well documented, but this patch > is not it. > > The comment-reversion part of the patch is not any better. It's poorly > factored (what the heck is guc-file.l doing patching up the source > settings after calling set_config_option?), which is surprising > considering the whole point of the refactoring was to support this. > And the handling of reversion to a PGC_S_ENV_VAR setting is just a kluge > involving duplicated code (what was that about refactoring again?). > > In short, whether or not it has any remaining undetected bugs, this > patch is a severe disimprovement from the standpoint of source code > quality, and I recommend rejecting it. > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 3: Have you checked our extensive FAQ? > > http://www.postgresql.org/docs/faq -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Am Freitag, 15. September 2006 20:32 schrieb Tom Lane: > I've finally taken a close look at this patch, and I don't like it any > more than Peter does. The refactoring might or might not be good at its > core, but as presented it is horrid. Joachim Wieland is in the process of reworking the original feature patch (resetting commented out parameters) in a much more compact form. But it turns out that there are a couple of very tricky situations involving custom variables, which may need more discussion than we have time for now. -- Peter Eisentraut http://developer.postgresql.org/~petere/
On Mon, Sep 18, 2006 at 01:13:45PM +0200, Peter Eisentraut wrote:
> Joachim Wieland is in the process of reworking the original feature patch
> (resetting commented out parameters) in a much more compact form. But it
> turns out that there are a couple of very tricky situations involving custom
> variables, which may need more discussion than we have time for now.
The problem with custom variables is the definition of their default value.
First I defined it to be "variable does not exist" and tried to have a close
analogy to what happens on set/set local/reset with normal variables.
However that means that custom variables have to vanish (where other
variables show their default value) and revive (where other variables show a
changed value). For example if you have:
begin;
savepoint a;
set foo.var to 3;
(here we delete foo.var from the configuration file and send SIGHUP)
savepoint b;
reset foo.var;
There seem to exist quite a few possible definitions as to what happens if you
run "show foo.var" after the "reset" directly or after rolling back to the
savepoints a and b.
The result of the discussion with Peter was that all that seems not worth
doing since it is a rare special case. In the current form of the patch if you
delete a custom variable from the configuration file it gets deleted
internally and all reference to it results in error. There is still one
special case which is:
begin;
set foo.var to 3;
(here we delete foo.var from the configuration file and send SIGHUP)
commit;
This transaction will fail because commit cannot change the value of the
variable to 3 as requested by the user.
I have the patch almost ready in the described form, if there is any chance
it will make it into 8.2 I will clean it up and post it ASAP but Peter wrote
me that chances are close to zero and so I stopped working on it for now.
Joachim
--
Joachim Wieland joe@mcknight.de
GPG key available
Joachim Wieland <joe@mcknight.de> writes:
> I have the patch almost ready in the described form, if there is any chance
> it will make it into 8.2 I will clean it up and post it ASAP but Peter wrote
> me that chances are close to zero and so I stopped working on it for now.
If you'd mentioned it a bit sooner I would have encouraged you to get
it done. But given your description it sounds a bit too
hairy/potentially-destabilizing to be dropping in the day before beta.
(I certainly hadn't thought about custom vars at all :-()
Let's plan it for 8.3 instead.
regards, tom lane
Tom Lane wrote: > Peter Eisentraut <peter_e@gmx.net> writes: >> That does not mean that the patch is bad, and I certainly support the >> feature change. But I can't efficiently review the patch. If someone >> else wants to do it, go ahead. > > I've finally taken a close look at this patch, and I don't like it any > more than Peter does. The refactoring might or might not be good at its > core, but as presented it is horrid. As just one example, it replaces one > reasonably well-commented function with three misnamed, poorly commented > functions. In place of Thanks Tom for your time to look on the code and for your feedback. It is very useful for me. Thanks Zdenek
On Wed, Sep 13, 2006 at 22:22:12 -0700, Tom Dunstan <pgsql@tomd.cc> wrote: > > That's a worthwhile point. How many patches come from the general > community vs out of the blue? Patches from regulars could probably get a > free pass, which might cut down the review burden substantially. And how were you planning to tell if a patch cam from a regular? Hopefully you weren't planning on blindly trusting the "from" header. Misuse of the build farm in a way the effects other sites could get the project a big black eye, so you want to be very careful building and executing code from the patch queue.
On Fri, Sep 22, 2006 at 11:58:04AM -0500, Bruno Wolff III wrote: > On Wed, Sep 13, 2006 at 22:22:12 -0700, > Tom Dunstan <pgsql@tomd.cc> wrote: > > > > That's a worthwhile point. How many patches come from the general > > community vs out of the blue? Patches from regulars could probably get a > > free pass, which might cut down the review burden substantially. > > And how were you planning to tell if a patch cam from a regular? Hopefully > you weren't planning on blindly trusting the "from" header. > Misuse of the build farm in a way the effects other sites could get the > project a big black eye, so you want to be very careful building and > executing code from the patch queue. Of course not, but there's any number of ways we could handle that problem. -- Jim Nasby jim@nasby.net EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
Jim C. Nasby wrote: > On Fri, Sep 22, 2006 at 11:58:04AM -0500, Bruno Wolff III wrote: >> On Wed, Sep 13, 2006 at 22:22:12 -0700, >> Tom Dunstan <pgsql@tomd.cc> wrote: >>> That's a worthwhile point. How many patches come from the general >>> community vs out of the blue? Patches from regulars could probably get a >>> free pass, which might cut down the review burden substantially. >> And how were you planning to tell if a patch cam from a regular? Hopefully >> you weren't planning on blindly trusting the "from" header. >> Misuse of the build farm in a way the effects other sites could get the >> project a big black eye, so you want to be very careful building and >> executing code from the patch queue. > > Of course not, but there's any number of ways we could handle that > problem. pgp signed patches? -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutionssince 1997 http://www.commandprompt.com/
On Sep 22, 2006, at 2:50 PM, Joshua D. Drake wrote: >>> And how were you planning to tell if a patch cam from a regular? >>> Hopefully >>> you weren't planning on blindly trusting the "from" header. >>> Misuse of the build farm in a way the effects other sites could >>> get the >>> project a big black eye, so you want to be very careful building and >>> executing code from the patch queue. >> Of course not, but there's any number of ways we could handle that >> problem. > > pgp signed patches? Just one possibility. Submitting the patches via a web page that you have to log into is another. -- Jim Nasby jimn@enterprisedb.com EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)