Thread: postponing some large patches to 9.2
On Mon, Feb 7, 2011 at 3:57 PM, Chris Browne <cbbrowne@acm.org> wrote: > It sure looks to me like there are going to be a bunch of items that, > based on the recognized policies, need to get deferred to 9.2, and the > prospects for Sync Rep getting into 9.1 don't look notably good to me. > > It's definitely readily arguable that fairness requires that: > > - Items not committable by 2011-02-15 be deferred to the 2011-Next fest > > There are around 25 items right now that are sitting with [Waiting > for Author] and [Returned with Feedback] statuses. They largely seem > like pretty fair game for "next fest." > > - Large items that weren't included in the 2010-11 fest be considered > problematic to try to integrate into 9.1 > > There sure seem to be some large items in the 2011-01 fest, which I > thought wasn't supposed to be the case. This discussion reveals that it's time to start making some discussions about what can be accomplished for 9.1 and what must be postponed to 9.2. The big ones I think we should postpone are: - Range Types. This is a large patch which was submitted for the first time to the last CommitFest of the cycle, and the first version that had no open TODO items was posted yesterday, three-quarters of the way through that last CommitFest. Some good review has been done.While more is probably needed, I think we should feelgood about what's been accomplished and mark this one Returned with Feedback. - ALTER EXTENSION UPGRADE. This is another large patch which was submitted for the first time to the last CommitFest of the cycle. The prerequisite patch to provide basic extension support has not been committed yet, although it sounds like that will happen soon. However, since that patch is undergoing a fair amount of surgery, it's reasonably certain that this will require significant rebasing. I think it would also be an overstatement to say that we have consensus on the design. My feeling is that, unless Tom thinks that failing to get this committed now is going to leave us with a mess later, we should mark this one Returned with Feedback and revisit it for 9.2. - FOR KEY LOCK tables. This patch, unfortunately, has not gotten a lot of review. But it's a major, potentially destabilizing change that was, like the last two, submitted for the first time to the last CommitFest of the cycle. Even if we assume for the sake of argument that the code is unusually good for a feature of this type, I don't think it's the right time to commit something like this. I would argue for putting this off until 9.2, though preferably with a bit more review than it's gotten so far. The other remaining "big" patches are: - extension support for pg_upgrade. Tom is planning to have this committed within a day or so, per latest news. - synchronous replication. Based on some looking at this today, I am somewhat doubtful about the possibility of me or anyone else beating this completely into shape in time for 9.2, unless we choose to extend the deadline by several weeks. Simon said that he would have time to finish this in the next two weeks, but, as noted, the CommitFest is scheduled to be over in ONE week, and it looks to me like this is still pretty rough. However, there's a lot of good stuff in here, and I think it might be practical to get some portion of it committed even if we can't agree on all of it. I recommend we give this one a little more time to shake out before giving up on it. - SQL/MED. This patch unfortunately kind of stalled for a long time. However, I believe that Heikki is now working actively on the core patch, so I'm hopeful for some activity here soon. - Writeable CTEs. Tom has promised to look at this, but doesn't seem to be in a hurry. - Per-column collation. This one has been lingering for a long time. But Noah Misch recently did a pretty thorough review, and it's now marked Ready for Committer. Peter, are you planning to commit this? - The PL/python extravaganza. I'm not really clear where we stand with this. There are a lot of patches here. There are a variety of smaller patches we need to make decisions about, too. But summarizing all of that here is going to be too much.I'll post to some of the other threads individually. Thoughts on these? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 11-02-07 10:37 PM, Robert Haas wrote: > - The PL/python extravaganza. I'm not really clear where we stand > with this. There are a lot of patches here. > Some of the patches have been committed a few others are ready (or almost ready) for a committer. The table function one is the only one in 'waiting for author' 4 of the patches haven't yet received any review. Jan Urbanski has been pretty good about posting updated patches as the dependent patches get updated. It would be good if a few people grabbed these. The individual patches tend to not be that large.
* Robert Haas (robertmhaas@gmail.com) wrote:
> - Range Types. This is a large patch which was submitted for the
> first time to the last CommitFest of the cycle, and the first version
> that had no open TODO items was posted yesterday, three-quarters of
> the way through that last CommitFest. Some good review has been done.
> While more is probably needed, I think we should feel good about
> what's been accomplished and mark this one Returned with Feedback.
I don't agree w/ punting Range Types. Range Types were discussed as far
back as the 2010 developer meeting, were discussed quite a bit again
starting in October and throughout the fall, and Jeff has regularly
been posting updates to it. Given how thorough Jeff is, my feeling is
that this patch is more than ready for beta. My impression is also that
it's not as invasive or destablizing as the others and while it wasn't
being posted to the previous commit fests, it was clearly being worked
on, updated, and improved.
> - synchronous replication. Based on some looking at this today, I am
> somewhat doubtful about the possibility of me or anyone else beating
> this completely into shape in time for 9.2, unless we choose to extend
> the deadline by several weeks. Simon said that he would have time to
> finish this in the next two weeks, but, as noted, the CommitFest is
> scheduled to be over in ONE week, and it looks to me like this is
> still pretty rough. However, there's a lot of good stuff in here, and
> I think it might be practical to get some portion of it committed even
> if we can't agree on all of it. I recommend we give this one a little
> more time to shake out before giving up on it.
It really would be nice to have this, but I agree that it's pretty late
in the game for it to be in the state is appears to be in. :/ It also
seems to have been stalled for the past couple of months, which doesn't
bode well for it, in my view.
Thanks,
Stephen
2011/2/8 Steve Singer <ssinger_pg@sympatico.ca>: > On 11-02-07 10:37 PM, Robert Haas wrote: > >> - The PL/python extravaganza. I'm not really clear where we stand >> with this. There are a lot of patches here. >> > > Some of the patches have been committed a few others are ready (or almost > ready) for a committer. The table function one is the only one in > 'waiting for author' I didn't quite finished my reviews on pl/python patches yet, but it seems that "don't remove argument" will be easy to review and unlikely to have issues. "quote functions" can be committed already as-is. "table function" should be in if Jan sends another patch soon. "custom datatype parser" looks premature yet, though we want to give more feedbacks about its design. I'm not sure about other patches. > 4 of the patches haven't yet received any review. Jan Urbanski has been > pretty good about posting updated patches as the dependent patches get > updated. It would be good if a few people grabbed these. The individual > patches tend to not be that large. I agree he did quite a lot of work well. But still some of the patches are very easy and others is like WIP stage. I hope anyone can review them as soon as possible. Regards, -- Hitoshi Harada
On 08/02/11 15:44, Hitoshi Harada wrote: > 2011/2/8 Steve Singer <ssinger_pg@sympatico.ca>: >> On 11-02-07 10:37 PM, Robert Haas wrote: >> >>> - The PL/python extravaganza. I'm not really clear where we stand >>> with this. There are a lot of patches here. >>> >> >> Some of the patches have been committed a few others are ready (or almost >> ready) for a committer. The table function one is the only one in >> 'waiting for author' > > I didn't quite finished my reviews on pl/python patches yet, but it > seems that "don't remove argument" will be easy to review and unlikely > to have issues. "quote functions" can be committed already as-is. > "table function" should be in if Jan sends another patch soon. "custom > datatype parser" looks premature yet, though we want to give more > feedbacks about its design. I'm not sure about other patches. From the outstanding PL/Python patches: * invalidate composites - it fixes a TODO item and is a general improvement, although the case being fixed is rather rare. Should be straightforward to review, it's small and localized. * tracebacks - IMHO a big improvement, but should get more testing than I gave it, as traversing Python stacks tends to be tricky. Still, it's very localized (basically just getting a traceback string). * table functions - I'm working on this one, should have an update today * custom datatype parsers - more of a PoC, and because the discussion about hstores in PL/Python died down I decided to go ahead and send a patch implementing the last idea from that thread. I'm fine with it not making 9.1 because this should be designed carefully, and will affect decisions similar to those that are now being taken WRT arrays in PL/Perl. * custom SPI exceptions - I'd really like this one to go in, because it allows writing UPSERT-kind functions in PL/Python very easily, and it's just a handful of lines of code * don't remove arguments - a bugfix, really, and a very small one So from the above, I'd say custom datatype parsers could get rejected if noone feels like having a discussion about it for 9.1. Table functions, custom SPI exceptions and tracebacks are niceties that if postponed to 9.2 will just mean that many features less in 9.1. The rest is bordering on bugfixes, and I think should go in. Cheers, Jan
sfrost@snowman.net (Stephen Frost) writes:
> * Robert Haas (robertmhaas@gmail.com) wrote:
>> - Range Types. This is a large patch which was submitted for the
>> first time to the last CommitFest of the cycle, and the first version
>> that had no open TODO items was posted yesterday, three-quarters of
>> the way through that last CommitFest. Some good review has been done.
>> While more is probably needed, I think we should feel good about
>> what's been accomplished and mark this one Returned with Feedback.
>
> I don't agree w/ punting Range Types. Range Types were discussed as
> far back as the 2010 developer meeting, were discussed quite a bit
> again starting in October and throughout the fall, and Jeff has
> regularly been posting updates to it. Given how thorough Jeff is, my
> feeling is that this patch is more than ready for beta. My impression
> is also that it's not as invasive or destablizing as the others and
> while it wasn't being posted to the previous commit fests, it was
> clearly being worked on, updated, and improved.
I generally mirror those thoughts. Range Types don't seem invasive or
destabilizing, and the code base has been deployed for quite some time
as an extension ("not quite contrib"). It would be disappointing to
drop this one when it is mighty close.
>> - synchronous replication. Based on some looking at this today, I am
>> somewhat doubtful about the possibility of me or anyone else beating
>> this completely into shape in time for 9.2, unless we choose to extend
>> the deadline by several weeks. Simon said that he would have time to
>> finish this in the next two weeks, but, as noted, the CommitFest is
>> scheduled to be over in ONE week, and it looks to me like this is
>> still pretty rough. However, there's a lot of good stuff in here, and
>> I think it might be practical to get some portion of it committed even
>> if we can't agree on all of it. I recommend we give this one a little
>> more time to shake out before giving up on it.
>
> It really would be nice to have this, but I agree that it's pretty late
> in the game for it to be in the state is appears to be in. :/ It also
> seems to have been stalled for the past couple of months, which doesn't
> bode well for it, in my view.
The "stall" troubles me, and doesn't bode terribly well for 9.1.
--
Rules of the Evil Overlord #39. "If I absolutely must ride into
battle, I will certainly not ride at the forefront of my Legions of
Terror, nor will I seek out my opposite number among his army."
<http://www.eviloverlord.com/>
On 11-02-08 10:07 AM, Jan Urbański wrote: > > * custom SPI exceptions - I'd really like this one to go in, because it > allows writing UPSERT-kind functions in PL/Python very easily, and it's > just a handful of lines of code > I will try to do a review of this one (probably tomorrow night) since I've reviewed many of the related patches. > * don't remove arguments - a bugfix, really, and a very small one > > So from the above, I'd say custom datatype parsers could get rejected if > noone feels like having a discussion about it for 9.1. Table functions, > custom SPI exceptions and tracebacks are niceties that if postponed to > 9.2 will just mean that many features less in 9.1. The rest is bordering > on bugfixes, and I think should go in. > > Cheers, > Jan >
On Tue, Feb 8, 2011 at 10:40 AM, Chris Browne <cbbrowne@acm.org> wrote:
> sfrost@snowman.net (Stephen Frost) writes:
>> * Robert Haas (robertmhaas@gmail.com) wrote:
>>> - Range Types. This is a large patch which was submitted for the
>>> first time to the last CommitFest of the cycle, and the first version
>>> that had no open TODO items was posted yesterday, three-quarters of
>>> the way through that last CommitFest. Some good review has been done.
>>> While more is probably needed, I think we should feel good about
>>> what's been accomplished and mark this one Returned with Feedback.
>>
>> I don't agree w/ punting Range Types. Range Types were discussed as
>> far back as the 2010 developer meeting, were discussed quite a bit
>> again starting in October and throughout the fall, and Jeff has
>> regularly been posting updates to it. Given how thorough Jeff is, my
>> feeling is that this patch is more than ready for beta. My impression
>> is also that it's not as invasive or destablizing as the others and
>> while it wasn't being posted to the previous commit fests, it was
>> clearly being worked on, updated, and improved.
>
> I generally mirror those thoughts. Range Types don't seem invasive or
> destabilizing, and the code base has been deployed for quite some time
> as an extension ("not quite contrib"). It would be disappointing to
> drop this one when it is mighty close.
It's a 5400 line patch that wasn't completed until the middle of the
current CommitFest. Nobody has ever submitted a major feature patch
of that size that got done in a single CommitFest, to my recollection,
or even half that size. Compare Hot Standby or True Serializability,
both of which required basically a full development cycle. It may be
true that the idea has been kicking around for a long time, but it's
been code-complete for one week.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Mon, Feb 07, 2011 at 10:37:06PM -0500, Robert Haas wrote: > On Mon, Feb 7, 2011 at 3:57 PM, Chris Browne <cbbrowne@acm.org> wrote: > > It sure looks to me like there are going to be a bunch of items that, > > based on the recognized policies, need to get deferred to 9.2, and the > > prospects for Sync Rep getting into 9.1 don't look notably good to me. > > > > It's definitely readily arguable that fairness requires that: > > > > - Items not committable by 2011-02-15 be deferred to the 2011-Next fest > > > > There are around 25 items right now that are sitting with [Waiting > > for Author] and [Returned with Feedback] statuses. They largely seem > > like pretty fair game for "next fest." > > > > - Large items that weren't included in the 2010-11 fest be considered > > problematic to try to integrate into 9.1 > > > > There sure seem to be some large items in the 2011-01 fest, which I > > thought wasn't supposed to be the case. > > This discussion reveals that it's time to start making some > discussions about what can be accomplished for 9.1 and what must be > postponed to 9.2. Given how things worked, i.e. that people were not clear that 9.1 development had actually started, etc., I am again proposing that we have one more CF starting March 15 to get this all cleaned up. Yes, I know that wasn't the plan, but I also know that we're really, really close on a whole bunch of things, some of which have been in the offing for years at this point, and we risk giving people the impression, if they don't already have it, that if they're not in the "inner circle," their patches get lower priority no matter what their merits. > The big ones I think we should postpone are: > > - Range Types. This is a large patch which was submitted for the > first time to the last CommitFest of the cycle, and the first version > that had no open TODO items was posted yesterday, three-quarters of > the way through that last CommitFest. Some good review has been done. > While more is probably needed, I think we should feel good about > what's been accomplished and mark this one Returned with Feedback. This one's a product differentiator for PostgreSQL. We can own this space for at least a year before it gets on any other RDBMS's roadmap. That the work wasn't submitted until it was in pretty good shape is a mark in Jeff's favor, not a reason to punt for another year. > - ALTER EXTENSION UPGRADE. This is another large patch which was > submitted for the first time to the last CommitFest of the cycle. > The prerequisite patch to provide basic extension support has not > been committed yet, although it sounds like that will happen soon. > However, since that patch is undergoing a fair amount of surgery, > it's reasonably certain that this will require significant rebasing. > I think it would also be an overstatement to say that we have > consensus on the design. My feeling is that, unless Tom thinks that > failing to get this committed now is going to leave us with a mess > later, we should mark this one Returned with Feedback and revisit it > for 9.2. If we're going to leave this out, we should probably pull EXTENSIONs out entirely. Is that really where we want to go? > - FOR KEY LOCK tables. This patch, unfortunately, has not gotten a > lot of review. But it's a major, potentially destabilizing change > that was, like the last two, submitted for the first time to the last > CommitFest of the cycle. Even if we assume for the sake of argument > that the code is unusually good for a feature of this type, I don't > think it's the right time to commit something like this. I would > argue for putting this off until 9.2, though preferably with a bit > more review than it's gotten so far. > > The other remaining "big" patches are: > > - extension support for pg_upgrade. Tom is planning to have this > committed within a day or so, per latest news. See above. > - synchronous replication. Based on some looking at this today, I am > somewhat doubtful about the possibility of me or anyone else beating > this completely into shape in time for 9.2, unless we choose to extend > the deadline by several weeks. +1 for extending that deadline. This is another product differentiator, and we can have it as that for about a year if we make sure it gets into 9.1. > Simon said that he would have time to > finish this in the next two weeks, but, as noted, the CommitFest is > scheduled to be over in ONE week, and it looks to me like this is > still pretty rough. However, there's a lot of good stuff in here, and > I think it might be practical to get some portion of it committed even > if we can't agree on all of it. I recommend we give this one a little > more time to shake out before giving up on it. > > - SQL/MED. This patch unfortunately kind of stalled for a long time. > However, I believe that Heikki is now working actively on the core > patch, so I'm hopeful for some activity here soon. > > - Writeable CTEs. Tom has promised to look at this, but doesn't seem > to be in a hurry. This patch is about the worst in terms of "the inner circle" vs. our open development model I referred to above. This, too, is a product differentiator that we can really lead the standard with, and there's no way to break any conceivable change to the standard with it. What's the latest? > - Per-column collation. This one has been lingering for a long time. > But Noah Misch recently did a pretty thorough review, and it's now > marked Ready for Committer. Peter, are you planning to commit this? I think we're faced with the hard reality that we can't get a perfect implementation of this in one go, as that would involve pulling the "correctness" bits away from the OS-supplied libraries, etc., and into PostgreSQL. I'm all for doing this eventually, and meanwhile getting what's already done in. > - The PL/python extravaganza. I'm not really clear where we stand > with this. There are a lot of patches here. Many of these are already a go. :) Cheers, David. -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
On tis, 2011-02-08 at 00:16 -0500, Steve Singer wrote: > On 11-02-07 10:37 PM, Robert Haas wrote: > > > - The PL/python extravaganza. I'm not really clear where we stand > > with this. There are a lot of patches here. > > > > Some of the patches have been committed a few others are ready (or > almost ready) for a committer. The table function one is the only one > in 'waiting for author' > > 4 of the patches haven't yet received any review. Jan Urbanski has been > pretty good about posting updated patches as the dependent patches get > updated. It would be good if a few people grabbed these. The individual > patches tend to not be that large. I'm available to commit these if someone else can do the initial review.
On Mon, 2011-02-07 at 22:37 -0500, Robert Haas wrote: > - Range Types. This is a large patch which was submitted for the > first time to the last CommitFest of the cycle, and the first version > that had no open TODO items was posted yesterday, three-quarters of > the way through that last CommitFest. Some good review has been done. > While more is probably needed, I think we should feel good about > what's been accomplished and mark this one Returned with Feedback. I submitted this clearly marked WIP, so I expected that it would likely be pushed to 9.2. However, I don't feel like I have the kind of feedback that will help me get it committed next commitfest. I did get some review, and that was helpful, but it was mostly on isolated details. The patch is a million little decisions: names, catalog structure, interface, representation, general usability, grammar, functionality, etc. Without some checkpoint, the chances that everyone agrees with all of these decisions at the beginning of the next commitfest is zero. Is the commitfest not the right place to do this? If not, then when? Regards,Jeff Davis
On Tue, Feb 8, 2011 at 1:02 PM, David Fetter <david@fetter.org> wrote: > Given how things worked, i.e. that people were not clear that 9.1 > development had actually started, etc., I am again proposing that we > have one more CF starting March 15 to get this all cleaned up. Yes, I > know that wasn't the plan, but I also know that we're really, really > close on a whole bunch of things, some of which have been in the > offing for years at this point, and we risk giving people the > impression, if they don't already have it, that if they're not in the > "inner circle," their patches get lower priority no matter what their > merits. I agree that we have some problems in that area - particularly with writeable CTEs - but prolonging the schedule isn't going to fix that problem. I don't think that's entirely fair to people who planned their work over the last eight months so that their patches would be committed before the deadline. I both worked hard to make sure the stuff I cared most about got committed in time, and passed over projects that I could not get done in time, even though I *could* have gotten them done given another two months. I realize there are all sorts of good reasons why people didn't get things done on time, like, say, the need to earn a living - but having a time frame and sticking with it is ultimately better for the project, at least in my opinion. If we have to slip the end of the CommitFest a little to get a few extra things in, OK, but adding another two months to the development schedule that's been published for most of a year is a pretty drastic change. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Tue, 2011-02-08 at 06:57 -0500, Stephen Frost wrote: > * Robert Haas (robertmhaas@gmail.com) wrote: > > - Range Types. This is a large patch which was submitted for the > > first time to the last CommitFest of the cycle, and the first version > > that had no open TODO items was posted yesterday, three-quarters of > > the way through that last CommitFest. Some good review has been done. > > While more is probably needed, I think we should feel good about > > what's been accomplished and mark this one Returned with Feedback. > > I don't agree w/ punting Range Types. Range Types were discussed as far > back as the 2010 developer meeting, were discussed quite a bit again > starting in October and throughout the fall, and Jeff has regularly > been posting updates to it. Given how thorough Jeff is, my feeling is > that this patch is more than ready for beta. I appreciate the sentiment, but in addition to some cleanup, any patch like this at least requires some discussion. It's a language change we'll be supporting for a long time. At minimum, we're a couple hundred emails shy of a real consensus on the naming ;) Regards,Jeff Davis
* Jeff Davis (pgsql@j-davis.com) wrote:
> I appreciate the sentiment, but in addition to some cleanup, any patch
> like this at least requires some discussion. It's a language change
> we'll be supporting for a long time.
My feeling was that we have had at least some of that discussion this
past fall... It's not like you went out and developed this completely
outside of any community contact. Perhaps it's just not as
controversial as you're expecting. :)
> At minimum, we're a couple hundred emails shy of a real consensus on the
> naming ;)
*smirk.
Thanks,
Stephen
On Tue, Feb 08, 2011 at 02:04:04PM -0500, Robert Haas wrote: > On Tue, Feb 8, 2011 at 1:02 PM, David Fetter <david@fetter.org> wrote: > > Given how things worked, i.e. that people were not clear that 9.1 > > development had actually started, etc., I am again proposing that we > > have one more CF starting March 15 to get this all cleaned up. Yes, I > > know that wasn't the plan, but I also know that we're really, really > > close on a whole bunch of things, some of which have been in the > > offing for years at this point, and we risk giving people the > > impression, if they don't already have it, that if they're not in the > > "inner circle," their patches get lower priority no matter what their > > merits. > > I agree that we have some problems in that area - particularly with > writeable CTEs - but prolonging the schedule isn't going to fix that > problem. What is? > I don't think that's entirely fair to people who planned their work > over the last eight months so that their patches would be committed > before the deadline. I both worked hard to make sure the stuff I > cared most about got committed in time, and passed over projects that > I could not get done in time, even though I *could* have gotten them > done given another two months. I realize there are all sorts of good > reasons why people didn't get things done on time, like, say, the need > to earn a living - but having a time frame and sticking with it is > ultimately better for the project, at least in my opinion. If we have > to slip the end of the CommitFest a little to get a few extra things > in, OK, but adding another two months to the development schedule > that's been published for most of a year is a pretty drastic change. This development cycle was a change from other development cycles in that it "began" before development had closed in the previous cycle. I will not take a position at the moment as to the wisdom of making this change, but it's been clear from feedback from lots of developers, even ones who'd be expected to be "in the know," that this vital piece of information did not gotten out in time. Let's assume for the moment that we're going with overlapping development cycles into the future. I'd submit that given the propagation delay of this change, the schedule for the first iteration of this never was reasonable, and "slipping" two months is a small price to pay for the huge flock of things we're epsilon short of getting. Cheers, David. -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
On Tue, 2011-02-08 at 11:56 -0500, Robert Haas wrote: > It's a 5400 line patch that wasn't completed until the middle of the > current CommitFest. Nobody has ever submitted a major feature patch > of that size that got done in a single CommitFest, to my recollection, > or even half that size. My concern is that, aside from code, my patch didn't make much progress this commitfest. And the code progress was mostly me working through my own TODO list on things like GiST support -- which didn't represent any real decisions, it was mostly just a matter of code. Regards,Jeff Davis
On Tue, Feb 8, 2011 at 2:02 PM, Jeff Davis <pgsql@j-davis.com> wrote: > On Mon, 2011-02-07 at 22:37 -0500, Robert Haas wrote: >> - Range Types. This is a large patch which was submitted for the >> first time to the last CommitFest of the cycle, and the first version >> that had no open TODO items was posted yesterday, three-quarters of >> the way through that last CommitFest. Some good review has been done. >> While more is probably needed, I think we should feel good about >> what's been accomplished and mark this one Returned with Feedback. > > I submitted this clearly marked WIP, so I expected that it would likely > be pushed to 9.2. > > However, I don't feel like I have the kind of feedback that will help me > get it committed next commitfest. I did get some review, and that was > helpful, but it was mostly on isolated details. > > The patch is a million little decisions: names, catalog structure, > interface, representation, general usability, grammar, functionality, > etc. Without some checkpoint, the chances that everyone agrees with all > of these decisions at the beginning of the next commitfest is zero. > > Is the commitfest not the right place to do this? If not, then when? That's a fair question, and I do understand the difficulty. I think a CommitFest is the right place to do that. On the other hand, as I'm sure you realize, I'm not keen to hold up 9.1beta for a feature that isn't going to be committed until 9.2. In the 9.0 and 9.1 cycles, it was my observation that patches which were submitted to the first or second CommitFest got a lot of this kind of review and went onto be committed without a problem, usually in the next CommitFest. Absorbing patches at the end of the cycle is a lot harder, because everyone who has been thinking about doing something for the release wakes up and submits it, often half-finished, often at the very last minute. Furthermore, it takes more time to review them even if they ARE code-complete, because it takes more time to do a real review-to-commit than it does to tell someone "call it a whisker instead of a potato and delete all the comments about sweet potatoes". I really don't think that getting this into 9.2 is going to be a problem. Given the level of interest in this patch, there will be no problem at all finding someone to do a detailed review when we're not quite so pressed for time, and I'm willing to put some time into it myself when I'm not quite so pressed for time. Although it doesn't feel like it at the moment, we have actually made great strides in absorbing large patches. We've already absorbed true seriailizability and SE-Linux integration (cut down basic version) this release cycle, and it looks like we're going to absorb extensions and hopefully SQL/MED also. Those are all big, big pieces of work *by non-committers*, and the fact that they've sailed in with hardly a cross word is, to me, a very good sign for the future of our development community. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Tue, Feb 8, 2011 at 2:13 PM, David Fetter <david@fetter.org> wrote: >> I agree that we have some problems in that area - particularly with >> writeable CTEs - but prolonging the schedule isn't going to fix that >> problem. > > What is? I think the best solution would probably be to find corporate sponsors for more PostgreSQL committers, or other senior community members who might become committers if they had more time to spend on it. The problem is that there are basically two people who understand the executor well enough to commit that patch: Tom, and Heikki. And Tom doesn't really like the feature (for reasons that are totally baffling to me, BTW). If I had three weeks to spend on that patch, it would be in by now, and I'd know the executor a lot better too, which would make things easier for the next guy who wants to do somethign of that type. But I (with some exceptions) don't get paid to commit other people's patches, so that limits the amount of time I can spend on it to (a) time when I'm not working on something that's a higher priority for my employer and (b) evenings and weekends. I invest as much of that time as I can in community work (which is why I have "nerd" tatooed on my forehead) but there's a limited amount of it, and I tend to invest it in patches where I'm already somewhat familiar with the relevant portion of the code, because that lets me get through more patches, if not always the best patches. > This development cycle was a change from other development cycles in > that it "began" before development had closed in the previous cycle. > I will not take a position at the moment as to the wisdom of making > this change, but it's been clear from feedback from lots of > developers, even ones who'd be expected to be "in the know," that this > vital piece of information did not gotten out in time. The schedule was posted publicly, so I'm not sure how that communication gap happened. > Let's assume for the moment that we're going with overlapping > development cycles into the future. I'd submit that given the > propagation delay of this change, the schedule for the first iteration > of this never was reasonable, and "slipping" two months is a small > price to pay for the huge flock of things we're epsilon short of > getting. I think you're being considerably over-optimistic about how close the remaining patches are to commit, and even more optimistic about the effects of another two months on the quality of the release. Here's my prediction: if we slip the schedule for two months, all the pressure that now exists to get things wrapped up will disappear, and we'll be back in approximately the same place two months from now. A few more things will have been committed, but at least as many new patches will materialize, so that the remaining volume of work is the same as before, or even larger. We'll still end up punting the same number of patches, but in the meantime we'll have managed to rush a few things in that will destabilize the tree and require months of extra beta time during which no new work will be able to be committed. The point is this: We're not going to punt major patches because they are close to commit but yet some arbitrary deadline has expired. We're going to punt them because they're NOT close to commit and a long-agreed deadline has expired. I don't think I'd get much support if I proposed punting everything that isn't yet committed at 2011-02-15 00:00:00, but it's just a mistake to think that if we only wait another day or another week, we'll get it all done. We've tried that before and it usually doesn't work out. The limiting factor isn't primarily the amount of time that it takes to write the code; it's the amount of motivation to get it done. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
pgsql@j-davis.com (Jeff Davis) writes: > On Tue, 2011-02-08 at 06:57 -0500, Stephen Frost wrote: >> * Robert Haas (robertmhaas@gmail.com) wrote: >> > - Range Types. This is a large patch which was submitted for the >> > first time to the last CommitFest of the cycle, and the first version >> > that had no open TODO items was posted yesterday, three-quarters of >> > the way through that last CommitFest. Some good review has been done. >> > While more is probably needed, I think we should feel good about >> > what's been accomplished and mark this one Returned with Feedback. >> >> I don't agree w/ punting Range Types. Range Types were discussed as far >> back as the 2010 developer meeting, were discussed quite a bit again >> starting in October and throughout the fall, and Jeff has regularly >> been posting updates to it. Given how thorough Jeff is, my feeling is >> that this patch is more than ready for beta. > > I appreciate the sentiment, but in addition to some cleanup, any patch > like this at least requires some discussion. It's a language change > we'll be supporting for a long time. > > At minimum, we're a couple hundred emails shy of a real consensus on the > naming ;) It's more than a bit sad... The RangeType change has the massive merit of enabling some substantial development changes, where we can get rid of whole classes of comparison clauses, and hopefully whole classes of range errors. That was my favorite would-be feature for 9.1. It'll take some time to get code changes into systems to use this; the sooner the feature's in a deployable version of Postgres, the earlier that kind of thing may start. -- (format nil "~S@~S" "cbbrowne" "gmail.com") http://www3.sympatico.ca/cbbrowne/internet.html Colorless green ideas sleep furiously. -- Noam Chomsky
> This discussion reveals that it's time to start making some
> discussions about what can be accomplished for 9.1 and what must be
> postponed to 9.2. The big ones I think we should postpone are:
First off, I think that this is a little premature. As others have
pointed out, the unusual schedule this development cycle indicates that
we ought to give patch authors *some* leeway, otherwise we're penalizing
patch authors who worked hard on 9.0 Beta compared to people who ignored
9.0. I don't think that extends to another commitfest, but I would be
OK with extending this commitfest by two weeks to ensure that every
feature gets a full review.
> - Range Types. This is a large patch which was submitted for the
> first time to the last CommitFest of the cycle, and the first version
> that had no open TODO items was posted yesterday, three-quarters of
> the way through that last CommitFest. Some good review has been done.
> While more is probably needed, I think we should feel good about
> what's been accomplished and mark this one Returned with Feedback.
Aside from it needing more feedback, and being personally disappointed,
I think this is inevitable. Jeff seems OK with it too.
> - ALTER EXTENSION UPGRADE. This is another large patch which was
> submitted for the first time to the last CommitFest of the cycle. The
I thought we'd *already* decided to postpone this, due to the spec still
being muddy. No?
> - FOR KEY LOCK tables. This patch, unfortunately, has not gotten a
> lot of review. But it's a major, potentially destabilizing change
I think this needs to get a full review before we make any decisions on it.
> - SQL/MED. This patch unfortunately kind of stalled for a long time.
> However, I believe that Heikki is now working actively on the core
> patch, so I'm hopeful for some activity here soon.
I'll point out that SQL/MED with File_FDW would be a majorly useful
feature, even if other FDWs didn't yet work. So a partial commit is
also a possibility.
> - Writeable CTEs. Tom has promised to look at this, but doesn't seem
> to be in a hurry.
I think it would be tremendously unfair to the author to punt this
feature unless there's fundamental flaws in it. Its development started
back in 9.0, and it's never really gotten enough review/attention.
--Josh Berkus
-- -- Josh Berkus PostgreSQL Experts Inc.
http://www.pgexperts.com
On Tue, 2011-02-08 at 14:25 -0500, Robert Haas wrote: > > The patch is a million little decisions: names, catalog structure, > > interface, representation, general usability, grammar, functionality, > > etc. Without some checkpoint, the chances that everyone agrees with all > > of these decisions at the beginning of the next commitfest is zero. > > > > Is the commitfest not the right place to do this? If not, then when? > > That's a fair question, and I do understand the difficulty. I think a > CommitFest is the right place to do that. On the other hand, as I'm > sure you realize, I'm not keen to hold up 9.1beta for a feature that > isn't going to be committed until 9.2. I'm not asking you to hold it up. Just don't mark it "returned with feedback" when that is not true, and a week still remains. Erik is still looking at it, and that might generate some interesting discussion. > ...everyone who has been thinking about doing something for the release > wakes up and submits it, often half-finished, often at the very last > minute. On the flip side, if we don't provide review to WIP patches during the 3rd commitfest, how do we expect to get anything close to committable on the 1st commitfest of the next cycle? > Although it doesn't > feel like it at the moment, we have actually made great strides in > absorbing large patches. I agree completely. Regards,Jeff Davis
On Tue, 2011-02-08 at 15:10 -0500, Chris Browne wrote: > It's more than a bit sad... The RangeType change has the massive merit > of enabling some substantial development changes, where we can get rid > of whole classes of comparison clauses, and hopefully whole classes of > range errors. That was my favorite would-be feature for 9.1. I appreciate the support. If you take the feature for a quick spin before the next commitfest, that would be a big help. If I get it in the first commitfest of 9.2 that may mean some follow-up features, like RANGE KEYs/FKs, and maybe even RANGE JOIN might have a chance for 9.2 as well. Or, maybe some other features might find it useful, like partitioning or audit logs. Regards,Jeff Davis
pgsql@j-davis.com (Jeff Davis) writes: > On Tue, 2011-02-08 at 15:10 -0500, Chris Browne wrote: >> It's more than a bit sad... The RangeType change has the massive merit >> of enabling some substantial development changes, where we can get rid >> of whole classes of comparison clauses, and hopefully whole classes of >> range errors. That was my favorite would-be feature for 9.1. > > I appreciate the support. > > If you take the feature for a quick spin before the next commitfest, > that would be a big help. If I get it in the first commitfest of 9.2 > that may mean some follow-up features, like RANGE KEYs/FKs, and maybe > even RANGE JOIN might have a chance for 9.2 as well. Or, maybe some > other features might find it useful, like partitioning or audit logs. I've found my "wish item"... I wish that queries could expand ranges in much the same fashion that BETWEEN expands into two query nodes. That way, you can use a range to pick data from a large table, and not revert to a Seq Scan+Filter, which is what I'm seeing for the following sort of query: select * from some_data where '[2010-01-01,2010-02-01)'::daterange @> whensit; -- let name="cbbrowne" and tld="gmail.com" in String.concat "@" [name;tld];; http://linuxfinances.info/info/lsf.html Rules of the Evil Overlord #162. "If I steal something very important to the hero, I will not put it on public display.
On Tue, Feb 8, 2011 at 7:58 PM, Jeff Davis <pgsql@j-davis.com> wrote: > On the flip side, if we don't provide review to WIP patches during the > 3rd commitfest, how do we expect to get anything close to committable on > the 1st commitfest of the next cycle? I'm not sure exactly what you're going for here, because I don't think I've ever proposed any special treatment of patches in the third CommitFest, and as far as I can remember everything got reviewed except for the two Tom promised to pick up and then sat on. But if you were to say that WIP patches *in general* get a lot less review than non-WIP patches, I would agree with you. To some extent, I think that's inevitable. It's not fun to review WIP patches. When you come across something that's screwed up, you say to yourself - I could mention this in the review, but maybe the author already knows about it. After all, it's WIP. In other words, it's hard to know what you should be looking for. I've found that it's nearly always better to post specific questions that you want to know the answer to, rather than a patch where people have to guess what parts you want feedback on. Now, once the patch is code-complete, I think we should review it. And I think we usually do a fairly good job with that, even as late as CF3. It's really CF4 where I think we get a bit less excited about reviewing patches that aren't going to make the release anyway, and that's not a great thing, but as procedural defects go it seems better than most. Yeah, there won't be a lot of big patches committed in 9.2CF1; we don't have the bandwidth to review major patches and get betas out the door at the same time, or at least we haven't in the past. But as long as the big patches that would have made 9.2CF1 still make it into the release, that doesn't seem like a disaster. On the flip side, if a few more people want to step out and help get the open items closed, I'd be more than happy to spend the time that I would otherwise have spent on that reviewing major feature patches for 9.2, where there's a CommitFest in progress or not. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Thu, 2011-02-10 at 09:46 -0500, Robert Haas wrote: > On Tue, Feb 8, 2011 at 7:58 PM, Jeff Davis <pgsql@j-davis.com> wrote: > > On the flip side, if we don't provide review to WIP patches during the > > 3rd commitfest, how do we expect to get anything close to committable on > > the 1st commitfest of the next cycle? > > I'm not sure exactly what you're going for here, because I don't think > I've ever proposed any special treatment of patches in the third > CommitFest, I actually meant 4th (this one). I forgot that the July one was actually a part of the 9.1 cycle. > But if > you were to say that WIP patches *in general* get a lot less review > than non-WIP patches, I would agree with you. > > To some extent, I think that's inevitable. It's not fun to review WIP > patches. Agreed, but it doesn't really apply to this situation. There was still a week left, and the reviewer was still reviewing. So I found it jarring when you said that it had received enough review, and bounced it. In my opinion, if we're going to entertain WIP patches during a commitfest, we shouldn't bounce them early for being WIP. We can bounce them for other causes, like "waiting on author" or "we couldn't find a reviewer" or "we're out of time". > I've found that it's > nearly always better to post specific questions that you want to know > the answer to, rather than a patch where people have to guess what > parts you want feedback on. Well, I've certainly posted some specific questions. I don't expect to get an answer to all of them right away, and certainly many have been answered -- but I didn't just throw the code out and wait. For instance: http://archives.postgresql.org/message-id/1297230650.27157.398.camel@jdavis Anyway, I don't think any of this affected the patch, I was just surprised. I'll leave it at that, because I'm sure you're busy wrapping up this commitfest. Regards,Jeff Davis