Thread: Re: commit fests
> Greg Stark wrote: >> Tom Lane wrote: >> What I think we really need for beta, and could reasonably hope to >> get, is a larger and better-organized beta testing effort. But we >> are not going to get that if people are thinking about new >> development and commit fests instead of testing what's already >> there. > > Incidentally I'm not convinced that's true. The people we really > want testing stuff are the people who have real-world test cases to > throw at the new version and they're the people who will be most > excited about a new release and the least interested in a > commitfest for a version that they won't be able to run for another > year. I tend to agree with you there. The primary risk seems to be that such discussions could distract people who are already working on the release process, not that there is a mob of people who could do much to help with release who will be misdirected into new development. > But I would be happy getting our current process working perfectly > before trying experiments like that. Seriously? *Perfectly*? I'm not even sure what objective metrics you could effectively use to measure success in the process, much less set targets for "perfection" which must be met before trying to solve acknowledged existing problems. Other posts have suggested that "review fests" might be helpful in this period. Again, it sounds to me, from other posts on this thread, as though the primary risk is that people working on the release could see something they couldn't resist getting drawn into -- taking them off-task and delaying the release. The obvious solution to that would be to create a pgsql-journeyman-peer-review list for review fests during the release window. As long as we we all realize that when the big dogs get around to their turn at reviewing things after the release is out it might all have to be redone from scratch, it seems like there could be significant benefit at low risk. -Kevin
Kevin Grittner wrote: > Other posts have suggested that "review fests" might be helpful in > this period. Again, it sounds to me, from other posts on this > thread, as though the primary risk is that people working on the > release could see something they couldn't resist getting drawn into > -- taking them off-task and delaying the release. The obvious > solution to that would be to create a pgsql-journeyman-peer-review > list for review fests during the release window. Be careful, you're wandering quickly down the classic path by which you'll find yourself in charge of doing some work here. I think it's completely reasonable to say that someone could organize pgsql-rrreviewers (as an initial working area, maybe another list eventually) for periodic ReviewFest during periods where those patches won't be considered for commit, such as beta. Now that most patch submitters have gotten used to doing a matching bit of peer review, the pool of people to do the reviews is there without having to pull anyone else into that. I could even see the rrreviewers list or another one split out of it grow into a somewhat gentler place for people to ask for help with their patch development too--just ban all the grumpy people from there (I'll unsubscribe myself). The important thing is that everyone would need to careful to respect not letting that spill over onto this list during the periods there is no official CommitFest going on, or there will be a net increase in said grumpy people. Looking at stats here for the recent CFs, about 40% of patches submitted are returned with feedback (or rejected) rather than being committed anyway. And I'm estimating that >80% of patches only reach comittable after a round of review+corrections first. Getting a lot of that work out of the way outside of the regular CF seems a worthwhile goal. Starting the first CommitFest of the next version (normally a quite painful one) with a set of patches already marked "Ready for Committer" or pruned out with feedback already, all because they've been through a round or two of initial review, would be a nice improvement. Just drop a summary of what's been done onto pgsql-hackers once the CF opens again for the ones still in the running and off you go. The existing CF app could be used to track the early review work too, so someone who wasn't on pgsql-rrreviewers could dig into the archives to catch up in a few minutes by following the message ID links. I do that all the time for patches I had previously been ignoring and deleting out of my mailbox. -- Greg Smith 2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support greg@2ndQuadrant.com www.2ndQuadrant.com
Greg Smith <greg@2ndquadrant.com> wrote: > Kevin Grittner wrote: >> Other posts have suggested that "review fests" might be helpful >> in this period. Again, it sounds to me, from other posts on this >> thread, as though the primary risk is that people working on the >> release could see something they couldn't resist getting drawn >> into -- taking them off-task and delaying the release. The >> obvious solution to that would be to create a >> pgsql-journeyman-peer-review list for review fests during the >> release window. > > Be careful, you're wandering quickly down the classic path by > which you'll find yourself in charge of doing some work here. Worse things could happen. ;-) I've shied away from stepping up to bigger commitments here because of family situations which can make unpredictable demands on my time; however, those seem to have settled down somewhat in recent months, so I might venture such a commitment. > I think it's completely reasonable to say that someone could > organize pgsql-rrreviewers (as an initial working area, maybe > another list eventually) for periodic ReviewFest during periods > where those patches won't be considered for commit, such as beta. > Now that most patch submitters have gotten used to doing a > matching bit of peer review, the pool of people to do the reviews > is there without having to pull anyone else into that. I could > even see the rrreviewers list or another one split out of it grow > into a somewhat gentler place for people to ask for help with > their patch development too--just ban all the grumpy people from > there (I'll unsubscribe myself). LOL. > The important thing is that everyone would need to careful to > respect not letting that spill over onto this list during the > periods there is no official CommitFest going on, or there will be > a net increase in said grumpy people. Understood. > Looking at stats here for the recent CFs, about 40% of patches > submitted are returned with feedback (or rejected) rather than > being committed anyway. And I'm estimating that >80% of patches > only reach comittable after a round of review+corrections first. > Getting a lot of that work out of the way outside of the regular > CF seems a worthwhile goal. > > Starting the first CommitFest of the next version (normally a > quite painful one) with a set of patches already marked "Ready for > Committer" or pruned out with feedback already, all because > they've been through a round or two of initial review, would be a > nice improvement. Just drop a summary of what's been done onto > pgsql-hackers once the CF opens again for the ones still in the > running and off you go. The existing CF app could be used to track > the early review work too, so someone who wasn't on > pgsql-rrreviewers could dig into the archives to catch up in a > few minutes by following the message ID links. I do that all the > time for patches I had previously been ignoring and deleting out > of my mailbox. I see all those benefits, plus the possibility of a few more subtle but potentially significant ones. What next? -Kevin
Robert Haas <robertmhaas@gmail.com> wrote: > Issues that are discussed and resolved in that forum will very > possibly get brought up again on -hackers and reach a different > conclusion the second time around. Now if people are OK with > that, maybe it's OK I would expect that. Heck, people get that to some degree posting twice to hackers, depending on who pays how much attention each time. To some extent it would depend on how far those doing the reviews would stretch their limits to provide a good review, knowing that someone might do a lot of work based on their advice before a senior PostgreSQL developer will put eyes on it. The expansion of the talent pool that could result is one of the possible "hidden benefits" which might accrue. Other possible benefits could come from simple "sounding board" discussions, non-critical "brainstorming" rounds of discussion, and the synergy of disparate views. Of course, all this assumes that people maintain the appropriate demeanor and put some thought and effort into their posts. > but I have a feeling that could be even more frustrating than the > system we have now. Possibly. We'd certainly need prominent caveats, so that people had the right expectations going in. -Kevin
On Mon, Jan 25, 2010 at 12:01 PM, Greg Smith <greg@2ndquadrant.com> wrote: > I think it's completely reasonable to say that someone could organize > pgsql-rrreviewers (as an initial working area, maybe another list > eventually) for periodic ReviewFest during periods where those patches won't > be considered for commit, such as beta. Now that most patch submitters have > gotten used to doing a matching bit of peer review, the pool of people to do > the reviews is there without having to pull anyone else into that. I could > even see the rrreviewers list or another one split out of it grow into a > somewhat gentler place for people to ask for help with their patch > development too--just ban all the grumpy people from there (I'll unsubscribe > myself). The important thing is that everyone would need to careful to > respect not letting that spill over onto this list during the periods there > is no official CommitFest going on, or there will be a net increase in said > grumpy people. I kind of like the ideal of a pgsql-fledgling-hackers list (that's intended to describe the charter, not that we'd necessarily call it that) as a space for people to ask novice development questions (as opposed to novice user questions). I'm less certain I like the idea that some part of patch review is going to happen off of pgsql-hackers. If people like Tom, Bruce, Peter, and Heikki are not reading the list where this review is happening (whether it's pgsql-rrreviewers, pgsql-fledgling-hackers, or otherwise), it may not actually be very successful in helping people move their patches forward. Issues that are discussed and resolved in that forum will very possibly get brought up again on -hackers and reach a different conclusion the second time around. Now if people are OK with that, maybe it's OK: but I have a feeling that could be even more frustrating than the system we have now. I am in favor of having a ReviewFest if we go too long without being able to have a CommitFest, but I would personally prefer to see it happen on -hackers. I wouldn't argue for scheduling a ReviewFest as soon as March, because I think we really do need to allow some time and resources for resolving open issues, and I think having another CommitFest so soon will distract from that. But later on in the release cycle, I think it would make sense - the work that can be done by people other than Tom and Bruce and Peter and whoever the other big guns are is going to be mostly done by then. I suspect that a ReviewFest in, say, May would give us a useful jump on 9.1 development without having much impact on getting 9.0 out the door. ...Robert