Thread: CommitFest July Over

Hackers,

Well, after a month the July CommitFest is officially closed.  At this 
point, we're operating with the defacto rule that commitfests shouldn't 
last more than a month.

Because some patches are still being discussed, they've been moved over 
automatically to the September commitfest.  A much large number of 
patches are now in "returned with feedback"; if your patch is in there, 
probably hackers is waiting for some kind of response from you.

Lots of stuff was committed, too.  8.4 is looking very exciting.

Post-mortem things we've learned about the commitfest are:

1) It's hard to get anything done in June-July.

2) The number of patches is going to keep increasing with each 
commitfest.  As such, the patch list is going to get harder to deal 
with.  We now urgently need to start working on CF management software.

3) Round Robin Reviewers didn't really work this time, aside from 
champion new reviewer Abhjit.  For the most part, RRR who were assigned 
patches did not review them for 2 weeks.  Two areas where this concept 
needs to be improved:a) we need to assign RRR to patches two days after the start of 
commitfest, not a week later;b) there needs to be the expectation that RRR will start reviewing or 
reject the assignment immediately.

4) We need to work better to train up new reviewers.  Some major 
committer(s) should have worked with Abhjit, Thomas and Martin 
particularly on getting them to effectively review patches; instead, 
committers just handled stuff *for* them for the most part, which isn't 
growing our pool of reviewers.

5) Patch submitters need to understand that patch submission isn't 
fire-and-forget.  They need to check back, and respond to queries from 
reviewers.  Of course, a patch-tracker which automatically notified the 
submitter would help.

6) Overall, I took a low-nag-factor approach to the first time as 
commitfest manager.  This does not seem to have been the best way; I'd 
suggest for september that the manager make more frequent nags.

Finally: who wants to be CF Manager for September?  I'm willing to do it 
again, but maybe someone else should get a turn.

--Josh

On Monday 04 August 2008 15:38:35 Josh Berkus wrote:
> Hackers,
>
> Well, after a month the July CommitFest is officially closed.  At this
> point, we're operating with the defacto rule that commitfests shouldn't
> last more than a month.
>
> Because some patches are still being discussed, they've been moved over
> automatically to the September commitfest.  A much large number of
> patches are now in "returned with feedback"; if your patch is in there,
> probably hackers is waiting for some kind of response from you.
>

People should understand they don't have to wait for a commitfest to continue 
development, right? (Ie. if your patch got rejected, start getting it in 
shape now, and ask questions now)

> Lots of stuff was committed, too.  8.4 is looking very exciting.
>

+1

> Post-mortem things we've learned about the commitfest are:
>
> 1) It's hard to get anything done in June-July.
>

True... vacations and conferences abound. September should be better in this 
regard I would think. 

> 2) The number of patches is going to keep increasing with each
> commitfest.  As such, the patch list is going to get harder to deal
> with.  We now urgently need to start working on CF management software.
>
> 3) Round Robin Reviewers didn't really work this time, aside from
> champion new reviewer Abhjit.  For the most part, RRR who were assigned
> patches did not review them for 2 weeks.  Two areas where this concept
> needs to be improved:
>     a) we need to assign RRR to patches two days after the start of
> commitfest, not a week later;

This seems tricky, since you want people to volunteer to review patches 
ideally, will two days be enough? Should people interested in reviewing be 
signing up ahead of time? Looking at the next commitfest, it is going to 
start on a Monday... maybe auto-assigning reviewers on Wednesday is OK. 

>     b) there needs to be the expectation that RRR will start reviewing or
> reject the assignment immediately.
>

I wonder if too much time was spent on patches like the WITH patch, which 
seemed pretty early on it was not ready for commit... thoughts? 

> 4) We need to work better to train up new reviewers.  Some major
> committer(s) should have worked with Abhjit, Thomas and Martin
> particularly on getting them to effectively review patches; instead,
> committers just handled stuff *for* them for the most part, which isn't
> growing our pool of reviewers.
>
> 5) Patch submitters need to understand that patch submission isn't
> fire-and-forget.  They need to check back, and respond to queries from
> reviewers.  Of course, a patch-tracker which automatically notified the
> submitter would help.
>

Reviewers should be responding to the email on -hackers that is pointed to by 
the wiki, so patch submitters should be getting notified... right ?

> 6) Overall, I took a low-nag-factor approach to the first time as
> commitfest manager.  This does not seem to have been the best way; I'd
> suggest for september that the manager make more frequent nags.
>

Is there something you want people to nag people about? 

> Finally: who wants to be CF Manager for September?  I'm willing to do it
> again, but maybe someone else should get a turn.
>

Why stop now when you've got the momentum? :-) 

Seriously though, I thought we were supposed to have 2 people working as CF 
Managers for each CF... is that not the case? 

-- 
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL

Robert Treat wrote:
> On Monday 04 August 2008 15:38:35 Josh Berkus wrote:
>> Post-mortem things we've learned about the commitfest are:
>>
>> 1) It's hard to get anything done in June-July.
>>
> 
> True... vacations and conferences abound. September should be better in this 
> regard I would think. 

Um. Looking at my calendar, the second half of september and all of
october is packed solid with conferences. Unlike June, July & August
which were completely empty.

Perhaps it's a US vs EU thing?

(Vacations are July/August though, so that matches)


>> 2) The number of patches is going to keep increasing with each
>> commitfest.  As such, the patch list is going to get harder to deal
>> with.  We now urgently need to start working on CF management software.
>>
>> 3) Round Robin Reviewers didn't really work this time, aside from
>> champion new reviewer Abhjit.  For the most part, RRR who were assigned
>> patches did not review them for 2 weeks.  Two areas where this concept
>> needs to be improved:
>>     a) we need to assign RRR to patches two days after the start of
>> commitfest, not a week later;
> 
> This seems tricky, since you want people to volunteer to review patches 
> ideally, will two days be enough? Should people interested in reviewing be 
> signing up ahead of time? Looking at the next commitfest, it is going to 
> start on a Monday... maybe auto-assigning reviewers on Wednesday is OK. 

Um, didn't they already sign up ahead of time? We can't very well hand
out patches to someone who's not interested, can we?


>>     b) there needs to be the expectation that RRR will start reviewing or
>> reject the assignment immediately.
>>
> 
> I wonder if too much time was spent on patches like the WITH patch, which 
> seemed pretty early on it was not ready for commit... thoughts? 

I think that happens a lot. Once discussion "takes off" on a patch, it
attracts more people to comment on it, etc.

Plus the whole "hey, i've added a git repo" starts it's own thread :-P


>> 4) We need to work better to train up new reviewers.  Some major
>> committer(s) should have worked with Abhjit, Thomas and Martin
>> particularly on getting them to effectively review patches; instead,
>> committers just handled stuff *for* them for the most part, which isn't
>> growing our pool of reviewers.

True.


>> 5) Patch submitters need to understand that patch submission isn't
>> fire-and-forget.  They need to check back, and respond to queries from
>> reviewers.  Of course, a patch-tracker which automatically notified the
>> submitter would help.
>>
> 
> Reviewers should be responding to the email on -hackers that is pointed to by 
> the wiki, so patch submitters should be getting notified... right ?

Well, there's really no way to easily do that. I mean, you can't hit
"reply" once you find something in the archives. You'll need to manually
put everybody back in the CC list, so it's much easier to just post to
-hackers.

Thus, I think requiring the submitters to check back on -hackers
regularly is necessary, for now.


>> 6) Overall, I took a low-nag-factor approach to the first time as
>> commitfest manager.  This does not seem to have been the best way; I'd
>> suggest for september that the manager make more frequent nags.

Yes, agreed. The manager role was fairly invisible this time around, I
think we should at least try and see what happens.


>> Finally: who wants to be CF Manager for September?  I'm willing to do it
>> again, but maybe someone else should get a turn.
>>
> 
> Why stop now when you've got the momentum? :-) 
> 
> Seriously though, I thought we were supposed to have 2 people working as CF 
> Managers for each CF... is that not the case? 

Umm, IIRC we said one, but we'd rotate.

That said, I think it'd be a good idea if Josh continued across the next
one, given that this one was more or less a "trial run" for the CF
Manager thingy. We can start switching once the role is a bit more
defined. (This is all based on the fact that Josh says he's ok with
doing it, of course :-P)

//Magnus

Hi,

Josh Berkus wrote:
> 2) The number of patches is going to keep increasing with each 
> commitfest.  As such, the patch list is going to get harder to deal 
> with.  We now urgently need to start working on CF management software.

Agreed.

> 3) Round Robin Reviewers didn't really work this time, aside from 
> champion new reviewer Abhjit.  For the most part, RRR who were assigned 
> patches did not review them for 2 weeks.  Two areas where this concept 
> needs to be improved:
>     a) we need to assign RRR to patches two days after the start of 
> commitfest, not a week later;

Maybe it's just me, but I don't quite understand the concept of RRR. If 
I get some spare cycles to review patches, I certainly want to choose 
mysqlf which patch I'm going to review. Why is the CF Manager doing any 
assignment of patches?

Of course, the reviewers need to coordinate, it doesn't make much sense 
for seven people concurrently reviewing the same patch. But shouldn't 
the reviewer take care of 'tagging' a patch as being reviewed?

Or do you think it's motivating to get nagged about accepting or 
rejecting a patch assignment? For my part, it's been the main reason I 
didn't sign up as an RRR: I didn't want to get into that situation. On 
the other hand, I must admit that I didn't review any of the outstanding 
patches either...

Regards

Markus Wanner

On Tuesday 05 August 2008 04:36:24 Magnus Hagander wrote:
> Robert Treat wrote:
> > On Monday 04 August 2008 15:38:35 Josh Berkus wrote:
> >> Post-mortem things we've learned about the commitfest are:
> >>
> >> 1) It's hard to get anything done in June-July.
> >
> > True... vacations and conferences abound. September should be better in
> > this regard I would think.
>
> Um. Looking at my calendar, the second half of september and all of
> october is packed solid with conferences. Unlike June, July & August
> which were completely empty.
>
> Perhaps it's a US vs EU thing?
>
> (Vacations are July/August though, so that matches)
>

Hmm... Pg.br is the only thing I could think of in September, which I don't 
think involves either of us, unless you're going on yet another world 
adventure ;-)

I do agree that October looks packed, I'm glad we're not doing a commitfest 
then. 

> >> 2) The number of patches is going to keep increasing with each
> >> commitfest.  As such, the patch list is going to get harder to deal
> >> with.  We now urgently need to start working on CF management software.
> >>
> >> 3) Round Robin Reviewers didn't really work this time, aside from
> >> champion new reviewer Abhjit.  For the most part, RRR who were assigned
> >> patches did not review them for 2 weeks.  Two areas where this concept
> >> needs to be improved:
> >>     a) we need to assign RRR to patches two days after the start of
> >> commitfest, not a week later;
> >
> > This seems tricky, since you want people to volunteer to review patches
> > ideally, will two days be enough? Should people interested in reviewing
> > be signing up ahead of time? Looking at the next commitfest, it is going
> > to start on a Monday... maybe auto-assigning reviewers on Wednesday is
> > OK.
>
> Um, didn't they already sign up ahead of time? We can't very well hand
> out patches to someone who's not interested, can we?
>

ISTR, and Josh's info above indicated, that after a week or so, patches with 
no volunteers simply got assigned to someone. Josh, want to confirm. 

> >>     b) there needs to be the expectation that RRR will start reviewing or
> >> reject the assignment immediately.
> >
> > I wonder if too much time was spent on patches like the WITH patch, which
> > seemed pretty early on it was not ready for commit... thoughts?
>
> I think that happens a lot. Once discussion "takes off" on a patch, it
> attracts more people to comment on it, etc.
>
> Plus the whole "hey, i've added a git repo" starts it's own thread :-P
>

I have a git repo for ppa, want to do some hacking? :-)

> >> 4) We need to work better to train up new reviewers.  Some major
> >> committer(s) should have worked with Abhjit, Thomas and Martin
> >> particularly on getting them to effectively review patches; instead,
> >> committers just handled stuff *for* them for the most part, which isn't
> >> growing our pool of reviewers.
>
> True.
>
> >> 5) Patch submitters need to understand that patch submission isn't
> >> fire-and-forget.  They need to check back, and respond to queries from
> >> reviewers.  Of course, a patch-tracker which automatically notified the
> >> submitter would help.
> >
> > Reviewers should be responding to the email on -hackers that is pointed
> > to by the wiki, so patch submitters should be getting notified... right ?
>
> Well, there's really no way to easily do that. I mean, you can't hit
> "reply" once you find something in the archives. You'll need to manually
> put everybody back in the CC list, so it's much easier to just post to
> -hackers.
>

Ah, I keep a healthy backlog of email sent to hackers, so if I want to respond 
to a patch, I find it in my email program and reply to that, with CC 
list/threading intact. 

> Thus, I think requiring the submitters to check back on -hackers
> regularly is necessary, for now.
>

Well, probably a good idea anyway, I certainly don't want to discourage it.  

-- 
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL

Markus,

> Maybe it's just me, but I don't quite understand the concept of RRR. If 
> I get some spare cycles to review patches, I certainly want to choose 
> mysqlf which patch I'm going to review. Why is the CF Manager doing any 
> assignment of patches?

This was actually by request of several reviewers, who said they could 
do something but didn't have a preference which patch to review.

--Josh