Thread: Well done, Hackers

Well done, Hackers

From
Simon Riggs
Date:
Unofficially, well done to everybody that submitted patches in time for
Commit Fest. The queue has grown significantly over last few days and
it's clear many people have been working very hard right up to the
deadline. 

Depending upon how you count it there are around 5 multi-month mega
patches, plus more than 50 other features. Very impressive.

I count nearly a dozen new names on the list. Congratulations to you.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support



Re: Well done, Hackers

From
Tom Lane
Date:
Simon Riggs <simon@2ndQuadrant.com> writes:
> Unofficially, well done to everybody that submitted patches in time for
> Commit Fest. The queue has grown significantly over last few days and
> it's clear many people have been working very hard right up to the
> deadline. 

"Well done"?  It seems to me that we are right where we hoped not to be,
ie with a ton of barely-completed (if not self-admitted WIP) patches
dropped on us immediately before feature freeze.  Today the commit fest
idea is looking like a failure.

If we actually manage to ship 8.4 within six months, *that* will be
"well done".
        regards, tom lane


Re: Well done, Hackers

From
Simon Riggs
Date:
On Sat, 2008-11-01 at 14:22 -0400, Tom Lane wrote:
> Simon Riggs <simon@2ndQuadrant.com> writes:
> > Unofficially, well done to everybody that submitted patches in time for
> > Commit Fest. The queue has grown significantly over last few days and
> > it's clear many people have been working very hard right up to the
> > deadline. 
> 
> "Well done"?  It seems to me that we are right where we hoped not to be,
> ie with a ton of barely-completed (if not self-admitted WIP) patches
> dropped on us immediately before feature freeze.  Today the commit fest
> idea is looking like a failure.
> 
> If we actually manage to ship 8.4 within six months, *that* will be
> "well done".

Work happens in stages. A lot of people have obviously worked very hard
and I wanted to make sure their efforts do not go unnoticed.

I don't see contributions and effort as a bad thing for Postgres. And
CommitFests have helped in reducing the volume of patches that appear at
the last deadline, just imagine what it would be like otherwise. If
there was a problem anywhere in sight it would be that the development
window closes for 6 months of the year, forcing developers to cram their
contributions into a tight window. Patches that take many months to plan
and write are simply more likely to appear at the last Commitfest of the
year.

I'm more than aware that much of the work from here rests on your
shoulders, and I understand how you must feel. But let's give credit to
the people that have worked hard to get to here and then lets give
*more* credit to all the people that contribute from here. We can say
thank you to each in their turn. If we can positively motivate people we
will be able to spread the load rather than have the load fall on you.
Believe me, none of us enjoy watching you struggle (capably) with the
huge workload and responsibility. I'll do more than I've done in the
past to help, both with my own work and others. Thanks in advance.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support



Re: Well done, Hackers

From
Emmanuel Cecchet
Date:
Tom,

> "Well done"? It seems to me that we are right where we hoped not to be,
> ie with a ton of barely-completed (if not self-admitted WIP) patches
> dropped on us immediately before feature freeze.  Today the commit fest
> idea is looking like a failure.
>   
If you don't want patches coming at given deadlines then yes the commit 
fest idea is a bad idea altogether. But what is the real issue?
- The core team is too small to absorb contributions or the development 
is not distributed enough?
- There are not enough guidelines or requirements for a patch to make it 
to the commit fest?
- There is not enough QA manpower/test cases to test the patches 
efficiently?
- Lack of roadmap? Hard to guess what contributions are going to come?

What are your views on how the process could be improved? If the commit 
fest does not work, what should we do instead?

I think that complaining after volunteer contributors is the best way to 
not get any more contributions and have nice empty commit fests in the 
future. Maybe that's the way to go to solve the issue at hand! ;-)

manu

-- 
Emmanuel Cecchet
FTO @ Frog Thinker 
Open Source Development & Consulting
--
Web: http://www.frogthinker.org
email: manu@frogthinker.org
Skype: emmanuel_cecchet



Re: Well done, Hackers

From
"Robert Haas"
Date:
> "Well done"?  It seems to me that we are right where we hoped not to be,
> ie with a ton of barely-completed (if not self-admitted WIP) patches
> dropped on us immediately before feature freeze.  Today the commit fest
> idea is looking like a failure.
>
> If we actually manage to ship 8.4 within six months, *that* will be
> "well done".

It looks to me like there are at least half a dozen patches submitted
in the last week that are pretty half-baked and fall into the category
of "Let's submit something before the deadline for CommitFest, in the
hopes of being allowed to finish it later."  This completely shafts
the process in two ways.  First, anyone who gets assigned to review
one of those patches might as well not bother, since the author is
probably still working frantically on the patch anyway and will likely
find and fix a lot of the issues that any review might turn up.
Second, since the authors are frantically working on their own
patches, they will have no (or diminished) time to review other
people's patches, which is the whole point of CommitFest.  It seems to
me that "Work In Progress" needs to mean "I need some feedback this
CommitFest so I can finish it for the NEXT CommitFest" and not "I'd
like to make an end-run around the submission deadline."

On the other hand, the number of patches that fall into this category
is actually not that large as a percentage of the total.  A lot of the
larger features have been under development for months and have been
extensively discussed on -hackers or were submitted for the previous
CommitFest.  If you could somehow wave your magic wand and get all of
those committed or rejected, I doubt the remaining list would be
terribly intimidating.  I have five patches in for this commitfest but
I bet you (tgl) could deal with all of them in an afternoon without
breaking a sweat.

...Robert


Re: Well done, Hackers

From
Simon Riggs
Date:
On Sat, 2008-11-01 at 16:21 -0400, Robert Haas wrote:
> > "Well done"?  It seems to me that we are right where we hoped not to be,
> > ie with a ton of barely-completed (if not self-admitted WIP) patches
> > dropped on us immediately before feature freeze.  Today the commit fest
> > idea is looking like a failure.
> >
> > If we actually manage to ship 8.4 within six months, *that* will be
> > "well done".
> 
> It looks to me like there are at least half a dozen patches submitted
> in the last week that are pretty half-baked

Robert,

Without review, how can we determine the true state of a patch? I would
guess that you haven't done a review of all of them yourself to
determine that. 

Your characterisation of other developers seems negative to me and I
really do hope unfair. Nobody I know has submitted anything half-baked,
but I haven't reviewed any patches yet. Review is about asking for help
from your peers, not an admission of crap software. We're all seeking to
learn more and be better, I hope.

If this sounds like a flame, its not. I just want to be positive about
the efforts of so many people who are all working together as a team.
There will be some hard, honest reviews and some patches may not make
it, so its a good time to say thanks for trying so hard.

For myself, I've submitted a patch touching more than 60 files. Am I
nervous I got something wrong? Damn right. Is there some horror lurking
in there that I'm not saying, but frantically fixing now? No way. That's
not the spirit.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support



Re: Well done, Hackers

From
Bruce Momjian
Date:
Emmanuel Cecchet wrote:
> Tom,
> 
> > "Well done"? It seems to me that we are right where we hoped not to be,
> > ie with a ton of barely-completed (if not self-admitted WIP) patches
> > dropped on us immediately before feature freeze.  Today the commit fest
> > idea is looking like a failure.
> >   
> If you don't want patches coming at given deadlines then yes the commit 
> fest idea is a bad idea altogether. But what is the real issue?
> - The core team is too small to absorb contributions or the development 
> is not distributed enough?

The list of people experienced enough to review complex patches has not
grown as fast as the number of people submitting complex patches.  That
will probably even out over time, but it might take years.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Well done, Hackers

From
Bruce Momjian
Date:
Simon Riggs wrote:
> 
> On Sat, 2008-11-01 at 16:21 -0400, Robert Haas wrote:
> > > "Well done"?  It seems to me that we are right where we hoped not to be,
> > > ie with a ton of barely-completed (if not self-admitted WIP) patches
> > > dropped on us immediately before feature freeze.  Today the commit fest
> > > idea is looking like a failure.
> > >
> > > If we actually manage to ship 8.4 within six months, *that* will be
> > > "well done".
> > 
> > It looks to me like there are at least half a dozen patches submitted
> > in the last week that are pretty half-baked
> 
> Robert,
> 
> Without review, how can we determine the true state of a patch? I would
> guess that you haven't done a review of all of them yourself to
> determine that. 

I think a complex patch that hasn't been discussed on the hackers list
recently, and that has deficiencies reported by the author can be safely
described as a work-in-progress that needs feedback, rather than
committed for 8.4.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Well done, Hackers

From
Andrew Sullivan
Date:
On Sat, Nov 01, 2008 at 04:21:30PM -0400, Robert Haas wrote:

> It looks to me like there are at least half a dozen patches submitted
> in the last week that are pretty half-baked and fall into the category
> of "Let's submit something before the deadline for CommitFest, in the
> hopes of being allowed to finish it later."  

Supposing this is true, one way to prevent that in future
commitfest-driven releases is to send them back as not ready this
time, and tell them they'll be 1st in line in the next go-round.

It's worth remebering that this is the first release using the
commitfest model, so there will be things to learn from the 1.0
attempt.

A

-- 
Andrew Sullivan
ajs@commandprompt.com
+1 503 667 4564 x104
http://www.commandprompt.com/


Re: Well done, Hackers

From
"Robert Haas"
Date:
> Without review, how can we determine the true state of a patch? I would
> guess that you haven't done a review of all of them yourself to
> determine that.

No, I'm talking about the comments that were made when submitting the
patch.  A few people said things along the lines of "this isn't really
finished" or "this has major bugs".

> Your characterisation of other developers seems negative to me and I
> really do hope unfair. Nobody I know has submitted anything half-baked,
> but I haven't reviewed any patches yet. Review is about asking for help
> from your peers, not an admission of crap software. We're all seeking to
> learn more and be better, I hope.

Sorry.  I don't mean to be negative, and certainly not about the
developers.  I'm actually quite impressed by the number of people
submitting patches, and I'm really looking forward to some of the
proposed new features.  My point is just that about 10% of them sound
like they aren't actually done.

And as I said in my previous email, I don't even think that's the main issue.

> For myself, I've submitted a patch touching more than 60 files. Am I
> nervous I got something wrong? Damn right. Is there some horror lurking
> in there that I'm not saying, but frantically fixing now? No way. That's
> not the spirit.

I wasn't talking about your patches.  :-)

...Robert


Re: Well done, Hackers

From
"Robert Haas"
Date:
> Supposing this is true, one way to prevent that in future
> commitfest-driven releases is to send them back as not ready this
> time, and tell them they'll be 1st in line in the next go-round.

Agreed!

But I'll be as happy as anyone if it doesn't come to that.

...Robert


Re: Well done, Hackers

From
Josh Berkus
Date:
Tom, Robert, Simon,

What, are people just on edge because of the US election?

It looks to me like the commitfest system is going really well.  Of 
course, we'll see how long it takes to close out 8.4.  But I think we're 
in much better shape than we were for 8.3.  We're even in better shape 
to reject things.

--Josh


Re: Well done, Hackers

From
"Joshua D. Drake"
Date:
Josh Berkus wrote:
> Tom, Robert, Simon,
> 
> What, are people just on edge because of the US election?
> 
> It looks to me like the commitfest system is going really well.  Of 
> course, we'll see how long it takes to close out 8.4.  But I think we're 
> in much better shape than we were for 8.3.  We're even in better shape 
> to reject things.

I have to agree with Berkus here. Compared to how we used to do things, 
this is quite marvelous.

Joshua D. Drake


> 
> --Josh
> 



Re: Well done, Hackers

From
Dave Page
Date:

Sent from my iPhone

On 2 Nov 2008, at 06:57, "Joshua D. Drake" <jd@commandprompt.com> wrote:

> Josh Berkus wrote:
>> Tom, Robert, Simon,
>> What, are people just on edge because of the US election?
>> It looks to me like the commitfest system is going really well.  Of  
>> course, we'll see how long it takes to close out 8.4.  But I think  
>> we're in much better shape than we were for 8.3.  We're even in  
>> better shape to reject things.
>
> I have to agree with Berkus here. Compared to how we used to do  
> things, this is

Yeah, another +1 from here. As I see it we have:

- Given developers timely feedback throughout the cycle, so no one has  
had to wait 6 months+.

- Committed new patches in a timely fashion, avoidind bit rot of those  
and subsequent patches they affect.

- Significantly reduced the amount of work we would have previously  
seen at feature freeze.

I would also note that this last commit fest is really feature freeze,  
and should be treated as such. That means that it's the cut off point  
for new features which are essentially ready, not that code must be  
100% complete and tested. I doubt there are many big patches that  
haven't require work after freeze.

For future clarity, perhaps we should have a feature freeze point,  
followed a  'polishing freeze' by which time the submitter must finish  
pre-review cleanups and last minute fixes.

/D 


Re: Well done, Hackers

From
Martijn van Oosterhout
Date:
On Sat, Nov 01, 2008 at 08:07:13PM -0400, Andrew Sullivan wrote:
> On Sat, Nov 01, 2008 at 04:21:30PM -0400, Robert Haas wrote:
>
> > It looks to me like there are at least half a dozen patches submitted
> > in the last week that are pretty half-baked and fall into the category
> > of "Let's submit something before the deadline for CommitFest, in the
> > hopes of being allowed to finish it later."
>
> Supposing this is true, one way to prevent that in future
> commitfest-driven releases is to send them back as not ready this
> time, and tell them they'll be 1st in line in the next go-round.

With some feedback, hopefully. I imagine a number of people submit
patches because they hope to get some feedback on them, something which
does always happen at other times...

I think all this development is good.

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> Please line up in a tree and maintain the heap invariant while
> boarding. Thank you for flying nlogn airlines.