Thread: Assisting developers

Assisting developers

From
Bruce Momjian
Date:
One failing that has appeared during the 7.5 development cycle is that
we as a community haven't been able to provide timely feedback to
developers working on large feature additions.

I am particularly thinking of Alvaro (nested transactions) and Simon
(PITR), where we haven't been able to give them sufficient feedback to
make them fully productive.

I am not sure what can be done to solve this in the future.  There are
only a limited number of us who have the experience and time to review
and comment on very complex patches.

Hopefully this is just growing pains and the community will grow to the
point where we can have more people focused on assisting developers
adding complex features.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: Assisting developers

From
Christopher Kings-Lynne
Date:
> One failing that has appeared during the 7.5 development cycle is that
> we as a community haven't been able to provide timely feedback to
> developers working on large feature additions.
> 
> I am particularly thinking of Alvaro (nested transactions) and Simon
> (PITR), where we haven't been able to give them sufficient feedback to
> make them fully productive.
> 
> I am not sure what can be done to solve this in the future.  There are
> only a limited number of us who have the experience and time to review
> and comment on very complex patches.
> 
> Hopefully this is just growing pains and the community will grow to the
> point where we can have more people focused on assisting developers
> adding complex features.

Well, I note (and I'm not being unkind or anything here) that a lot of 
the high level committers we have haven't been so active this release. 
Peter and Joe haven't been around much and Jan has been busy with Slony.  We also lost Thomas Lockhart.  Neil's also
awayon holidays.  You and 
 
Tom have basically been doing all the reviewing - a great job - but I 
can't believe Tom hasn't cracked yet :P

I've been around for years, but I've never really gotten into the depths 
of things, so I'm not much use for checking complex patches, but I'm 
willing to review simpler stuff :)

Maybe you should promote a new committer?  (Not me!)

Chris



Re: Assisting developers

From
Bruce Momjian
Date:
Christopher Kings-Lynne wrote:
> > One failing that has appeared during the 7.5 development cycle is that
> > we as a community haven't been able to provide timely feedback to
> > developers working on large feature additions.
> > 
> > I am particularly thinking of Alvaro (nested transactions) and Simon
> > (PITR), where we haven't been able to give them sufficient feedback to
> > make them fully productive.
> > 
> > I am not sure what can be done to solve this in the future.  There are
> > only a limited number of us who have the experience and time to review
> > and comment on very complex patches.
> > 
> > Hopefully this is just growing pains and the community will grow to the
> > point where we can have more people focused on assisting developers
> > adding complex features.
> 
> Well, I note (and I'm not being unkind or anything here) that a lot of 
> the high level committers we have haven't been so active this release. 
> Peter and Joe haven't been around much and Jan has been busy with Slony. 
>   We also lost Thomas Lockhart.  Neil's also away on holidays.  You and 
> Tom have basically been doing all the reviewing - a great job - but I 
> can't believe Tom hasn't cracked yet :P

Excellent analysis.

> I've been around for years, but I've never really gotten into the depths 
> of things, so I'm not much use for checking complex patches, but I'm 
> willing to review simpler stuff :)
> 
> Maybe you should promote a new committer?  (Not me!)

The committing isn't really the issue.  It is reviewing and giving
feedback to developers of complex features.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: Assisting developers

From
Marko Karppinen
Date:
On Jul 13, 2004, at 17:02, Bruce Momjian wrote:
> One failing that has appeared during the 7.5 development cycle is that
> we as a community haven't been able to provide timely feedback to
> developers working on large feature additions.
>
> I am particularly thinking of Alvaro (nested transactions) and Simon
> (PITR), where we haven't been able to give them sufficient feedback to
> make them fully productive.

I'm just a bystander here, but it seems to me that in-depth discussion
of a feature only starts when someone realizes that he must speak now
or the darn thing might get committed. In other words, the emphasis is
placed in preventing something half-baked getting included. And that's
perfectly natural because it is much easier and quicker than commenting
thoughtfully on every idea that someone might come up with.

But it of course means that the price of admission is a patch that
poses a real risk of getting committed.
From a pure resource utilization perspective, I don't see a way around
this. There's not enough expertise of pgsql internals to go around. As
long as that's the case, there will always be a barrier to entry. But
a high-risk patch isn't the only thing that can get you over such
a barrier; the only thing to control the distribution of this scarce
resource.

Cash comes to mind as an alternative.

mk



Re: Assisting developers

From
Peter Eisentraut
Date:
Bruce Momjian wrote:
> I am not sure what can be done to solve this in the future.  There
> are only a limited number of us who have the experience and time to
> review and comment on very complex patches.

The issue as I see it is not reviewing patches, but defining features.  
Someone sets out to develop "nested transactions", and three days after 
feature freeze we have the first large discussion about what nested 
transactions really are, what they are good for, and how they should 
work.  Maybe next time think more about the old requirements, design, 
implementation, testing cycle.  Of course people did post plans, status 
updates, etc., but maybe it wasn't enough, not clear enough, or 
something else.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/



Re: Assisting developers

From
Josh Berkus
Date:
Chris,  all:

> Well, I note (and I'm not being unkind or anything here) that a lot of
> the high level committers we have haven't been so active this release.
> Peter and Joe haven't been around much and Jan has been busy with Slony.
>   We also lost Thomas Lockhart.  Neil's also away on holidays.  You and
> Tom have basically been doing all the reviewing - a great job - but I
> can't believe Tom hasn't cracked yet :P

Actually, this is a *big* part of the problem.   In 7.3, by unplanned 
circumstance, we got into a cycle of doing feature freeze and starting beta 
during the summer.   This is a bad cycle -- the Europeans take their 3-week 
vacations, the Americans go to (and spend many hours preparing for) a bunch 
of conventions, Students go to internships, people take long weekends, get 
married, etc.   ( This is probably why many American corporations end their 
fiscal year in midsummer; nothing is going to get done anyway, might as well 
clean up. )

The result is that we chronically have less manpower when just when we need 
everybody.   And some of us end up spending 11 hours a day in front of the 
screen when we should be outside soaking up Vitamin D.

Therefore, I propose that the next version either feature freeze in March or 
in October, but NOT in May-August.  Which we do -- March or October -- can be 
based on an evaluation of the outstanding post-7.5 patches in January.   

-- 
Josh Berkus
Aglio Database Solutions
San Francisco


Re: Assisting developers

From
Bruce Momjian
Date:
Josh Berkus wrote:
> Chris,  all:
> 
> > Well, I note (and I'm not being unkind or anything here) that a lot of
> > the high level committers we have haven't been so active this release.
> > Peter and Joe haven't been around much and Jan has been busy with Slony.
> >   We also lost Thomas Lockhart.  Neil's also away on holidays.  You and
> > Tom have basically been doing all the reviewing - a great job - but I
> > can't believe Tom hasn't cracked yet :P
> 
> Actually, this is a *big* part of the problem.   In 7.3, by unplanned 
> circumstance, we got into a cycle of doing feature freeze and starting beta 
> during the summer.   This is a bad cycle -- the Europeans take their 3-week 
> vacations, the Americans go to (and spend many hours preparing for) a bunch 
> of conventions, Students go to internships, people take long weekends, get 
> married, etc.   ( This is probably why many American corporations end their 
> fiscal year in midsummer; nothing is going to get done anyway, might as well 
> clean up. )
> 
> The result is that we chronically have less manpower when just when we need 
> everybody.   And some of us end up spending 11 hours a day in front of the 
> screen when we should be outside soaking up Vitamin D.
> 
> Therefore, I propose that the next version either feature freeze in March or 
> in October, but NOT in May-August.  Which we do -- March or October -- can be 
> based on an evaluation of the outstanding post-7.5 patches in January.   

True, but northern-hemisphere summer is only 6 weeks old, and we have
had these issues for many months --- it isn't a new problem.   Alvaro
didn't get the feedback he needed in March either.  :-(

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: Assisting developers

From
Tom Lane
Date:
Peter Eisentraut <peter_e@gmx.net> writes:
> The issue as I see it is not reviewing patches, but defining features.  
> Someone sets out to develop "nested transactions", and three days after 
> feature freeze we have the first large discussion about what nested 
> transactions really are, what they are good for, and how they should 
> work.

Bear in mind though that what we have here is a huge discussion about
something that represents much less than 1% of the work involved in the
feature.  The hard part of nested transactions (or savepoints or
whatever you care to call 'em) is the implementation support for
reverting the backend's state to an earlier point without going all the
way back to ground-zero-idle state.  Alvaro's naturally spent most of
his time on the implementation, because without that there is no point
in debating syntax.  And it was the state of the implementation, not the
API which was understood to be unfinished, that drove the decision about
whether this was ready to be included in 7.5.

If we end up backing this out of 7.5, it will be because the remaining
implementation work doesn't get done, not because we are unable to agree
on a syntax.  (In which connection I'm a bit disturbed that Alvaro seems
to be spending time arguing with people rather than continuing to work
on internals...)
        regards, tom lane


Re: Assisting developers

From
Bruce Momjian
Date:
Bruce Momjian wrote:
> > The result is that we chronically have less manpower when just when we need 
> > everybody.   And some of us end up spending 11 hours a day in front of the 
> > screen when we should be outside soaking up Vitamin D.
> > 
> > Therefore, I propose that the next version either feature freeze in March or 
> > in October, but NOT in May-August.  Which we do -- March or October -- can be 
> > based on an evaluation of the outstanding post-7.5 patches in January.   
> 
> True, but northern-hemisphere summer is only 6 weeks old, and we have
> had these issues for many months --- it isn't a new problem.   Alvaro
> didn't get the feedback he needed in March either.  :-(

Oh, and neither did Simon.  :-(

In fact Simon coded a whole client-side approach to PITR before I
studied his ideas and realized it needs to be done as a server
subprocess.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: Assisting developers

From
"Marc G. Fournier"
Date:
On Tue, 13 Jul 2004, Bruce Momjian wrote:

> True, but northern-hemisphere summer is only 6 weeks old, and we have 
> had these issues for many months --- it isn't a new problem.  Alvaro 
> didn't get the feedback he needed in March either.  :-(

This is true, but Josh does have a point ... you took a much needed 12 
days off end of June, Tom took a few days in July ... both in middle of 
the feature freeze ... if we were in 'dev mode' through the summer months, 
those wouldn't have been as critical, but even once you got back, you had, 
what, 2k messages to weed through?

At least up in Canada, our summers are so short, we try and squeeze as 
much out of it as possible, so our time is more split then the rest of the 
year when we tend to stay indoors alot more ...



----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664


Re: Assisting developers

From
Bruce Momjian
Date:
Marc G. Fournier wrote:
> On Tue, 13 Jul 2004, Bruce Momjian wrote:
> 
> > True, but northern-hemisphere summer is only 6 weeks old, and we have 
> > had these issues for many months --- it isn't a new problem.  Alvaro 
> > didn't get the feedback he needed in March either.  :-(
> 
> This is true, but Josh does have a point ... you took a much needed 12 
> days off end of June, Tom took a few days in July ... both in middle of 
> the feature freeze ... if we were in 'dev mode' through the summer months, 
> those wouldn't have been as critical, but even once you got back, you had, 
> what, 2k messages to weed through?
> 
> At least up in Canada, our summers are so short, we try and squeeze as 
> much out of it as possible, so our time is more split then the rest of the 
> year when we tend to stay indoors alot more ...

True.  Things for the past few weeks have been worse, but we are geting
though that quickly.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: Assisting developers

From
"Marc G. Fournier"
Date:
On Tue, 13 Jul 2004, Bruce Momjian wrote:

> Marc G. Fournier wrote:
>> On Tue, 13 Jul 2004, Bruce Momjian wrote:
>>
>>> True, but northern-hemisphere summer is only 6 weeks old, and we have
>>> had these issues for many months --- it isn't a new problem.  Alvaro
>>> didn't get the feedback he needed in March either.  :-(
>>
>> This is true, but Josh does have a point ... you took a much needed 12
>> days off end of June, Tom took a few days in July ... both in middle of
>> the feature freeze ... if we were in 'dev mode' through the summer months,
>> those wouldn't have been as critical, but even once you got back, you had,
>> what, 2k messages to weed through?
>>
>> At least up in Canada, our summers are so short, we try and squeeze as
>> much out of it as possible, so our time is more split then the rest of the
>> year when we tend to stay indoors alot more ...
>
> True.  Things for the past few weeks have been worse, but we are geting
> though that quickly.

Granted, but that wasn't Josh's point :)  The point is, a March/October 
feature freeze would (should) have alot less issues as far as ppls time is 
concerned then a summer one (either Southern or Northern hemisphere 
summer) ...

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664


Re: Assisting developers

From
Peter Eisentraut
Date:
Josh Berkus wrote:
> ( This is
> probably why many American corporations end their fiscal year in
> midsummer; nothing is going to get done anyway, might as well clean
> up. )

And that is also the time when volunteers will have the most time to 
spare.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/



Re: Assisting developers

From
Joe Conway
Date:
Bruce Momjian wrote:
> Christopher Kings-Lynne wrote:
>>Well, I note (and I'm not being unkind or anything here) that a lot of 
>>the high level committers we have haven't been so active this release. 
>>Peter and Joe haven't been around much and Jan has been busy with Slony. 

I think this is (in part at least) a reflection of the economy finally 
picking up. Last year this time I had one major project going on -- 
right now I'm juggling about a half dozen or so, any one of which could 
keep me busy full time. Add a couple of OSCON presentations to prepare, 
and several serial disasters/distractions at home (nothing life 
threatening, but all time consuming), and that leaves little-to-no time 
for recreational Postgres coding :(

> The committing isn't really the issue.  It is reviewing and giving
> feedback to developers of complex features.
> 

This is very true. But the fact that I have been able to commit what 
little I have done this release has at least off-loaded some work from 
you Bruce, hasn't it? I'm still green enough though that I'm not in a 
position to review anything too complex -- someone like Tom or Bruce 
would still need to follow up behind me, so it wouldn't really save them 
any time.

Joe




Re: Assisting developers

From
Bruce Momjian
Date:
Joe Conway wrote:
> Bruce Momjian wrote:
> > Christopher Kings-Lynne wrote:
> >>Well, I note (and I'm not being unkind or anything here) that a lot of 
> >>the high level committers we have haven't been so active this release. 
> >>Peter and Joe haven't been around much and Jan has been busy with Slony. 
> 
> I think this is (in part at least) a reflection of the economy finally 
> picking up. Last year this time I had one major project going on -- 
> right now I'm juggling about a half dozen or so, any one of which could 
> keep me busy full time. Add a couple of OSCON presentations to prepare, 
> and several serial disasters/distractions at home (nothing life 
> threatening, but all time consuming), and that leaves little-to-no time 
> for recreational Postgres coding :(
> 
> > The committing isn't really the issue.  It is reviewing and giving
> > feedback to developers of complex features.
> > 
> 
> This is very true. But the fact that I have been able to commit what 
> little I have done this release has at least off-loaded some work from 
> you Bruce, hasn't it? I'm still green enough though that I'm not in a 
> position to review anything too complex -- someone like Tom or Bruce 
> would still need to follow up behind me, so it wouldn't really save them 
> any time.

Yes, others have stepped up to commit things while I was busy --- Neil
and Joe come to mind.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: Assisting developers

From
Karel Zak
Date:
On Tue, Jul 13, 2004 at 06:29:51PM +0200, Peter Eisentraut wrote:
> Bruce Momjian wrote:
> > I am not sure what can be done to solve this in the future.  There
> > are only a limited number of us who have the experience and time to
> > review and comment on very complex patches.
> 
> The issue as I see it is not reviewing patches, but defining features.  
You're  right. The other  problem is  that about  some features  nobodywants to talk over, because a feature is out of
mainstream interest oralmost  nobody really  understand  a problem. In  this  case all  startdiscussion if something is
alreadydone and they try use it.
 
   Karel

-- Karel Zak  <zakkr@zf.jcu.cz>http://home.zf.jcu.cz/~zakkr/