Thread: Postgres development model (was Re: CVS comment)

Postgres development model (was Re: CVS comment)

From
Alvaro Herrera
Date:
On Sat, Aug 07, 2004 at 12:38:10PM +0200, Gaetano Mendola wrote:
> Alvaro Herrera wrote:

> >Subversion and arch have been mentioned, but so far there is no
> >compelling reason to change.  It'd take convincing at least a couple
> >of core hackers to get the ball rolling ...
> 
> Well, I think having seen what's happening at the 8.0 relase I think
> that committers are too overloaded and someone else have to be
> "promoted" to be a committers, and I believe that having betters tools
> can improve the process too.

I don't think it was a problem of committers.  To me it was a problem of
reviewers.  Those are very scarce (for the bigger items it's mostly only
Tom).  Maybe a better SCM could help with this, but I doubt it.  As an
example, I did read the autovacuum patch, but I had no useful comment to
make on it.  Why didn't Jan or Bruce say something about it?  What about
Neil or Joe Conway?  If any of them could have had useful feedback, they
didn't have the time to do it.

The Linux model, heavily changed by the BitKeeper phenomenom (and I
think Andrew Morton plays a big role there too), does not really apply
here because the difference in manpower is huge.  They currently have a
very succesful model, very different from what was two years ago, but
they do have several very capable mantainers/reviewers.

Maybe our development process does need a revision.  Neil Conway seemed
to agree some time ago, but the rest of the people doesn't seem to have
an opinion on the subject that I remember.

But it's not a problem of committers.  Oleg Bartunov is a committer too,
as is Dennis Bjorklund, Peter Eisentraut, Marc Fournier and others.  But
apparently these are issues that they are not able to help with.

-- 
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Some men are heterosexual, and some are bisexual, and some
men don't think about sex at all... they become lawyers" (Woody Allen)



Re: Postgres development model (was Re: CVS comment)

From
Joe Conway
Date:
Alvaro Herrera wrote:
> I don't think it was a problem of committers.  To me it was a problem of
> reviewers.  Those are very scarce (for the bigger items it's mostly only
> Tom).  Maybe a better SCM could help with this, but I doubt it.  As an
> example, I did read the autovacuum patch, but I had no useful comment to
> make on it.  Why didn't Jan or Bruce say something about it?  What about
> Neil or Joe Conway?  If any of them could have had useful feedback, they
> didn't have the time to do it.

Unfortunately due to other commitments, personal and professional, I 
haven't had time to do much this development cycle :-(. I'm just now 
able to start getting a bit more active again.

But in any case I think your second sentence above gets to the heart of 
the issue. Postgres is a complex piece of code, and even though I have 
commit access, I don't understand many parts of it well enough to do a 
credible job reviewing others' code (at least not without days of effort 
just trying to understand what it is that I'm reviewing). Where I have 
both the time and some knowledge I do try to help, e.g. with plperl.

Joe


Re: Postgres development model (was Re: CVS comment)

From
Tom Lane
Date:
Alvaro Herrera <alvherre@dcc.uchile.cl> writes:
> I don't think it was a problem of committers.  To me it was a problem of
> reviewers.  Those are very scarce (for the bigger items it's mostly only
> Tom).

Yah.  We have plenty of people authorized to commit, and we add more
on a pretty regular basis.  (FWIW, Alvaro, you are high on the list
of people to appoint as new committers.)  The problem is finding
adequate review talent.  You don't have to be a committer to help review
patches --- feel free to look at anything that goes by, and if you see
a problem say so!  But the difficulty is that PG is a pretty large and
complex system, and it takes a good deal of familiarity with it to spot
some of the more esoteric problems.

Right at the moment we are a bit short of uber-hackers.  Vadim Mikheev
knew a lot about the code, but he's dropped out of sight and not been
replaced.  Tom Lockhart is sorely missed as well.  You, Manfred, and
Neil are up-and-coming but you each probably need another couple years
fooling with the code before you really have the full wizard's rating.

I don't have any magic solution to this.  I do say that people who know
the code get there by doing things with it --- at least that's how I got
there --- so I certainly encourage anyone with the time and interest
to pursue it.  When you see a bizarre bug report, find the cause and
fix it.  Or pick a project that you almost know how to do, but not
quite, and learn until you can do it.  Repeat as needed.

> Maybe a better SCM could help with this, but I doubt it.

I haven't seen any particular reason why we should adopt another SCM.
Perhaps BitKeeper or SubVersion would be better for our purposes than
CVS, but are they enough better to justify the switchover costs?
I doubt it.
        regards, tom lane


Re: Postgres development model (was Re: CVS comment)

From
"Ross J. Reedstrom"
Date:
On Sun, Aug 08, 2004 at 01:18:02AM -0400, Tom Lane wrote:
> Alvaro Herrera <alvherre@dcc.uchile.cl> writes:
> 
> > Maybe a better SCM could help with this, but I doubt it.
> 
> I haven't seen any particular reason why we should adopt another SCM.
> Perhaps BitKeeper or SubVersion would be better for our purposes than
> CVS, but are they enough better to justify the switchover costs?
> I doubt it.

CVS has it problems, and there are other SCM packages available that
address those, but the but the main problem for postgresql is shortage
of uber-hackers, as Tom says. Once that problems solved (more good
coders seasoned in the codebase) then it might be time to switch.
By then, they'll be even easier to use. ;-)

Ross
-- 
Ross Reedstrom, Ph.D.                                 reedstrm@rice.edu
Research Scientist                                  phone: 713-348-6166
The Connexions Project      http://cnx.rice.edu       fax: 713-348-3665
Rice University MS-375, Houston, TX 77005
GPG Key fingerprint = F023 82C8 9B0E 2CC6 0D8E  F888 D3AE 810E 88F0 BEDE


Re: Postgres development model (was Re: CVS comment)

From
Peter Eisentraut
Date:
Tom Lane wrote:
> I haven't seen any particular reason why we should adopt another SCM.
> Perhaps BitKeeper or SubVersion would be better for our purposes than
> CVS, but are they enough better to justify the switchover costs?

BitKeeper ist not open source, so it's out of the question for most 
people.  Subversion is shockingly unstable.  I'm very open for 
something that replaces CVS, but I'd rather not use any than one of 
these two.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/



Re: Postgres development model (was Re: CVS comment)

From
Gaetano Mendola
Date:
Peter Eisentraut wrote:

> Tom Lane wrote:
> 
>>I haven't seen any particular reason why we should adopt another SCM.
>>Perhaps BitKeeper or SubVersion would be better for our purposes than
>>CVS, but are they enough better to justify the switchover costs?
> 
> 
> BitKeeper ist not open source, so it's out of the question for most 
> people.  

Not for Linus Torvalds apparently.



Regards
Gaetano Mendola









Re: Postgres development model (was Re: CVS comment)

From
Joerg Hessdoerfer
Date:
Hi,

On Monday 09 August 2004 09:30, you wrote:
> Tom Lane wrote:
> > I haven't seen any particular reason why we should adopt another SCM.
> > Perhaps BitKeeper or SubVersion would be better for our purposes than
> > CVS, but are they enough better to justify the switchover costs?
>
> BitKeeper ist not open source, so it's out of the question for most
> people.  Subversion is shockingly unstable.  I'm very open for
> something that replaces CVS, but I'd rather not use any than one of
> these two.

Wow, that's a remark. 'Shockingly unstable'... I wonder where you got that 
from? As someone who is using Subversion very heavily in production 
environments, with code repositories which outgrow PostgreSQL's codebase size 
by factors of up to 20 (mainly due to being binary source code for a strange 
development platform) and having a very high update rate I *never* had any 
problems except one or two short periods of inaccessibility due to web server 
probs.

I would really like you to substantiate those claims, and please not from the 
pre-beta time area.

'K, 'nough said!

Greetings,Joerg Hessdoerfer
-- 
Leading SW developer  - S.E.A GmbH
Mail: joerg.hessdoerfer@sea-gmbh.com
WWW:  http://www.sea-gmbh.com


Re: Postgres development model (was Re: CVS comment)

From
James William Pye
Date:
On 08/09/04:32/1, Peter Eisentraut wrote:
> BitKeeper ist not open source, so it's out of the question for most
> people.  Subversion is shockingly unstable.  I'm very open for
> something that replaces CVS, but I'd rather not use any than one of
> these two.

From my casual usage of svn, I haven't noticed any stability issues. If it
were shockingly unstable, I would have expected to have had problems with
it.

Using the ssh tunnel, served on an fbsd jail(yeah, rented from Marc), and
connecting with svn client 1.0.4 on my home fbsd 4.10 box..

--
Regards,       James William Pye

Re: Postgres development model (was Re: CVS comment)

From
Reinoud van Leeuwen
Date:
On Mon, Aug 09, 2004 at 09:30:09AM +0200, Peter Eisentraut wrote:
> Tom Lane wrote:
> > I haven't seen any particular reason why we should adopt another SCM.
> > Perhaps BitKeeper or SubVersion would be better for our purposes than
> > CVS, but are they enough better to justify the switchover costs?
> 
> BitKeeper ist not open source, so it's out of the question for most 
> people.  

Why? I understood that using BitKeeper for free for Open Source projects 
is allowed. (but IANAL).
It is available (on many platforms). It works great. Once you use 
changesets you'll never want to go back to cvs.

Producing an Open Source product does not mean that all tools are Open 
Source. Windows isn't and Postgresql is going to support windows.

-- 
__________________________________________________
"Nothing is as subjective as reality"
Reinoud van Leeuwen    reinoud.v@n.leeuwen.net
http://www.xs4all.nl/~reinoud
__________________________________________________


Re: Postgres development model (was Re: CVS comment)

From
Peter Eisentraut
Date:
Reinoud van Leeuwen wrote:
> Why? I understood that using BitKeeper for free for Open Source
> projects is allowed. (but IANAL).
> It is available (on many platforms). It works great. Once you use
> changesets you'll never want to go back to cvs.

There is a world of a difference between being free (of charge) for open 
source projects and being open source itself.

> Producing an Open Source product does not mean that all tools are
> Open Source. Windows isn't and Postgresql is going to support
> windows.

There is also a big difference between supporting some proprietary 
software and making proprietary software a de facto requirement for 
participating in the development effort.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/



Re: Postgres development model (was Re: CVS comment)

From
"Andrew Dunstan"
Date:
Reinoud van Leeuwen said:
> On Mon, Aug 09, 2004 at 09:30:09AM +0200, Peter Eisentraut wrote:
>> Tom Lane wrote:
>> > I haven't seen any particular reason why we should adopt another
>> > SCM. Perhaps BitKeeper or SubVersion would be better for our
>> > purposes than CVS, but are they enough better to justify the
>> > switchover costs?
>>
>> BitKeeper ist not open source, so it's out of the question for most
>> people.
>
> Why? I understood that using BitKeeper for free for Open Source
> projects  is allowed. (but IANAL).
> It is available (on many platforms). It works great. Once you use
> changesets you'll never want to go back to cvs.
>
> Producing an Open Source product does not mean that all tools are Open
> Source. Windows isn't and Postgresql is going to support windows.
>

tools we use <> platforms we support. With one tiny exception all the tools
we use for producing the Windows binaries are in fact open source.

Anyway, when people who actually make substantial contributions start making
noises about changing source repository systems it should be considered -
nobody else should care or be listened to, IMNSHO.

cheers

andrew




Re: Postgres development model

From
Christopher Browne
Date:
Oops! reinoud.v@n.leeuwen.net (Reinoud van Leeuwen) was seen spray-painting on a wall:
> On Mon, Aug 09, 2004 at 09:30:09AM +0200, Peter Eisentraut wrote:
>> Tom Lane wrote:
>> > I haven't seen any particular reason why we should adopt another SCM.
>> > Perhaps BitKeeper or SubVersion would be better for our purposes than
>> > CVS, but are they enough better to justify the switchover costs?
>> 
>> BitKeeper ist not open source, so it's out of the question for most 
>> people.  
>
> Why? I understood that using BitKeeper for free for Open Source projects 
> is allowed. (but IANAL).
> It is available (on many platforms). It works great. Once you use 
> changesets you'll never want to go back to cvs.

Ah, but there's a problem with BK _actually seen in production_ in
that people that work on competing products are not permitted to use
it.
   (d)  Notwithstanding any other terms in this License, this        License is not available to You if You and/or your
      employer develop, produce, sell, and/or resell a        product which contains substantially similar capabil-
  ities of the BitKeeper Software, or, in the reason-        able opinion of BitMover, competes with the BitKeeper
 Software.
 

I don't think it would be "way out there" to consider that this means
that anyone that contributes to FreeBSD or to Debian Linux (two
systems that include packages for both SubVersion and Arch) is
forbidden to use BitKeeper.  

After all, they are involved in "developing/producing" a product that
contains substantially similar capabilities.
-- 
wm(X,Y):-write(X),write('@'),write(Y). wm('cbbrowne','cbbrowne.com').
http://www3.sympatico.ca/cbbrowne/oses.html
"Linux  is not  ready  for  the Enterprise.   There  is not a   single
voice-controlled app for any of the  mission critical functions of the
Enterprise. Conspicuously absent  are warp  core control, phaser  bank
activation,  interstellar  navigation,  transporter operation, and the
all-important  self-destruct  sequence. Until  these and  thousands of
other important apps are  written and deployed, Linux  will just be  a
toy in the Enterprise." -- Kevin Novak, Network Computing Magazine


Re: Postgres development model (was Re: CVS comment)

From
Andrew Dunstan
Date:

Peter Eisentraut wrote:

>There is also a big difference between supporting some proprietary 
>software and making proprietary software a de facto requirement for 
>participating in the development effort.
>
>  
>

Just to complete the information on this, I have it on good authority 
(i.e. from Larry McVoy) that they have a prototype BK<->CVS 
bidirectional read/write gateway. So nobody would be required to use 
anything.

My personal opinion (which I expressed to Larry) is that the pain is not 
great enough to warrant any change right now, but I guess the committers 
have a better view of that than I do.

cheers

andrew


Re: Postgres development model

From
Tom Lane
Date:
Christopher Browne <cbbrowne@acm.org> writes:
> Oops! reinoud.v@n.leeuwen.net (Reinoud van Leeuwen) was seen spray-painting on a wall:
>> Why? I understood that using BitKeeper for free for Open Source projects 
>> is allowed. (but IANAL).

> Ah, but there's a problem with BK _actually seen in production_ in
> that people that work on competing products are not permitted to use
> it.

In particular, I would have to resign from the project if we went over
to BK, as my employer (Red Hat) is affected by this restriction.  BK
does not meet the accepted definition of Open Source because of this
unfriendly license clause.
        regards, tom lane


Re: Postgres development model

From
Bruce Momjian
Date:
Tom Lane wrote:
> Christopher Browne <cbbrowne@acm.org> writes:
> > Oops! reinoud.v@n.leeuwen.net (Reinoud van Leeuwen) was seen spray-painting on a wall:
> >> Why? I understood that using BitKeeper for free for Open Source projects 
> >> is allowed. (but IANAL).
> 
> > Ah, but there's a problem with BK _actually seen in production_ in
> > that people that work on competing products are not permitted to use
> > it.
> 
> In particular, I would have to resign from the project if we went over
> to BK, as my employer (Red Hat) is affected by this restriction.  BK
> does not meet the accepted definition of Open Source because of this
> unfriendly license clause.

How do the Linux kernel developer guys go from no revision system (just
Linus's hard drive) to Bitkeeper and requring a signed authorization
letter from each contributor?  They went from too little to too much,
and never hit the happy medium.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: Postgres development model

From
Christopher Browne
Date:
In the last exciting episode, pgman@candle.pha.pa.us (Bruce Momjian) wrote:
> Tom Lane wrote:
>> Christopher Browne <cbbrowne@acm.org> writes:
>> > Oops! reinoud.v@n.leeuwen.net (Reinoud van Leeuwen) was seen spray-painting on a wall:
>> >> Why? I understood that using BitKeeper for free for Open Source projects 
>> >> is allowed. (but IANAL).
>> 
>> > Ah, but there's a problem with BK _actually seen in production_ in
>> > that people that work on competing products are not permitted to use
>> > it.
>> 
>> In particular, I would have to resign from the project if we went over
>> to BK, as my employer (Red Hat) is affected by this restriction.  BK
>> does not meet the accepted definition of Open Source because of this
>> unfriendly license clause.
>
> How do the Linux kernel developer guys go from no revision system (just
> Linus's hard drive) to Bitkeeper and requring a signed authorization
> letter from each contributor?  They went from too little to too much,
> and never hit the happy medium.

What happened was that Linus Torvalds essentially said "what are the
features I need to better merge together contributions?" and Larry
McVoy responded with "We'll build what you need!"  (There was
certainly a lot of back and forth conversation, but that's the short
answer as to how it happened...)

It is pretty interesting how successful BK turned out to be for the
purpose.  But definitely "some rights were VERY MUCH reserved."
-- 
If this was helpful, <http://svcs.affero.net/rm.php?r=cbbrowne> rate me
http://www3.sympatico.ca/cbbrowne/linux.html
"The social dynamics  of the net are a direct  consequence of the fact
that nobody  has yet developed  a Remote Strangulation  Protocol."  
-- Larry Wall