Thread: Not ready for 8.3

Not ready for 8.3

From
Bruce Momjian
Date:
Based on our progress during this feature freeze, we will not complete
the feature freeze until August/September.  I think we need adjust
expectations about an 8.3 release date, and decide if we want to
radically change our work process.

Basically, to make a release anywhere near July, we need 10x more
progress than we have had, which is unlikely.

We could ship what is in CVS now, but that just pushes the patches for
8.4, and isn't fair to patch submitters.  We could shove what we have
now into CVS, but that is unwise.

I think the only other thing we _could_ do is to re-open normal 8.3
development, so we aren't hampering updates to trivial parts of the
code. Many of the patches now in the queue had been developed for months
before 8.3 started, so the hope is that we wouldn't have many more new
large patches, but several small ones we could deal with while we
whittle away at the larger patches during the next few months.

The question is whether it is healthy for us to remain in feature freeze
for months, and if it is unhealthy, what are our options?

Patch status:
http://developer.postgresql.org/index.php/Todo:PatchStatus

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
Gregory Stark
Date:
"Bruce Momjian" <bruce@momjian.us> writes:

> I think the only other thing we _could_ do is to re-open normal 8.3
> development, so we aren't hampering updates to trivial parts of the
> code. Many of the patches now in the queue had been developed for months
> before 8.3 started, so the hope is that we wouldn't have many more new
> large patches, but several small ones we could deal with while we
> whittle away at the larger patches during the next few months.
>
> The question is whether it is healthy for us to remain in feature freeze
> for months, and if it is unhealthy, what are our options?

I don't see any reason development has to stop while the tree is in feature
freeze. If it led to patches being ready for review and getting reviewed and
committed early in the cycle rather than just before release I think it would
actually be extremely healthy.

--  Gregory Stark EnterpriseDB          http://www.enterprisedb.com



Re: Not ready for 8.3

From
"Joshua D. Drake"
Date:
Bruce Momjian wrote:
> Based on our progress during this feature freeze, we will not complete
> the feature freeze until August/September.  I think we need adjust
> expectations about an 8.3 release date, and decide if we want to
> radically change our work process.
> 
> Basically, to make a release anywhere near July, we need 10x more
> progress than we have had, which is unlikely.

Or we have another short release cycle, basically accepting what we have 
now that can be worked through in the next 3 weeks and committed. If it 
can't be done in that time, it waits. Then we have beta etc...

> 
> We could ship what is in CVS now, but that just pushes the patches for
> 8.4, and isn't fair to patch submitters.  We could shove what we have
> now into CVS, but that is unwise.

Sure it is, if we have a short release cycle. There are plenty of things 
out there that are not quite ready yet, that could be if we released 
again next January...


> 
> I think the only other thing we _could_ do is to re-open normal 8.3
> development, so we aren't hampering updates to trivial parts of the
> code. Many of the patches now in the queue had been developed for months

Gah, that sounds like a horrible solution. Sorry Bruce.

> 
> The question is whether it is healthy for us to remain in feature freeze
> for months, and if it is unhealthy, what are our options?
> 
> Patch status:
> 
>     http://developer.postgresql.org/index.php/Todo:PatchStatus

If... this is actually a problem (I leave to other committers and 
reviewers to comment) then I suggest we push all patches without a 
reviewer as of now to 8.4.

Leaving only those patches that have confirmed reviewers to be worked 
through.

FYI, whoever did that Todo:Patch status, Bravo! That is easily one of 
the smallest but best improvements to the process I have seen in recent 
memory.

Sincerely,

Joshua D. Drake





Re: Not ready for 8.3

From
Bruce Momjian
Date:
Joshua D. Drake wrote:
> Bruce Momjian wrote:
> > Based on our progress during this feature freeze, we will not complete
> > the feature freeze until August/September.  I think we need adjust
> > expectations about an 8.3 release date, and decide if we want to
> > radically change our work process.
> > 
> > Basically, to make a release anywhere near July, we need 10x more
> > progress than we have had, which is unlikely.
> 
> Or we have another short release cycle, basically accepting what we have 
> now that can be worked through in the next 3 weeks and committed. If it 
> can't be done in that time, it waits. Then we have beta etc...

That is not fair to patch submitters, and pushes the problem to 8.4,
where it will be no better.

> > We could ship what is in CVS now, but that just pushes the patches for
> > 8.4, and isn't fair to patch submitters.  We could shove what we have
> > now into CVS, but that is unwise.
> 
> Sure it is, if we have a short release cycle. There are plenty of things 
> out there that are not quite ready yet, that could be if we released 
> again next January...

Huh, you didn't answer my issue about unfair.

> > The question is whether it is healthy for us to remain in feature freeze
> > for months, and if it is unhealthy, what are our options?
> > 
> > Patch status:
> > 
> >     http://developer.postgresql.org/index.php/Todo:PatchStatus
> 
> If... this is actually a problem (I leave to other committers and 
> reviewers to comment) then I suggest we push all patches without a 
> reviewer as of now to 8.4.
> 
> Leaving only those patches that have confirmed reviewers to be worked 
> through.
> 
> FYI, whoever did that Todo:Patch status, Bravo! That is easily one of 
> the smallest but best improvements to the process I have seen in recent 
> memory.

Fairness and not pushing our problems out to the future --- address
those issues.

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
Bruce Momjian
Date:
Joshua D. Drake wrote:
> > Patch status:
> > 
> >     http://developer.postgresql.org/index.php/Todo:PatchStatus
> 
> If... this is actually a problem (I leave to other committers and 
> reviewers to comment) then I suggest we push all patches without a 
> reviewer as of now to 8.4.
> 
> Leaving only those patches that have confirmed reviewers to be worked 
> through.
> 
> FYI, whoever did that Todo:Patch status, Bravo! That is easily one of 
> the smallest but best improvements to the process I have seen in recent 
> memory.

I did one of those for previous releases.  I guess you forgot.  It was
done by someone else this time only because I was going to be traveling.

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
Gregory Stark
Date:
"Bruce Momjian" <bruce@momjian.us> writes:

> Joshua D. Drake wrote:
>> Bruce Momjian wrote:
>> > Based on our progress during this feature freeze, we will not complete
>> > the feature freeze until August/September.  I think we need adjust
>> > expectations about an 8.3 release date, and decide if we want to
>> > radically change our work process.
>> > 
>> > Basically, to make a release anywhere near July, we need 10x more
>> > progress than we have had, which is unlikely.
>> 
>> Or we have another short release cycle, basically accepting what we have 
>> now that can be worked through in the next 3 weeks and committed. If it 
>> can't be done in that time, it waits. Then we have beta etc...
>
> That is not fair to patch submitters, and pushes the problem to 8.4,
> where it will be no better.

It's nice to be fair but it's not really the goal here.

However I think the latter point is key. This *was* a short release cycle. If
we push off these patches to later there's no reason to think the situation
will be any different in a few months.

I suspect as much as Tom finds reviewing boring he would rather get it out of
the way and then have a chance to do development before more major patches
arrive than start the new cycle with major patches sitting in the review queue
and spend the whole cycle reviewing them.

As much as I hate that the patches sit in the queue until feature freeze
there's nothing else that will really force us to make a yea or nay decision
on them. If we start the next release with things in the patch queue they're
liable to sit there with no feedback for months again.

I think the key here is to give feedback for the authors but not allow the
time to respond to that feedback to grow. If you review a patch and find
problems with it and there's only three weeks for the author to deal with
those problems and it's not enough time then he can deal with them and submit
it for next release. But letting the patch ride over without giving any
feedback just seems pointless.

--  Gregory Stark EnterpriseDB          http://www.enterprisedb.com



Re: Not ready for 8.3

From
Stefan Kaltenbrunner
Date:
Joshua D. Drake wrote:

> FYI, whoever did that Todo:Patch status, Bravo! That is easily one of
> the smallest but best improvements to the process I have seen in recent
> memory.

well bruce asked for something like that:
http://archives.postgresql.org/pgsql-hackers/2007-05/msg00249.php

and I simply went ahead and did it:

http://archives.postgresql.org/pgsql-hackers/2007-05/msg00265.php


Stefan


Re: Not ready for 8.3

From
Josh Berkus
Date:
Bruce,

Realistically I just don't see getting everything in the ToDo patch list in; 
my vote is that we start deferring stuff for 8.4 if it doesn't have a 
reviewer, except for items which were submitted early in the cycle (and to 
whom it would be unfair). 

If that means shortening the 8.4 cycle somewhat, I'm for that ... feature 
freeze in Feburary would be even better than April, because it means we could 
be in Beta for the May-June-July conferences, and increase our probability of 
being able to release at a major conference or PostgreSQL conference.

Obviously for 8.4 reviewers need to start reviewing stuff from the first week 
of the development cycle.  I also don't actually see anything wrong with a 
3-month feature freeze if we can somehow branch development earlier.  Easy 
for me to say, I know. 

-- 
Josh Berkus
PostgreSQL @ Sun
San Francisco


Re: Not ready for 8.3

From
"Joshua D. Drake"
Date:
> That is not fair to patch submitters, and pushes the problem to 8.4,
> where it will be no better.

If it isn't done, it isn't done. We aren't dropping the patch. The patch 
has been accepted, just not reviewed. It is just delayed.

>> Sure it is, if we have a short release cycle. There are plenty of things 
>> out there that are not quite ready yet, that could be if we released 
>> again next January...
> 
> Huh, you didn't answer my issue about unfair.

Sure I did see above :). Unfair would be to let go all together. We are 
just delaying. If we don't have reviewers, we don't have reviewers and 
as I look at the open list we have more reviewers than we did for 8.2 so 
I would suspect that 8.4 is going to go smoother as it is.

>> Leaving only those patches that have confirmed reviewers to be worked 
>> through.
>>
>> FYI, whoever did that Todo:Patch status, Bravo! That is easily one of 
>> the smallest but best improvements to the process I have seen in recent 
>> memory.
> 
> Fairness and not pushing our problems out to the future --- address
> those issues.

Life isn't fair. It isn't like we are telling these people to bugger 
off. We are just delaying the review to what is a reasonable workload 
for the current set of reviewers.

Life... is a perpetual problem. We do what we can, when we can, and the 
best that we can.

Taking my suggestion above for leaving patches that don't have reviewers  let's look at some of them.

Tsearch2 in core. I know that Tom has some reservations, he I and Treat 
all commented on how it was done and to my knowledge those reservations 
have not been resolved.

HOT is a huge patch that has gone round and round and round. Lots of 
people want it, including me but it is invasive enough to where it may 
due it good to work through another cycle.

Grouped Index Tuples. I like the result of this patch. I tested the 
performance and it did work as advertised but we didn't get much 
feedback as a whole from known hackers. Either there isn't much interest  or we are too busy. Either way it is
certainlynot a critical patch 
 
and can be pushed.

Concurrent psql, nifty but still trying to decide on actual interface.

Full Page Writes Improvement, doesn't actually do anything *yet* (as far 
as I can tell) it just makes it so something can be done in the future. 
It is also apparently a small patch.

UTF8 text matching performance improvements (todo wiki link broke), so I 
don't have much comment.

On Disk Bitmap indexes (todo wiki link broke): Doesn't support Vacuum 
(to my knowledge), community member tested, reported problems... no 
response yet from author. The author is known to be time constrained, 
boot it till 8.4.

Posix shared memory, not usable in current state per Tom's comments and 
Apples, agreement. Dumped till 8.4 or further.

Stream bitmaps, apparently needed more info from Gavin (see previous 
comment about author's time). No feed back since March 9th. Dumped till 8.4.

Bitmapscan change, this one is unfortunate because at the time Heikki 
had resources and desire but was basically ignored. Sometimes we have to 
punt although Heikki is doing patch review right now and it is not 
unheard of for a patch reviewer to commit his own patch (in this case we 
would need a comitter to actually accept it.).

Updateable cursors, apparently breaks explain and patch has been 
updated. Punt!

Group Commit, Heikki has already suggested that it may be a good idea to 
push for 8.4 on this patch: http://momjian.us/mhonarc/patches/msg00127.html

Index Advisor.. I think the link is wrong:

http://momjian.us/mhonarc/patches/msg00119.html

Ctid chain patch, apparently no discussion since last January even 
though requests had been made to change the patch to some degree. Punt.
I will note that no one was negative about the patch, it just doesn't 
appear that the requests were ever finished.

Error correct for n_dead_tuples, patch was requested for hold in Feb. No 
discussion since. Punt!

DSM, apparently includes n_dead_tuples, please remove n_dead_tuples line 
on todo. Significant memory allocation enhancements are expected in 8.4 
for this. Discussion died in April. Concerns were raised about how 
memory is allocated (fixed, shared) which author already admints to 
wanting to change for 8.4.

PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be 
developed outside of core. Don't get me wrong I like the feature but it 
can take advantage of facilities outside of core.

So there ya go... thoughts, flames?

Sincerely,

Joshua D. Drake






> 



Re: Not ready for 8.3

From
Andrew Dunstan
Date:

Gregory Stark wrote:
> "Bruce Momjian" <bruce@momjian.us> writes:
>
>   
>> I think the only other thing we _could_ do is to re-open normal 8.3
>> development, so we aren't hampering updates to trivial parts of the
>> code. Many of the patches now in the queue had been developed for months
>> before 8.3 started, so the hope is that we wouldn't have many more new
>> large patches, but several small ones we could deal with while we
>> whittle away at the larger patches during the next few months.
>>
>> The question is whether it is healthy for us to remain in feature freeze
>> for months, and if it is unhealthy, what are our options?
>>     
>
> I don't see any reason development has to stop while the tree is in feature
> freeze. If it led to patches being ready for review and getting reviewed and
> committed early in the cycle rather than just before release I think it would
> actually be extremely healthy.
>
>   

If we had multiple dev branches it might be more possible. That has its 
own costs, of course - having a single dev branch makes management much 
easier, and we never have to worry about things like merging.

Patches seem to be getting larger, more complex, and harder to review. 
That is stressing our processes more than somewhat.

Short cycles would only make matters worse - the frictional cost of each 
dev cycle is growing. I think at least we have learned not to repeat 
this exercise.

cheers

andrew


Re: Not ready for 8.3

From
Oleg Bartunov
Date:
We have new patch available 
http://www.sigaev.ru/misc/tsearch_core-0.47.gz
to sync with CVS HEAD.

Oleg
On Tue, 15 May 2007, Joshua D. Drake wrote:

> Bruce Momjian wrote:
>> Based on our progress during this feature freeze, we will not complete
>> the feature freeze until August/September.  I think we need adjust
>> expectations about an 8.3 release date, and decide if we want to
>> radically change our work process.
>> 
>> Basically, to make a release anywhere near July, we need 10x more
>> progress than we have had, which is unlikely.
>
> Or we have another short release cycle, basically accepting what we have now 
> that can be worked through in the next 3 weeks and committed. If it can't be 
> done in that time, it waits. Then we have beta etc...
>
>> 
>> We could ship what is in CVS now, but that just pushes the patches for
>> 8.4, and isn't fair to patch submitters.  We could shove what we have
>> now into CVS, but that is unwise.
>
> Sure it is, if we have a short release cycle. There are plenty of things out 
> there that are not quite ready yet, that could be if we released again next 
> January...
>
>
>> 
>> I think the only other thing we _could_ do is to re-open normal 8.3
>> development, so we aren't hampering updates to trivial parts of the
>> code. Many of the patches now in the queue had been developed for months
>
> Gah, that sounds like a horrible solution. Sorry Bruce.
>
>> 
>> The question is whether it is healthy for us to remain in feature freeze
>> for months, and if it is unhealthy, what are our options?
>> 
>> Patch status:
>>
>>     http://developer.postgresql.org/index.php/Todo:PatchStatus
>
> If... this is actually a problem (I leave to other committers and reviewers 
> to comment) then I suggest we push all patches without a reviewer as of now 
> to 8.4.
>
> Leaving only those patches that have confirmed reviewers to be worked 
> through.
>
> FYI, whoever did that Todo:Patch status, Bravo! That is easily one of the 
> smallest but best improvements to the process I have seen in recent memory.
>
> Sincerely,
>
> Joshua D. Drake
>
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
>              http://archives.postgresql.org
>
    Regards,        Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83


Re: Not ready for 8.3

From
Chris Browne
Date:
josh@agliodbs.com (Josh Berkus) writes:
> Bruce,
>
> Realistically I just don't see getting everything in the ToDo patch
> list in; my vote is that we start deferring stuff for 8.4 if it
> doesn't have a reviewer, except for items which were submitted early
> in the cycle (and to whom it would be unfair).
>
> If that means shortening the 8.4 cycle somewhat, I'm for that
> ... feature freeze in Feburary would be even better than April,
> because it means we could be in Beta for the May-June-July
> conferences, and increase our probability of being able to release
> at a major conference or PostgreSQL conference.

If we're pushing stuff planned for 8.3 off into 8.4, then doesn't that
mean that we'd be inclined to _lengthen_ the 8.4 cycle?

If we were to shorten the 8.4 cycle, that seems likely to me to worsen
the problem, so that I'd expect to see even more stuff deferred for
8.5 than was the case with 8.3...

> Obviously for 8.4 reviewers need to start reviewing stuff from the
> first week of the development cycle.  I also don't actually see
> anything wrong with a 3-month feature freeze if we can somehow
> branch development earlier.  Easy for me to say, I know.

Well, if people are essentially already working on patches for 8.4
*now*, but just deferring merging until after 8.3 is committed +
released, then we've got 8.4 work underway already.

Unfortunately, that's also likely to worsen the problem of the
reviewers' queues being even more overflowing.

I'd sorta like to see the Tom Lanes and Bruce Momjians of the project
getting to have some time to work on things that *they'd* like add, as
opposed to turning things into a spiral cycle where they keep spending
more and more of their time reviewing patches, as opposed to doing
things they find neat and new.  Too many iterations of that sort of
thing, and they'll not want to ever see a patch again...

-- 
output = ("cbbrowne" "@" "linuxdatabases.info")
http://cbbrowne.com/info/wp.html
"...It is also  possible to post imbecilic  articles with any browser,
especially when you  toppost    and omit snippage."    --   CBFalconer
<cbfalconer@yahoo.com> - seen on comp.lang.c 


Re: Not ready for 8.3

From
Oleg Bartunov
Date:
On Tue, 15 May 2007, Joshua D. Drake wrote:

> Tsearch2 in core. I know that Tom has some reservations, he I and Treat all 
> commented on how it was done and to my knowledge those reservations have not 
> been resolved.

We'd like to know about these reservations. If I understand you mean there 
are issues with the patch ? Our patch is several months old. We permanently
keep it in sync with CVS HEAD, latest version is 0.47.

We're really bored with the whole process of development. We're not
full-time developers, we just devote our spare time which we withdraw from 
time we should spend on many other things. We have no time even to discuss
those very useful threads about community management, patches, etc. We just
rely on community decision.
    Regards,        Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83


Re: Not ready for 8.3

From
Stefan Kaltenbrunner
Date:
Joshua D. Drake wrote:

[...]
> Concurrent psql, nifty but still trying to decide on actual interface.
> 
> Full Page Writes Improvement, doesn't actually do anything *yet* (as far
> as I can tell) it just makes it so something can be done in the future.
> It is also apparently a small patch.
> 
> UTF8 text matching performance improvements (todo wiki link broke), so I
> don't have much comment.

url should be fixed now

> 
> On Disk Bitmap indexes (todo wiki link broke): Doesn't support Vacuum
> (to my knowledge), community member tested, reported problems... no
> response yet from author. The author is known to be time constrained,
> boot it till 8.4.

url fixed too


> 
> Bitmapscan change, this one is unfortunate because at the time Heikki
> had resources and desire but was basically ignored. Sometimes we have to
> punt although Heikki is doing patch review right now and it is not
> unheard of for a patch reviewer to commit his own patch (in this case we
> would need a comitter to actually accept it.).
> 
> Updateable cursors, apparently breaks explain and patch has been
> updated. Punt!
> 
> Group Commit, Heikki has already suggested that it may be a good idea to
> push for 8.4 on this patch: http://momjian.us/mhonarc/patches/msg00127.html
> 
> Index Advisor.. I think the link is wrong:
> 
> http://momjian.us/mhonarc/patches/msg00119.html

url fixed

> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
> developed outside of core. Don't get me wrong I like the feature but it
> can take advantage of facilities outside of core.

url fixed - I wonder why we had so much of them and all those pointing
to the patch list bruce maintains - are the urls to the mails there not
stable somehow ?


Stefan


Re: Not ready for 8.3

From
"Joshua D. Drake"
Date:
Oleg Bartunov wrote:
> On Tue, 15 May 2007, Joshua D. Drake wrote:
> 
>> Tsearch2 in core. I know that Tom has some reservations, he I and 
>> Treat all commented on how it was done and to my knowledge those 
>> reservations have not been resolved.
> 
> We'd like to know about these reservations. If I understand you mean 
> there are issues with the patch ? Our patch is several months old. We 
> permanently
> keep it in sync with CVS HEAD, latest version is 0.47.

http://archives.postgresql.org/pgsql-hackers/2007-01/msg01172.php

Sincerely,

Joshua D. Drake


Re: Not ready for 8.3

From
Alvaro Herrera
Date:
Stefan Kaltenbrunner wrote:
> Joshua D. Drake wrote:

> > PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
> > developed outside of core. Don't get me wrong I like the feature but it
> > can take advantage of facilities outside of core.
> 
> url fixed - I wonder why we had so much of them and all those pointing
> to the patch list bruce maintains - are the urls to the mails there not
> stable somehow ?

They are not stable.  The items should point to the archives, which are
supposedly more stable.  (I had already fixed one item in PatchStatus
this morning).  Really it would be much nicer to have links using the
Message-Id but I doubt that's at all doable.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.


Re: Not ready for 8.3

From
"Joshua D. Drake"
Date:
Alvaro Herrera wrote:
> Stefan Kaltenbrunner wrote:
>> Joshua D. Drake wrote:
> 
>>> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
>>> developed outside of core. Don't get me wrong I like the feature but it
>>> can take advantage of facilities outside of core.
>> url fixed - I wonder why we had so much of them and all those pointing
>> to the patch list bruce maintains - are the urls to the mails there not
>> stable somehow ?
> 
> They are not stable.  The items should point to the archives, which are
> supposedly more stable.  (I had already fixed one item in PatchStatus
> this morning).  Really it would be much nicer to have links using the
> Message-Id but I doubt that's at all doable.

I think we discussed it once... I don't remember the reason why we 
didn't go that direction.

Joshua D. Drake

> 



Re: Not ready for 8.3

From
Stefan Kaltenbrunner
Date:
Alvaro Herrera wrote:
> Stefan Kaltenbrunner wrote:
>> Joshua D. Drake wrote:
> 
>>> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
>>> developed outside of core. Don't get me wrong I like the feature but it
>>> can take advantage of facilities outside of core.
>> url fixed - I wonder why we had so much of them and all those pointing
>> to the patch list bruce maintains - are the urls to the mails there not
>> stable somehow ?
> 
> They are not stable.  The items should point to the archives, which are
> supposedly more stable.  (I had already fixed one item in PatchStatus
> this morning).  Really it would be much nicer to have links using the
> Message-Id but I doubt that's at all doable.

hrm - I see so is there a particular reason for that behaviour ?

I will work on pointing to the archives tomorrow ...


Stefan


Re: Not ready for 8.3

From
Aidan Van Dyk
Date:
 
> They are not stable.  The items should point to the archives, which are
> supposedly more stable.  (I had already fixed one item in PatchStatus
> this morning).  Really it would be much nicer to have links using the
> Message-Id but I doubt that's at all doable.

I use this all the time:http://news.gmane.org/find-root.php?message_id=$MSGID

It uses non-PostgreSQL infrastructure, but hey, I think gmane does list
archives better than PostgreSQL right now ;-)

Try it:http://news.gmane.org/find-root.php?message_id=20070515201925.GL12731@alvh.no-ip.org
a.

-- 
Aidan Van Dyk                                             Create like a god,
aidan@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.

Re: Not ready for 8.3

From
Alvaro Herrera
Date:
Stefan Kaltenbrunner wrote:
> Alvaro Herrera wrote:
> > Stefan Kaltenbrunner wrote:
> >> Joshua D. Drake wrote:
> > 
> >>> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
> >>> developed outside of core. Don't get me wrong I like the feature but it
> >>> can take advantage of facilities outside of core.
> >> url fixed - I wonder why we had so much of them and all those pointing
> >> to the patch list bruce maintains - are the urls to the mails there not
> >> stable somehow ?
> > 
> > They are not stable.  The items should point to the archives, which are
> > supposedly more stable.  (I had already fixed one item in PatchStatus
> > this morning).  Really it would be much nicer to have links using the
> > Message-Id but I doubt that's at all doable.
> 
> hrm - I see so is there a particular reason for that behaviour ?
> 
> I will work on pointing to the archives tomorrow ...

Bruce adds and removes emails from the "pgpatches" inbox and I assume he
regenerates the MHonarc archives when he does, which may change the URL
for each specific item.  I don't think it was ever intended that the
URLs were to be stable.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


Re: Not ready for 8.3

From
Magnus Hagander
Date:
Stefan Kaltenbrunner wrote:
>> They are not stable.  The items should point to the archives, which are
>> supposedly more stable.  (I had already fixed one item in PatchStatus
>> this morning).  Really it would be much nicer to have links using the
>> Message-Id but I doubt that's at all doable.
> 
> hrm - I see so is there a particular reason for that behaviour ?

They're stable until Bruce removes something from the queue. When
something is removed, it's renumbered.

It's how mhonarc works. It's the same with the archives - if we delete a
mail, they get renumbered. So we never should delete, we should just
blank out, but it has happened a couple of times.

//Magnus


Re: Not ready for 8.3

From
Alvaro Herrera
Date:
Aidan Van Dyk wrote:
>  
> > They are not stable.  The items should point to the archives, which are
> > supposedly more stable.  (I had already fixed one item in PatchStatus
> > this morning).  Really it would be much nicer to have links using the
> > Message-Id but I doubt that's at all doable.
> 
> I use this all the time:
>     http://news.gmane.org/find-root.php?message_id=$MSGID
> 
> It uses non-PostgreSQL infrastructure, but hey, I think gmane does list
> archives better than PostgreSQL right now ;-)
> 
> Try it:
>     http://news.gmane.org/find-root.php?message_id=20070515201925.GL12731@alvh.no-ip.org

Wow, this is really nice, thanks.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


Re: Not ready for 8.3

From
Oleg Bartunov
Date:
On Tue, 15 May 2007, Joshua D. Drake wrote:

> Oleg Bartunov wrote:
>> On Tue, 15 May 2007, Joshua D. Drake wrote:
>> 
>>> Tsearch2 in core. I know that Tom has some reservations, he I and Treat 
>>> all commented on how it was done and to my knowledge those reservations 
>>> have not been resolved.
>> 
>> We'd like to know about these reservations. If I understand you mean there 
>> are issues with the patch ? Our patch is several months old. We permanently
>> keep it in sync with CVS HEAD, latest version is 0.47.
>
> http://archives.postgresql.org/pgsql-hackers/2007-01/msg01172.php

there are several others threads.

well, we did answer all raised questions and I don't want to begin
another wave. The only problem I see is that some people doesn't like
introducing SQL syntax, they claim functions would be enough. SQL is a
nice language to manipulate db objects, finally. 
Also, select to fts_create_dictionary(bla-bla....) looks pretty ugly,
pretty artificial.

CREATE FULLTEXT DICTIONARY dictname 
[    {   INIT  init_function        | LEXIZE  lexize_function        | OPTION opt_text } 
[ ... ]] LIKE template_dictname;

select

>
> Sincerely,
>
> Joshua D. Drake
>
    Regards,        Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83


Re: Not ready for 8.3

From
Bruce Momjian
Date:
Gregory Stark wrote:
> "Bruce Momjian" <bruce@momjian.us> writes:
> 
> > I think the only other thing we _could_ do is to re-open normal 8.3
> > development, so we aren't hampering updates to trivial parts of the
> > code. Many of the patches now in the queue had been developed for months
> > before 8.3 started, so the hope is that we wouldn't have many more new
> > large patches, but several small ones we could deal with while we
> > whittle away at the larger patches during the next few months.
> >
> > The question is whether it is healthy for us to remain in feature freeze
> > for months, and if it is unhealthy, what are our options?
> 
> I don't see any reason development has to stop while the tree is in feature
> freeze. If it led to patches being ready for review and getting reviewed and
> committed early in the cycle rather than just before release I think it would
> actually be extremely healthy.

So you are saying just let people keep developing for 8.4 and we will
use it as soon as we start for 8.4.  OK.  We might get to a point where
we can just open development for 8.4, apply outstanding patches, and
head for beta.  ;-)

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
Bruce Momjian
Date:
Josh Berkus wrote:
> Bruce,
> 
> Realistically I just don't see getting everything in the ToDo patch list in; 
> my vote is that we start deferring stuff for 8.4 if it doesn't have a 
> reviewer, except for items which were submitted early in the cycle (and to 
> whom it would be unfair). 

It seems unfair to disinguish between early/last in cycle postings.

> If that means shortening the 8.4 cycle somewhat, I'm for that ... feature 
> freeze in Feburary would be even better than April, because it means we could 
> be in Beta for the May-June-July conferences, and increase our probability of 
> being able to release at a major conference or PostgreSQL conference.
> 
> Obviously for 8.4 reviewers need to start reviewing stuff from the first week 
> of the development cycle.  I also don't actually see anything wrong with a 
> 3-month feature freeze if we can somehow branch development earlier.  Easy 
> for me to say, I know. 

Yea, this is just pushing off work until later, which I don't support. 
There is no easy out here and I am afraid we will just have to accept a
3-month feature freeze.

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
Bruce Momjian
Date:
Joshua D. Drake wrote:
> > That is not fair to patch submitters, and pushes the problem to 8.4,
> > where it will be no better.
> 
> If it isn't done, it isn't done. We aren't dropping the patch. The patch 
> has been accepted, just not reviewed. It is just delayed.

Delayed isn't rejected, but it isn't accepted either.  I see that as
unfair to patch submitters, and not making things any easier.  It just
pushes our problem into 8.4.

> >> Sure it is, if we have a short release cycle. There are plenty of things 
> >> out there that are not quite ready yet, that could be if we released 
> >> again next January...
> > 
> > Huh, you didn't answer my issue about unfair.
> 
> Sure I did see above :). Unfair would be to let go all together. We are 
> just delaying. If we don't have reviewers, we don't have reviewers and 
> as I look at the open list we have more reviewers than we did for 8.2 so 
> I would suspect that 8.4 is going to go smoother as it is.

And even more patches for 8.4.

Given your list below, let me match it up with Tom's comments and see if
I can defer some patches that are not ready for application.  I think
bitmapped indexes are one of them.

---------------------------------------------------------------------------


> >> Leaving only those patches that have confirmed reviewers to be worked 
> >> through.
> >>
> >> FYI, whoever did that Todo:Patch status, Bravo! That is easily one of 
> >> the smallest but best improvements to the process I have seen in recent 
> >> memory.
> > 
> > Fairness and not pushing our problems out to the future --- address
> > those issues.
> 
> Life isn't fair. It isn't like we are telling these people to bugger 
> off. We are just delaying the review to what is a reasonable workload 
> for the current set of reviewers.

Again, it just pushed things off.  I see just keeping going as a better
approach.

> Life... is a perpetual problem. We do what we can, when we can, and the 
> best that we can.

Yea, but we have to do that best we can, not give up and hope something
improves for 8.4, because I see no basis for such a belief.

> Taking my suggestion above for leaving patches that don't have reviewers 
>   let's look at some of them.
> 
> Tsearch2 in core. I know that Tom has some reservations, he I and Treat 
> all commented on how it was done and to my knowledge those reservations 
> have not been resolved.
> 
> HOT is a huge patch that has gone round and round and round. Lots of 
> people want it, including me but it is invasive enough to where it may 
> due it good to work through another cycle.
> 
> Grouped Index Tuples. I like the result of this patch. I tested the 
> performance and it did work as advertised but we didn't get much 
> feedback as a whole from known hackers. Either there isn't much interest 
>   or we are too busy. Either way it is certainly not a critical patch 
> and can be pushed.
> 
> Concurrent psql, nifty but still trying to decide on actual interface.
> 
> Full Page Writes Improvement, doesn't actually do anything *yet* (as far 
> as I can tell) it just makes it so something can be done in the future. 
> It is also apparently a small patch.
> 
> UTF8 text matching performance improvements (todo wiki link broke), so I 
> don't have much comment.
> 
> On Disk Bitmap indexes (todo wiki link broke): Doesn't support Vacuum 
> (to my knowledge), community member tested, reported problems... no 
> response yet from author. The author is known to be time constrained, 
> boot it till 8.4.
> 
> Posix shared memory, not usable in current state per Tom's comments and 
> Apples, agreement. Dumped till 8.4 or further.
> 
> Stream bitmaps, apparently needed more info from Gavin (see previous 
> comment about author's time). No feed back since March 9th. Dumped till 8.4.
> 
> Bitmapscan change, this one is unfortunate because at the time Heikki 
> had resources and desire but was basically ignored. Sometimes we have to 
> punt although Heikki is doing patch review right now and it is not 
> unheard of for a patch reviewer to commit his own patch (in this case we 
> would need a comitter to actually accept it.).
> 
> Updateable cursors, apparently breaks explain and patch has been 
> updated. Punt!
> 
> Group Commit, Heikki has already suggested that it may be a good idea to 
> push for 8.4 on this patch: http://momjian.us/mhonarc/patches/msg00127.html
> 
> Index Advisor.. I think the link is wrong:
> 
> http://momjian.us/mhonarc/patches/msg00119.html
> 
> Ctid chain patch, apparently no discussion since last January even 
> though requests had been made to change the patch to some degree. Punt.
> I will note that no one was negative about the patch, it just doesn't 
> appear that the requests were ever finished.
> 
> Error correct for n_dead_tuples, patch was requested for hold in Feb. No 
> discussion since. Punt!
> 
> DSM, apparently includes n_dead_tuples, please remove n_dead_tuples line 
> on todo. Significant memory allocation enhancements are expected in 8.4 
> for this. Discussion died in April. Concerns were raised about how 
> memory is allocated (fixed, shared) which author already admints to 
> wanting to change for 8.4.
> 
> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be 
> developed outside of core. Don't get me wrong I like the feature but it 
> can take advantage of facilities outside of core.
> 
> So there ya go... thoughts, flames?
> 
> Sincerely,
> 
> Joshua D. Drake
> 
> 
> 
> 
> 
> 
> > 
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
>        subscribe-nomail command to majordomo@postgresql.org so that your
>        message can get through to the mailing list cleanly

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
Bruce Momjian
Date:
Andrew Dunstan wrote:
> >> I think the only other thing we _could_ do is to re-open normal 8.3
> >> development, so we aren't hampering updates to trivial parts of the
> >> code. Many of the patches now in the queue had been developed for months
> >> before 8.3 started, so the hope is that we wouldn't have many more new
> >> large patches, but several small ones we could deal with while we
> >> whittle away at the larger patches during the next few months.
> >>
> >> The question is whether it is healthy for us to remain in feature freeze
> >> for months, and if it is unhealthy, what are our options?
> >>     
> >
> > I don't see any reason development has to stop while the tree is in feature
> > freeze. If it led to patches being ready for review and getting reviewed and
> > committed early in the cycle rather than just before release I think it would
> > actually be extremely healthy.
> >
> >   
> 
> If we had multiple dev branches it might be more possible. That has its 
> own costs, of course - having a single dev branch makes management much 
> easier, and we never have to worry about things like merging.
> 
> Patches seem to be getting larger, more complex, and harder to review. 
> That is stressing our processes more than somewhat.
> 
> Short cycles would only make matters worse - the frictional cost of each 
> dev cycle is growing. I think at least we have learned not to repeat 
> this exercise.

Agreed.  Good summary.  Let's look at our problems honestly now and find
a direction, rather than pushing them off for later.

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
Alvaro Herrera
Date:
Bruce Momjian wrote:
> Gregory Stark wrote:
> > "Bruce Momjian" <bruce@momjian.us> writes:
> > 
> > > I think the only other thing we _could_ do is to re-open normal 8.3
> > > development, so we aren't hampering updates to trivial parts of the
> > > code. Many of the patches now in the queue had been developed for months
> > > before 8.3 started, so the hope is that we wouldn't have many more new
> > > large patches, but several small ones we could deal with while we
> > > whittle away at the larger patches during the next few months.
> > >
> > > The question is whether it is healthy for us to remain in feature freeze
> > > for months, and if it is unhealthy, what are our options?
> > 
> > I don't see any reason development has to stop while the tree is in feature
> > freeze. If it led to patches being ready for review and getting reviewed and
> > committed early in the cycle rather than just before release I think it would
> > actually be extremely healthy.
> 
> So you are saying just let people keep developing for 8.4 and we will
> use it as soon as we start for 8.4.  OK.  We might get to a point where
> we can just open development for 8.4, apply outstanding patches, and
> head for beta.  ;-)

This is what happens with the Linux kernel.  They have hundreds of
developers getting their hands dirty during a previous period.  Then
2.6.20 is released; the 2.6.21 "merge window" opens, and all sort of
patches are flooded in.  The merge window closes some time after that
(it's something like 2 or 3 weeks), and the stabilization period follows
(2 months?), during which 2.6.21-rc1, -rc2 etc are released.  When
stability is reached, 2.6.21 is released, and the cycle starts again.

Sadly, we are missing the hundreds of developers.  We are nowhere near
the scale of Linux.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.


Re: Not ready for 8.3

From
Bruce Momjian
Date:
Chris Browne wrote:
> josh@agliodbs.com (Josh Berkus) writes:
> > Bruce,
> >
> > Realistically I just don't see getting everything in the ToDo patch
> > list in; my vote is that we start deferring stuff for 8.4 if it
> > doesn't have a reviewer, except for items which were submitted early
> > in the cycle (and to whom it would be unfair).
> >
> > If that means shortening the 8.4 cycle somewhat, I'm for that
> > ... feature freeze in Feburary would be even better than April,
> > because it means we could be in Beta for the May-June-July
> > conferences, and increase our probability of being able to release
> > at a major conference or PostgreSQL conference.
> 
> If we're pushing stuff planned for 8.3 off into 8.4, then doesn't that
> mean that we'd be inclined to _lengthen_ the 8.4 cycle?
> 
> If we were to shorten the 8.4 cycle, that seems likely to me to worsen
> the problem, so that I'd expect to see even more stuff deferred for
> 8.5 than was the case with 8.3...

And, as I remember, we already deferred some stuff during 8.2 for 8.3. 
:-(

> > Obviously for 8.4 reviewers need to start reviewing stuff from the
> > first week of the development cycle.  I also don't actually see
> > anything wrong with a 3-month feature freeze if we can somehow
> > branch development earlier.  Easy for me to say, I know.
> 
> Well, if people are essentially already working on patches for 8.4
> *now*, but just deferring merging until after 8.3 is committed +
> released, then we've got 8.4 work underway already.
> 
> Unfortunately, that's also likely to worsen the problem of the
> reviewers' queues being even more overflowing.
> 
> I'd sorta like to see the Tom Lanes and Bruce Momjians of the project
> getting to have some time to work on things that *they'd* like add, as
> opposed to turning things into a spiral cycle where they keep spending
> more and more of their time reviewing patches, as opposed to doing
> things they find neat and new.  Too many iterations of that sort of
> thing, and they'll not want to ever see a patch again...

Another good summary.

Myself, I have always done what no one wanted to do so that more people
can do what they want to do, and I am OK with that.  It helps the
project move forward faster than if I did only what I wanted to do.

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
"Joshua D. Drake"
Date:
Oleg Bartunov wrote:
> On Tue, 15 May 2007, Joshua D. Drake wrote:
> 
>> Oleg Bartunov wrote:
>>> On Tue, 15 May 2007, Joshua D. Drake wrote:
>>>
>>>> Tsearch2 in core. I know that Tom has some reservations, he I and 
>>>> Treat all commented on how it was done and to my knowledge those 
>>>> reservations have not been resolved.
>>>
>>> We'd like to know about these reservations. If I understand you mean 
>>> there are issues with the patch ? Our patch is several months old. We 
>>> permanently
>>> keep it in sync with CVS HEAD, latest version is 0.47.
>>
>> http://archives.postgresql.org/pgsql-hackers/2007-01/msg01172.php
> 
> there are several others threads.
> 
> well, we did answer all raised questions and I don't want to begin
> another wave.

Did you address them? For example:

http://archives.postgresql.org/pgsql-hackers/2007-03/msg00914.php

Sincerely,

Joshua d. drake

 The only problem I see is that some people doesn't like
> introducing SQL syntax, they claim functions would be enough. SQL is a
> nice language to manipulate db objects, finally. Also, select to 
> fts_create_dictionary(bla-bla....) looks pretty ugly,
> pretty artificial.
> 
> CREATE FULLTEXT DICTIONARY dictname [    {   INIT  init_function
>         | LEXIZE  lexize_function
>         | OPTION opt_text } [ ... ]] LIKE template_dictname;
> 
> select
> 
>>
>> Sincerely,
>>
>> Joshua D. Drake
>>
> 
>     Regards,
>         Oleg
> _____________________________________________________________
> Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
> Sternberg Astronomical Institute, Moscow University, Russia
> Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
> phone: +007(495)939-16-83, +007(495)939-23-83
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 7: You can help support the PostgreSQL project by donating at
> 
>                http://www.postgresql.org/about/donate
> 



Re: Not ready for 8.3

From
Bruce Momjian
Date:
Joshua D. Drake wrote:
> Oleg Bartunov wrote:
> > On Tue, 15 May 2007, Joshua D. Drake wrote:
> > 
> >> Tsearch2 in core. I know that Tom has some reservations, he I and 
> >> Treat all commented on how it was done and to my knowledge those 
> >> reservations have not been resolved.
> > 
> > We'd like to know about these reservations. If I understand you mean 
> > there are issues with the patch ? Our patch is several months old. We 
> > permanently
> > keep it in sync with CVS HEAD, latest version is 0.47.
> 
> http://archives.postgresql.org/pgsql-hackers/2007-01/msg01172.php

Oleg has been producing new versions of his patch, and no one has
objected to any of it, so I assume all the issues were addressed.

As far was whether tsearch2 should be in core, I think most agreed that
SQL support for full-text would make it easier to use, and that it is a
fundamental technology.  As I remember, the only holdback was full
multi-byte support, but that is done.

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
Oleg Bartunov
Date:
On Tue, 15 May 2007, Joshua D. Drake wrote:

> Oleg Bartunov wrote:
>> On Tue, 15 May 2007, Joshua D. Drake wrote:
>> 
>>> Oleg Bartunov wrote:
>>>> On Tue, 15 May 2007, Joshua D. Drake wrote:
>>>> 
>>>>> Tsearch2 in core. I know that Tom has some reservations, he I and Treat 
>>>>> all commented on how it was done and to my knowledge those reservations 
>>>>> have not been resolved.
>>>> 
>>>> We'd like to know about these reservations. If I understand you mean 
>>>> there are issues with the patch ? Our patch is several months old. We 
>>>> permanently
>>>> keep it in sync with CVS HEAD, latest version is 0.47.
>>> 
>>> http://archives.postgresql.org/pgsql-hackers/2007-01/msg01172.php
>> 
>> there are several others threads.
>> 
>> well, we did answer all raised questions and I don't want to begin
>> another wave.
>
> Did you address them? For example:
>
> http://archives.postgresql.org/pgsql-hackers/2007-03/msg00914.php

we added support of gin index for text data type, so one could just
    create index fts_idx on TABLE using gin(body);

and should be able to do full-text search. Of course, ranking is not
available, since index doesn't store positional information.

 About this syntax see
http://archives.postgresql.org/pgsql-hackers/2007-03/msg00936.php

>
> Sincerely,
>
> Joshua d. drake
>
>
> The only problem I see is that some people doesn't like
>> introducing SQL syntax, they claim functions would be enough. SQL is a
>> nice language to manipulate db objects, finally. Also, select to 
>> fts_create_dictionary(bla-bla....) looks pretty ugly,
>> pretty artificial.
>> 
>> CREATE FULLTEXT DICTIONARY dictname [    {   INIT  init_function
>>         | LEXIZE  lexize_function
>>         | OPTION opt_text } [ ... ]] LIKE template_dictname;
>> 
>> select
>> 
>>> 
>>> Sincerely,
>>> 
>>> Joshua D. Drake
>>> 
>>
>>     Regards,
>>         Oleg
>> _____________________________________________________________
>> Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
>> Sternberg Astronomical Institute, Moscow University, Russia
>> Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
>> phone: +007(495)939-16-83, +007(495)939-23-83
>> 
>> ---------------------------(end of broadcast)---------------------------
>> TIP 7: You can help support the PostgreSQL project by donating at
>>
>>                http://www.postgresql.org/about/donate
>> 
>
    Regards,        Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83


Re: Not ready for 8.3

From
Bruce Momjian
Date:
Oleg Bartunov wrote:
> On Tue, 15 May 2007, Joshua D. Drake wrote:
> 
> > Tsearch2 in core. I know that Tom has some reservations, he I and Treat all 
> > commented on how it was done and to my knowledge those reservations have not 
> > been resolved.
> 
> We'd like to know about these reservations. If I understand you mean there 
> are issues with the patch ? Our patch is several months old. We permanently
> keep it in sync with CVS HEAD, latest version is 0.47.
> 
> We're really bored with the whole process of development. We're not
> full-time developers, we just devote our spare time which we withdraw from 
> time we should spend on many other things. We have no time even to discuss
> those very useful threads about community management, patches, etc. We just
> rely on community decision.

This is a good example of how developers can get frustrated.  Pushing a
patch to 8.4 that was completed before 8.3 feature freeze is guaranteed
to add to that --- and if we lose our developers, we might as well shut
down the PostgreSQL project.

One good thing is that we have community discussion this now, so at
least we are focusing on it.

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
Bruce Momjian
Date:
Stefan Kaltenbrunner wrote:
> > PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
> > developed outside of core. Don't get me wrong I like the feature but it
> > can take advantage of facilities outside of core.
> 
> url fixed - I wonder why we had so much of them and all those pointing
> to the patch list bruce maintains - are the urls to the mails there not
> stable somehow ?

Yep, the URLs are unstable.  I am using mhonarc and regenerate the page
when the patches mbox file changes.  When I append to the mailbox, the
URLs don't change, but when I delete, they do.

Does anyone know a way to make those URLs stable?

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
Bruce Momjian
Date:
Alvaro Herrera wrote:
> Stefan Kaltenbrunner wrote:
> > Alvaro Herrera wrote:
> > > Stefan Kaltenbrunner wrote:
> > >> Joshua D. Drake wrote:
> > > 
> > >>> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
> > >>> developed outside of core. Don't get me wrong I like the feature but it
> > >>> can take advantage of facilities outside of core.
> > >> url fixed - I wonder why we had so much of them and all those pointing
> > >> to the patch list bruce maintains - are the urls to the mails there not
> > >> stable somehow ?
> > > 
> > > They are not stable.  The items should point to the archives, which are
> > > supposedly more stable.  (I had already fixed one item in PatchStatus
> > > this morning).  Really it would be much nicer to have links using the
> > > Message-Id but I doubt that's at all doable.
> > 
> > hrm - I see so is there a particular reason for that behaviour ?
> > 
> > I will work on pointing to the archives tomorrow ...
> 
> Bruce adds and removes emails from the "pgpatches" inbox and I assume he
> regenerates the MHonarc archives when he does, which may change the URL
> for each specific item.  I don't think it was ever intended that the
> URLs were to be stable.

Right.  Is there no way to make the mhonarc URLs stable?

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
Alvaro Herrera
Date:
Bruce Momjian wrote:
> Stefan Kaltenbrunner wrote:
> > > PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
> > > developed outside of core. Don't get me wrong I like the feature but it
> > > can take advantage of facilities outside of core.
> > 
> > url fixed - I wonder why we had so much of them and all those pointing
> > to the patch list bruce maintains - are the urls to the mails there not
> > stable somehow ?
> 
> Yep, the URLs are unstable.  I am using mhonarc and regenerate the page
> when the patches mbox file changes.  When I append to the mailbox, the
> URLs don't change, but when I delete, they do.
> 
> Does anyone know a way to make those URLs stable?

Does it matter?  I assume that we should not consider pgpatches to be a
stable source anyway ... those emails are there just as a reminder for
short-lived information anyway, so why bother?

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


Re: Not ready for 8.3

From
"Joshua D. Drake"
Date:
Bruce Momjian wrote:
> Oleg Bartunov wrote:
> This is a good example of how developers can get frustrated.  Pushing a
> patch to 8.4 that was completed before 8.3 feature freeze is guaranteed
> to add to that --- and if we lose our developers, we might as well shut
> down the PostgreSQL project.

Let's not call the sky falling just yet. We are doing what we can. There 
are several sub projects afoot trying to eliminate some of this pain. If 
you recall, Lukas and I both spent time before feature freeze helping 
keep people in communication.

Secondly and although you aren't 100% on board, there is also the 
tracker project.

Lastly, more people are starting to review code. Even people that can't 
code are starting to at least participate in testing features.

> 
> One good thing is that we have community discussion this now, so at
> least we are focusing on it.
> 

Agreed, but it certainly is not a critical mass problem either. We are 
starting to bounce off the wall, and are starting to take a step back to 
reflect what size ladder or rope we need to get over it.

Joshua D. Drake



Re: Not ready for 8.3

From
"Joshua D. Drake"
Date:
Alvaro Herrera wrote:
> Bruce Momjian wrote:
>> Stefan Kaltenbrunner wrote:
>>>> PL/PSM, link wrong (goes to guc temp_tablespaces). This can certainly be
>>>> developed outside of core. Don't get me wrong I like the feature but it
>>>> can take advantage of facilities outside of core.
>>> url fixed - I wonder why we had so much of them and all those pointing
>>> to the patch list bruce maintains - are the urls to the mails there not
>>> stable somehow ?
>> Yep, the URLs are unstable.  I am using mhonarc and regenerate the page
>> when the patches mbox file changes.  When I append to the mailbox, the
>> URLs don't change, but when I delete, they do.
>>
>> Does anyone know a way to make those URLs stable?
> 
> Does it matter?  I assume that we should not consider pgpatches to be a
> stable source anyway ... those emails are there just as a reminder for
> short-lived information anyway, so why bother?

Yeah let's just source archives, the urls are reasonable stable there.

Joshua D. Drake

> 



Re: Not ready for 8.3

From
Bruce Momjian
Date:
Joshua D. Drake wrote:
> Bruce Momjian wrote:
> > Oleg Bartunov wrote:
> > This is a good example of how developers can get frustrated.  Pushing a
> > patch to 8.4 that was completed before 8.3 feature freeze is guaranteed
> > to add to that --- and if we lose our developers, we might as well shut
> > down the PostgreSQL project.
> 
> Let's not call the sky falling just yet. We are doing what we can. There 
> are several sub projects afoot trying to eliminate some of this pain. If 
> you recall, Lukas and I both spent time before feature freeze helping 
> keep people in communication.
> 
> Secondly and although you aren't 100% on board, there is also the 
> tracker project.
> 
> Lastly, more people are starting to review code. Even people that can't 
> code are starting to at least participate in testing features.
> 
> > 
> > One good thing is that we have community discussion this now, so at
> > least we are focusing on it.
> > 
> 
> Agreed, but it certainly is not a critical mass problem either. We are 
> starting to bounce off the wall, and are starting to take a step back to 
> reflect what size ladder or rope we need to get over it.

Right.  I just don't want a short-term solution (push patches for 8.4)
that avoids a long-term analysis.

Lots of people complained we didn't have a patches status page.  Someone
has done that and it is current, but it is not moving us forward fast
enough.

What is our next move, or do we just keep moving and settle on August?

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
"Joshua D. Drake"
Date:
Bruce Momjian wrote:
> Joshua D. Drake wrote:

>>> One good thing is that we have community discussion this now, so at
>>> least we are focusing on it.
>>>
>> Agreed, but it certainly is not a critical mass problem either. We are 
>> starting to bounce off the wall, and are starting to take a step back to 
>> reflect what size ladder or rope we need to get over it.
> 
> Right.  I just don't want a short-term solution (push patches for 8.4)
> that avoids a long-term analysis.
> 
> Lots of people complained we didn't have a patches status page.  Someone
> has done that and it is current, but it is not moving us forward fast
> enough.
> 
> What is our next move, or do we just keep moving and settle on August?

Well I think what you said about reviewing my list versus Tom's triage 
is a good start. If there are things we can push, let's push them. Plus 
based on this conversation I think we need reviewers to step up and 
say... "Hey! this is where I am at...".

Sincerely,

Joshua D. Drake





Re: Not ready for 8.3

From
Bruce Momjian
Date:
Joshua D. Drake wrote:
> Bruce Momjian wrote:
> > Joshua D. Drake wrote:
> 
> >>> One good thing is that we have community discussion this now, so at
> >>> least we are focusing on it.
> >>>
> >> Agreed, but it certainly is not a critical mass problem either. We are 
> >> starting to bounce off the wall, and are starting to take a step back to 
> >> reflect what size ladder or rope we need to get over it.
> > 
> > Right.  I just don't want a short-term solution (push patches for 8.4)
> > that avoids a long-term analysis.
> > 
> > Lots of people complained we didn't have a patches status page.  Someone
> > has done that and it is current, but it is not moving us forward fast
> > enough.
> > 
> > What is our next move, or do we just keep moving and settle on August?
> 
> Well I think what you said about reviewing my list versus Tom's triage 
> is a good start. If there are things we can push, let's push them. Plus 
> based on this conversation I think we need reviewers to step up and 
> say... "Hey! this is where I am at...".

Agreed. I have already contacted all reviewers individually to ask them
for a status, and mentioned I was concerned.

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
"Jim C. Nasby"
Date:
On Tue, May 15, 2007 at 12:42:28PM -0400, Bruce Momjian wrote:
> Joshua D. Drake wrote:
> > > Patch status:
> > > 
> > >     http://developer.postgresql.org/index.php/Todo:PatchStatus
> > 
> > If... this is actually a problem (I leave to other committers and 
> > reviewers to comment) then I suggest we push all patches without a 
> > reviewer as of now to 8.4.
> > 
> > Leaving only those patches that have confirmed reviewers to be worked 
> > through.
> > 
> > FYI, whoever did that Todo:Patch status, Bravo! That is easily one of 
> > the smallest but best improvements to the process I have seen in recent 
> > memory.
> 
> I did one of those for previous releases.  I guess you forgot.  It was
> done by someone else this time only because I was going to be traveling.

Unless you're really in love with doing that sort of thing it's really
good that someone else did it. You're one of a handful of folks that can
actually review patches, while there's any number of us that can update
a wiki.

Speaking of reviewers... should we put some thought into how we can
increase the number of people who can review code? It seems that's one
of our biggest bottlenecks...
-- 
Jim Nasby                                      decibel@decibel.org
EnterpriseDB      http://enterprisedb.com      512.569.9461 (cell)


Re: Not ready for 8.3

From
"Jim C. Nasby"
Date:
On Tue, May 15, 2007 at 10:32:14PM +0200, Magnus Hagander wrote:
> Stefan Kaltenbrunner wrote:
> >> They are not stable.  The items should point to the archives, which are
> >> supposedly more stable.  (I had already fixed one item in PatchStatus
> >> this morning).  Really it would be much nicer to have links using the
> >> Message-Id but I doubt that's at all doable.
> > 
> > hrm - I see so is there a particular reason for that behaviour ?
> 
> They're stable until Bruce removes something from the queue. When
> something is removed, it's renumbered.
> 
> It's how mhonarc works. It's the same with the archives - if we delete a
> mail, they get renumbered. So we never should delete, we should just
> blank out, but it has happened a couple of times.

Isn't there any other archiver we could use? The lack of URL stability
in mhonarc is bad enough, but the cross-month issue is just horrible.
-- 
Jim Nasby                                      decibel@decibel.org
EnterpriseDB      http://enterprisedb.com      512.569.9461 (cell)


Re: Not ready for 8.3

From
Bruce Momjian
Date:
Jim C. Nasby wrote:
> On Tue, May 15, 2007 at 12:42:28PM -0400, Bruce Momjian wrote:
> > Joshua D. Drake wrote:
> > > > Patch status:
> > > > 
> > > >     http://developer.postgresql.org/index.php/Todo:PatchStatus
> > > 
> > > If... this is actually a problem (I leave to other committers and 
> > > reviewers to comment) then I suggest we push all patches without a 
> > > reviewer as of now to 8.4.
> > > 
> > > Leaving only those patches that have confirmed reviewers to be worked 
> > > through.
> > > 
> > > FYI, whoever did that Todo:Patch status, Bravo! That is easily one of 
> > > the smallest but best improvements to the process I have seen in recent 
> > > memory.
> > 
> > I did one of those for previous releases.  I guess you forgot.  It was
> > done by someone else this time only because I was going to be traveling.
> 
> Unless you're really in love with doing that sort of thing it's really
> good that someone else did it. You're one of a handful of folks that can
> actually review patches, while there's any number of us that can update
> a wiki.

Sure.  Good idea.  All I had to do was ask (10 times, though).  ;-)

> Speaking of reviewers... should we put some thought into how we can
> increase the number of people who can review code? It seems that's one
> of our biggest bottlenecks...

Sure.  That is going to move us forward.

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
Bruce Momjian
Date:
Jim C. Nasby wrote:
> On Tue, May 15, 2007 at 10:32:14PM +0200, Magnus Hagander wrote:
> > Stefan Kaltenbrunner wrote:
> > >> They are not stable.  The items should point to the archives, which are
> > >> supposedly more stable.  (I had already fixed one item in PatchStatus
> > >> this morning).  Really it would be much nicer to have links using the
> > >> Message-Id but I doubt that's at all doable.
> > > 
> > > hrm - I see so is there a particular reason for that behaviour ?
> > 
> > They're stable until Bruce removes something from the queue. When
> > something is removed, it's renumbered.
> > 
> > It's how mhonarc works. It's the same with the archives - if we delete a
> > mail, they get renumbered. So we never should delete, we should just
> > blank out, but it has happened a couple of times.
> 
> Isn't there any other archiver we could use? The lack of URL stability
> in mhonarc is bad enough, but the cross-month issue is just horrible.

Agreed.  It is just crippled.

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
"Jim C. Nasby"
Date:
On Tue, May 15, 2007 at 07:01:39PM -0400, Bruce Momjian wrote:
> > Unless you're really in love with doing that sort of thing it's really
> > good that someone else did it. You're one of a handful of folks that can
> > actually review patches, while there's any number of us that can update
> > a wiki.
> 
> Sure.  Good idea.  All I had to do was ask (10 times, though).  ;-)
Even having to ask 10 times undoubtedly was a lot faster than doing it
yourself though. :)

> > Speaking of reviewers... should we put some thought into how we can
> > increase the number of people who can review code? It seems that's one
> > of our biggest bottlenecks...
> 
> Sure.  That is going to move us forward.

So... any ideas? :)
-- 
Jim Nasby                                      decibel@decibel.org
EnterpriseDB      http://enterprisedb.com      512.569.9461 (cell)


Re: Not ready for 8.3

From
Bruce Momjian
Date:
Jim C. Nasby wrote:
> On Tue, May 15, 2007 at 07:01:39PM -0400, Bruce Momjian wrote:
> > > Unless you're really in love with doing that sort of thing it's really
> > > good that someone else did it. You're one of a handful of folks that can
> > > actually review patches, while there's any number of us that can update
> > > a wiki.
> > 
> > Sure.  Good idea.  All I had to do was ask (10 times, though).  ;-)
>  
> Even having to ask 10 times undoubtedly was a lot faster than doing it
> yourself though. :)

Very true.

> > > Speaking of reviewers... should we put some thought into how we can
> > > increase the number of people who can review code? It seems that's one
> > > of our biggest bottlenecks...
> > 
> > Sure.  That is going to move us forward.
> 
> So... any ideas? :)

No, that is my problem.  I was hoping others would come up with some.  I
think communicating the problem is part of the solution.

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
Tatsuo Ishii
Date:
> Stefan Kaltenbrunner wrote:
> >> They are not stable.  The items should point to the archives, which are
> >> supposedly more stable.  (I had already fixed one item in PatchStatus
> >> this morning).  Really it would be much nicer to have links using the
> >> Message-Id but I doubt that's at all doable.
> > 
> > hrm - I see so is there a particular reason for that behaviour ?
> 
> They're stable until Bruce removes something from the queue. When
> something is removed, it's renumbered.
> 
> It's how mhonarc works. It's the same with the archives - if we delete a
> mail, they get renumbered. So we never should delete, we should just
> blank out, but it has happened a couple of times.

As I proposed for many times, why don't we add message number to each
subject line in mail? For example like this:

[HACKERS: 12345] Re: Not ready for 8.3

This way, we could always obtain stable (logical) pointer, without
reling on particular archival infrastructure.
--
Tatsuo Ishii
SRA OSS, Inc. Japan


Re: Not ready for 8.3

From
"Nathan Buchanan"
Date:
On 5/15/07, Tatsuo Ishii <ishii@postgresql.org> wrote:
> Stefan Kaltenbrunner wrote:
> >> They are not stable.  The items should point to the archives, which are
> >> supposedly more stable.  (I had already fixed one item in PatchStatus
> >> this morning).  Really it would be much nicer to have links using the
> >> Message-Id but I doubt that's at all doable.
> >
> > hrm - I see so is there a particular reason for that behaviour ?
>
> They're stable until Bruce removes something from the queue. When
> something is removed, it's renumbered.
>
> It's how mhonarc works. It's the same with the archives - if we delete a
> mail, they get renumbered. So we never should delete, we should just
> blank out, but it has happened a couple of times.

As I proposed for many times, why don't we add message number to each
subject line in mail? For example like this:

[HACKERS: 12345] Re: Not ready for 8.3

This way, we could always obtain stable (logical) pointer, without
reling on particular archival infrastructure.

This sounds like a good idea to me - though I'm just a lurker on the list.

Nathan

Re: Not ready for 8.3

From
Bruce Momjian
Date:
Tatsuo Ishii wrote:
> > Stefan Kaltenbrunner wrote:
> > >> They are not stable.  The items should point to the archives, which are
> > >> supposedly more stable.  (I had already fixed one item in PatchStatus
> > >> this morning).  Really it would be much nicer to have links using the
> > >> Message-Id but I doubt that's at all doable.
> > > 
> > > hrm - I see so is there a particular reason for that behaviour ?
> > 
> > They're stable until Bruce removes something from the queue. When
> > something is removed, it's renumbered.
> > 
> > It's how mhonarc works. It's the same with the archives - if we delete a
> > mail, they get renumbered. So we never should delete, we should just
> > blank out, but it has happened a couple of times.
> 
> As I proposed for many times, why don't we add message number to each
> subject line in mail? For example like this:
> 
> [HACKERS: 12345] Re: Not ready for 8.3
> 
> This way, we could always obtain stable (logical) pointer, without
> reling on particular archival infrastructure.

While that helps, it does not actually allow software to link the
threads together, i.e. it can't tell which is the head and tail of the
thread.  The actual software needs to improve because the thread
information is already part of the email headers.

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Not ready for 8.3

From
Robert Treat
Date:
On Tuesday 15 May 2007 16:48, Bruce Momjian wrote:
> Joshua D. Drake wrote:
> > > That is not fair to patch submitters, and pushes the problem to 8.4,
> > > where it will be no better.
> >
> > If it isn't done, it isn't done. We aren't dropping the patch. The patch
> > has been accepted, just not reviewed. It is just delayed.
>
> Delayed isn't rejected, but it isn't accepted either.  I see that as
> unfair to patch submitters, and not making things any easier.  It just
> pushes our problem into 8.4.
>

So, I'm not sure that these are actual rules, but it's how I have always seen 
the process working:

a) if we cannot resolve a submitted patch of technical issues within a 
reasonable time, it gets pushed to the next release. 

it is at the discression of -core/reviewer to determine both what are 
unresolvable issues and what is an unreasonable time... but I think patches 
that are not making progress within a period of months probably qualify on 
both counts.   

b) the development branch for the next release is not opened until delayed 
patches are dealt with 

dealing with a patch may not mean committing it, if it determined that the 
flaws are enough that it just isnt ready, or the author decides they aren't 
ready to devote time to it. 

i think everyone understands we all want everyones patches to get in, but it 
isn't realistic to think that will happen every release.  

one of the reasons we have a feature freeze is because the community has 
decided that holding up a release for feature X isn't desirable; aiui all of 
the patches in the current queue have been given a once over, and some more 
than that; i'm sure that cycle could go on quite some time for some patches, 
so as unfortunate as it is, I think at some point you have to just suck it up 
and call it day.   all imho.

-- 
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL


Re: Not ready for 8.3

From
Josh Berkus
Date:
Bruce,

> It seems unfair to disinguish between early/last in cycle postings.

I think it's fair.  A patch which was submitted for 8.2 and didn't get in 
should get consideration over a patch which was still in prototype form a 
week before feature freeze.  Also, I really don't think it's a crime to push 
back patches which are under active redevelopment (seqscan) if that 
redevelopment isn't a result of review neglect.

> > Obviously for 8.4 reviewers need to start reviewing stuff from the first
> > week of the development cycle.  I also don't actually see anything wrong
> > with a 3-month feature freeze if we can somehow branch development
> > earlier.  Easy for me to say, I know.
>
> Yea, this is just pushing off work until later, which I don't support.
> There is no easy out here and I am afraid we will just have to accept a
> 3-month feature freeze.

I think that may be where we're heading.  In that case, we may need to talk 
about branching earlier so that developers can work on the new version who 
are frozen out of the in-process one.

> Oleg has been producing new versions of his patch, and no one has
> objected to any of it, so I assume all the issues were addressed.
>
> As far was whether tsearch2 should be in core, I think most agreed that
> SQL support for full-text would make it easier to use, and that it is a
> fundamental technology.  As I remember, the only holdback was full
> multi-byte support, but that is done.

I'm going to echo Bruce on this; I've mentioned that TSearch was going into 
Core at conferences and the reaction from existing users has been very 
enthusiastic, ranging from "yippee!" to "about time!".  Our users hate the 
fact that FTS is a separate module.

-- 
Josh Berkus
PostgreSQL @ Sun
San Francisco


Re: Not ready for 8.3

From
Tatsuo Ishii
Date:
> Tatsuo Ishii wrote:
> > > Stefan Kaltenbrunner wrote:
> > > >> They are not stable.  The items should point to the archives, which are
> > > >> supposedly more stable.  (I had already fixed one item in PatchStatus
> > > >> this morning).  Really it would be much nicer to have links using the
> > > >> Message-Id but I doubt that's at all doable.
> > > > 
> > > > hrm - I see so is there a particular reason for that behaviour ?
> > > 
> > > They're stable until Bruce removes something from the queue. When
> > > something is removed, it's renumbered.
> > > 
> > > It's how mhonarc works. It's the same with the archives - if we delete a
> > > mail, they get renumbered. So we never should delete, we should just
> > > blank out, but it has happened a couple of times.
> > 
> > As I proposed for many times, why don't we add message number to each
> > subject line in mail? For example like this:
> > 
> > [HACKERS: 12345] Re: Not ready for 8.3
> > 
> > This way, we could always obtain stable (logical) pointer, without
> > reling on particular archival infrastructure.
> 
> While that helps, it does not actually allow software to link the
> threads together, i.e. it can't tell which is the head and tail of the
> thread.  The actual software needs to improve because the thread
> information is already part of the email headers.

I know. My point is, subject-with-sequence will guarantee at least we
do not lose the pointer completely.

Also it will help us searching particular message in the private mail
box.
--
Tatsuo Ishii
SRA OSS, Inc. Japan


Re: Not ready for 8.3

From
Andrew Dunstan
Date:

Josh Berkus wrote:
>> Yea, this is just pushing off work until later, which I don't support.
>> There is no easy out here and I am afraid we will just have to accept a
>> 3-month feature freeze.
>>     
>
> I think that may be where we're heading.  In that case, we may need to talk 
> about branching earlier so that developers can work on the new version who 
> are frozen out of the in-process one.
>
>   


I've argued this in the past. But be aware that it will make more work 
for committers. It is very far from cost-free.

cheers

andrew



Re: Not ready for 8.3

From
Dave Page
Date:
Jim C. Nasby wrote:
> On Tue, May 15, 2007 at 10:32:14PM +0200, Magnus Hagander wrote:
>> Stefan Kaltenbrunner wrote:
>>>> They are not stable.  The items should point to the archives, which are
>>>> supposedly more stable.  (I had already fixed one item in PatchStatus
>>>> this morning).  Really it would be much nicer to have links using the
>>>> Message-Id but I doubt that's at all doable.
>>> hrm - I see so is there a particular reason for that behaviour ?
>> They're stable until Bruce removes something from the queue. When
>> something is removed, it's renumbered.
>>
>> It's how mhonarc works. It's the same with the archives - if we delete a
>> mail, they get renumbered. So we never should delete, we should just
>> blank out, but it has happened a couple of times.
> 
> Isn't there any other archiver we could use? The lack of URL stability
> in mhonarc is bad enough, but the cross-month issue is just horrible.

Nothing useful last time I looked (a year or two back admittedley). I
have a design for one in mind that I was looking to prototype - there
are some php classes that would make it quite simple to get messages
into a database either via procmail, or from an mbox.

the stumbling block I was running into was rewriting the old archives
URLs to the new ones.

Regards, Dave


Re: Not ready for 8.3

From
tomas@tuxteam.de
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, May 16, 2007 at 10:16:43AM +0900, Tatsuo Ishii wrote:
> > Stefan Kaltenbrunner wrote:
> > >> They are not stable. [...]

> As I proposed for many times, why don't we add message number to each
> subject line in mail? For example like this:
> 
> [HACKERS: 12345] Re: Not ready for 8.3

What I don't understand is why mailing list software doesn't use message
ID as primary index. Granted, it's ugly, but it is globally stable (and
hopefully unique) _across all mailboxes_

Sorry for the random rant
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFGSrYGBcgs9XrR2kYRAlY2AJ9Vq5JZZEqX/y/DkN/HJ4Wg47RMyQCfbdgh
Z6KnR4eJHh/oDHr7GI/OiPU=
=Nxev
-----END PGP SIGNATURE-----



Re: Not ready for 8.3

From
Aidan Van Dyk
Date:
* Tatsuo Ishii <ishii@postgresql.org> [070515 21:19]:
> As I proposed for many times, why don't we add message number to each
> subject line in mail? For example like this:
> 
> [HACKERS: 12345] Re: Not ready for 8.3
> 
> This way, we could always obtain stable (logical) pointer, without
> reling on particular archival infrastructure.

Isn't that what the "Message-Id" field is for?

http://news.gmane.org/find-root.php?message_id=20070516.101643.94564776.t-ishii@sraoss.co.jp
a.

-- 
Aidan Van Dyk                                             Create like a god,
aidan@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.

Re: Not ready for 8.3

From
Tatsuo Ishii
Date:
> * Tatsuo Ishii <ishii@postgresql.org> [070515 21:19]:
>  
> > As I proposed for many times, why don't we add message number to each
> > subject line in mail? For example like this:
> > 
> > [HACKERS: 12345] Re: Not ready for 8.3
> > 
> > This way, we could always obtain stable (logical) pointer, without
> > reling on particular archival infrastructure.
> 
> Isn't that what the "Message-Id" field is for?
> 
> http://news.gmane.org/find-root.php?message_id=20070516.101643.94564776.t-ishii@sraoss.co.jp
> a.

Maybe. However I think "subject-sequence" has some advantages over
Message-Id:

- Easy to identify. Message-Id may not appear on some MUA with default setting

- More handy than lengthy message Id

- Easy to detect messages not delivered, by knowing that the sequence number was skipped
--
Tatsuo Ishii
SRA OSS, Inc. Japan


Re: Not ready for 8.3

From
"Robert Haas"
Date:
> I'm going to echo Bruce on this; I've mentioned that TSearch was going

> into Core at conferences and the reaction from existing users has been

> very enthusiastic, ranging from "yippee!" to "about time!".  Our users

> hate the fact that FTS is a separate module.

Here here.

And with respect to the debate about syntax, who cares?  I think I
prefer introducing real SQL-ish syntax over a bunch of pg_* functions,
but doing it either way is IMHO better than doing nothing.

...Robert


Re: Not ready for 8.3

From
Peter Eisentraut
Date:
Am Mittwoch, 16. Mai 2007 05:37 schrieb Josh Berkus:
> In that case, we may need to talk
> about branching earlier so that developers can work on the new version who
> are frozen out of the in-process one.

Well, we could branch right now, but who is going to commit anything into that 
new head branch?  The committers are already claimed to be falling behind.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/


Re: Not ready for 8.3

From
Peter Eisentraut
Date:
Am Mittwoch, 16. Mai 2007 05:37 schrieb Josh Berkus:
> I'm going to echo Bruce on this; I've mentioned that TSearch was going into
> Core at conferences and the reaction from existing users has been very
> enthusiastic, ranging from "yippee!" to "about time!".  Our users hate the
> fact that FTS is a separate module.

At the same time they are subconsciously counting on us to make decisions 
based on rationality, not on enthusiasm.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/


Re: Not ready for 8.3

From
Andrew Dunstan
Date:

Robert Haas wrote:
>>  Our users hate the fact that FTS is a separate module.
>>     
>
> Here here.
>   

Where? Where? Oh, you mean "Hear Hear." (sorry - one of my pet peeves)
> And with respect to the debate about syntax, who cares?  I think I
> prefer introducing real SQL-ish syntax over a bunch of pg_* functions,
> but doing it either way is IMHO better than doing nothing.
>
>
>   

We do have a responsibility, I think, to keep the grammar fairly clean, 
so the answer to your question "who cares?" is "we do."

That said, last time I looked most of the warts seemed to have been 
knocked off, IIRC, and the functional syntax would have been 
sufficiently ugly and cumbersome to weigh against it. So, like most 
others, I'm in favor of going with this unless there is some 
overwhelming reason I haven't heard of yet not to.

cheers

andrew




Re: Not ready for 8.3

From
Alvaro Herrera
Date:
tomas@tuxteam.de wrote:
> On Wed, May 16, 2007 at 10:16:43AM +0900, Tatsuo Ishii wrote:
> > > Stefan Kaltenbrunner wrote:
> > > >> They are not stable. [...]
> 
> > As I proposed for many times, why don't we add message number to each
> > subject line in mail? For example like this:
> > 
> > [HACKERS: 12345] Re: Not ready for 8.3
> 
> What I don't understand is why mailing list software doesn't use message
> ID as primary index. Granted, it's ugly, but it is globally stable (and
> hopefully unique) _across all mailboxes_

It does.  The problem is that it only considers messages posted in the
last calendar month.  So on the 1st of each month, all threads "start
again" as far as the archive goes.

There is just one remaining problem: Outlook and derivatives don't set
the In-Reply-To: nor References: headers.  This breaks the threads (the
best the software can do is group the messages by subject, but it works
only partially).

I've tried a coupld of times to decode how the Thread-Id stuff works,
which is what Outlook seems to use, but I haven't been able to detect a
pattern.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.


Re: Not ready for 8.3

From
Alvaro Herrera
Date:
Tatsuo Ishii wrote:
> > * Tatsuo Ishii <ishii@postgresql.org> [070515 21:19]:
> >  
> > > As I proposed for many times, why don't we add message number to each
> > > subject line in mail? For example like this:
> > > 
> > > [HACKERS: 12345] Re: Not ready for 8.3
> > > 
> > > This way, we could always obtain stable (logical) pointer, without
> > > reling on particular archival infrastructure.
> > 
> > Isn't that what the "Message-Id" field is for?
> > 
> > http://news.gmane.org/find-root.php?message_id=20070516.101643.94564776.t-ishii@sraoss.co.jp
> > a.
> 
> Maybe. However I think "subject-sequence" has some advantages over
> Message-Id:
> 
> - Easy to identify. Message-Id may not appear on some MUA with default
>   setting

Message-Ids are present in all messages.  When the MUA doesn't set it,
the MTA does.  The problem starts when the MUA doesn't set the
In-Reply-To header.

> - More handy than lengthy message Id

True.

> - Easy to detect messages not delivered, by knowing that the sequence
>   number was skipped

The problem is that the number would be possibly set at a later stage of
email delivery by the list software, so it doesn't help if the message
is skipped in an earlier stage (spam filter, etc).

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


Re: Not ready for 8.3

From
Aidan Van Dyk
Date:
* Tatsuo Ishii <ishii@postgresql.org> [070516 07:23]:
> Maybe. However I think "subject-sequence" has some advantages over
> Message-Id:
> 
> - Easy to identify. Message-Id may not appear on some MUA with default
>   setting
> 
> - More handy than lengthy message Id
> 
> - Easy to detect messages not delivered, by knowing that the sequence
>   number was skipped

IMNSHO, we should be encouraging lists to move *away* from subject
munging.  Adding more characters into a subject line which can already
be quite long is just making the situation worse.

And of course, once you realize that subject munging is wrong, putting
the "list-sequence" into a header of no real value, unless you believe
your MTA/MUA filtering to be somehow dropping list messages, and your
sequence numbers will prove to you if that's true or not.  Oh - but
wait, don't we all, as PostgreSQL users realize that sequences aren't
generally guaranteed to be gapless anyways (sure, there are work
"solutions", but I'm not about to audit any MTA/list software for
that...)

;-)

-- 
Aidan Van Dyk                                             Create like a god,
aidan@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.

Re: Not ready for 8.3

From
tomas@tuxteam.de
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, May 16, 2007 at 10:03:47AM -0400, Alvaro Herrera wrote:
> tomas@tuxteam.de wrote:

[...]
> There is just one remaining problem: Outlook and derivatives don't set
> the In-Reply-To: nor References: headers.  This breaks the threads (the
> best the software can do is group the messages by subject, but it works
> only partially).
> 
> I've tried a coupld of times to decode how the Thread-Id stuff works,
> which is what Outlook seems to use, but I haven't been able to detect a
> pattern.

Heh. Seems to be a well-known problem:
 <http://osdir.com/ml/discuss/2003-10/threads.html#00194>

Ackthpttp.

Regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFGSxVVBcgs9XrR2kYRAsRrAJ9GC7fHJgaS424yoylnzB/9YPtE/wCbBCRE
C62mzkj1BVXKlIau0TKlOS4=
=GVAM
-----END PGP SIGNATURE-----



Re: Not ready for 8.3

From
"Robert Haas"
Date:
> > Here here.
> >
> Where? Where? Oh, you mean "Hear Hear." (sorry - one of my pet peeves)

Oops.  Of course since it's in written form perhaps I should be writing
"Read! Read!" instead.

> We do have a responsibility, I think, to keep the grammar fairly
clean,
> so the answer to your question "who cares?" is "we do."

Sure.  I'm asking that rhetorical question under the assumption that
none of the options on the table are so bad as to constitute an
abrogation of that responsibility.  If that's not the case, then it's an
issue; otherwise, we shouldn't let differences of opinion over a
question that may not have only one right answer prevent us from doing
anything at all.

...Robert


Re: Not ready for 8.3

From
"Jim C. Nasby"
Date:
On Wed, May 16, 2007 at 08:58:44AM +0100, Dave Page wrote:
> Jim C. Nasby wrote:
> > On Tue, May 15, 2007 at 10:32:14PM +0200, Magnus Hagander wrote:
> >> Stefan Kaltenbrunner wrote:
> >>>> They are not stable.  The items should point to the archives, which are
> >>>> supposedly more stable.  (I had already fixed one item in PatchStatus
> >>>> this morning).  Really it would be much nicer to have links using the
> >>>> Message-Id but I doubt that's at all doable.
> >>> hrm - I see so is there a particular reason for that behaviour ?
> >> They're stable until Bruce removes something from the queue. When
> >> something is removed, it's renumbered.
> >>
> >> It's how mhonarc works. It's the same with the archives - if we delete a
> >> mail, they get renumbered. So we never should delete, we should just
> >> blank out, but it has happened a couple of times.
> > 
> > Isn't there any other archiver we could use? The lack of URL stability
> > in mhonarc is bad enough, but the cross-month issue is just horrible.
> 
> Nothing useful last time I looked (a year or two back admittedley). I
> have a design for one in mind that I was looking to prototype - there
> are some php classes that would make it quite simple to get messages
> into a database either via procmail, or from an mbox.
> 
> the stumbling block I was running into was rewriting the old archives
> URLs to the new ones.

How much visibility do we have into the mhonarc database? We should be
able to come up with a simple redirector that would point the old
mhonarc URLs to URLs for the new system...
-- 
Jim Nasby                                      decibel@decibel.org
EnterpriseDB      http://enterprisedb.com      512.569.9461 (cell)


Re: Not ready for 8.3

From
Dave Page
Date:
Jim C. Nasby wrote:

> How much visibility do we have into the mhonarc database? We should be
> able to come up with a simple redirector that would point the old
> mhonarc URLs to URLs for the new system...

<cough>database?</cough>

It's a file system. It simply generates HTML (or in our case) PHP files 
from each message in an mbox. That's one of the reasons for the monthly 
break - without it, the directories would be unusably full of files.

I the current URLs represent the month, and the ID of the message as it 
comes out of the mbox I believe. We could probably write a script to 
dump a list of message IDs, directories and mbox positions I imagine, 
and then import that into a new database.

It's been on my list to rewrite the whole archive system for a while for 
various reasons. There is quite a bit of crossover with the patch 
tracker I proposed so I was hoping to look at both together.

Regards, Dave


Re: Not ready for 8.3

From
Alvaro Herrera
Date:
Aidan Van Dyk wrote:
> * Tatsuo Ishii <ishii@postgresql.org> [070516 07:23]:
>  
> > Maybe. However I think "subject-sequence" has some advantages over
> > Message-Id:
> > 
> > - Easy to identify. Message-Id may not appear on some MUA with default
> >   setting
> > 
> > - More handy than lengthy message Id
> > 
> > - Easy to detect messages not delivered, by knowing that the sequence
> >   number was skipped
> 
> IMNSHO, we should be encouraging lists to move *away* from subject
> munging.  Adding more characters into a subject line which can already
> be quite long is just making the situation worse.

+1 on that.  It gets worse when messages are crossposted to multiple
lists and so multiple [FOO] strings are put into the subject.  On the
other hand I know there is people who remove the [FOO] strings in their
local machines!  (I don't do it just because I have been too lazy to
write a procmail rule about it).

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


Re: Not ready for 8.3

From
"Jim C. Nasby"
Date:
On Wed, May 16, 2007 at 04:34:56PM +0100, Dave Page wrote:
> >How much visibility do we have into the mhonarc database? We should be
> >able to come up with a simple redirector that would point the old
> >mhonarc URLs to URLs for the new system...
> 
> <cough>database?</cough>
And here I thought the reason we used tho POS was because it was the
only archiver that used PostgreSQL as it's backend...

> It's a file system. It simply generates HTML (or in our case) PHP files 
> from each message in an mbox. That's one of the reasons for the monthly 
> break - without it, the directories would be unusably full of files.
> 
> I the current URLs represent the month, and the ID of the message as it 
> comes out of the mbox I believe. We could probably write a script to 
> dump a list of message IDs, directories and mbox positions I imagine, 
> and then import that into a new database.
Yeah, if the files still resemble real emails then we can probably come
up with a way to pull the data in.

> It's been on my list to rewrite the whole archive system for a while for 
> various reasons. There is quite a bit of crossover with the patch 
> tracker I proposed so I was hoping to look at both together.

Let me know when you start on that...
-- 
Jim Nasby                                      decibel@decibel.org
EnterpriseDB      http://enterprisedb.com      512.569.9461 (cell)


Re: Not ready for 8.3

From
Chris Browne
Date:
aidan@highrise.ca (Aidan Van Dyk) writes:
> * Tatsuo Ishii <ishii@postgresql.org> [070516 07:23]:
>  
>> Maybe. However I think "subject-sequence" has some advantages over
>> Message-Id:
>> 
>> - Easy to identify. Message-Id may not appear on some MUA with default
>>   setting
>> 
>> - More handy than lengthy message Id
>> 
>> - Easy to detect messages not delivered, by knowing that the sequence
>>   number was skipped
>
> IMNSHO, we should be encouraging lists to move *away* from subject
> munging.  Adding more characters into a subject line which can already
> be quite long is just making the situation worse.
>
> And of course, once you realize that subject munging is wrong, putting
> the "list-sequence" into a header of no real value, unless you believe
> your MTA/MUA filtering to be somehow dropping list messages, and your
> sequence numbers will prove to you if that's true or not.  Oh - but
> wait, don't we all, as PostgreSQL users realize that sequences aren't
> generally guaranteed to be gapless anyways (sure, there are work
> "solutions", but I'm not about to audit any MTA/list software for
> that...)
>
> ;-)

The message ID *ought* to be usable for at least some of the desired
purposes; having a web client that uses message IDs as a query
mechanism, and where switching months doesn't ludicrously break
things, would seem to be one part of the solution.

Adding some sort of (I don't know...) database that associates bug
tracking system items with message IDs would seem like an "along-side"
way of linking in relevant information.

Adding x-???: tags might be a way of adding bug tracking information
into messages; it wouldn't help with original copies, but could be a
reasonable change to make in a given message repository.
-- 
output = ("cbbrowne" "@" "acm.org")
http://cbbrowne.com/info/rdbms.html
If nothing ever sticks to Teflon, how do they make Teflon stick to the
pan?


Re: Not ready for 8.3

From
"Joshua D. Drake"
Date:
Robert Haas wrote:
> 
>> hate the fact that FTS is a separate module.
> 
> Here here.
> 
> And with respect to the debate about syntax, who cares?  I think I
> prefer introducing real SQL-ish syntax over a bunch of pg_* functions,
> but doing it either way is IMHO better than doing nothing.
> 

I care. I want a professional easy to understand and easy to maintain 
that doesn't cause potential conflict with future and past development 
syntax.

Or should we be writing SQL in assembly?

Joshua D. Drake



> ...Robert
> 



Re: Not ready for 8.3

From
"Robert Haas"
Date:
> I care. I want a professional easy to understand and easy to maintain
> that doesn't cause potential conflict with future and past development

> syntax.

I agree with this.  The point of my comment was that ISTM that an
arbitrary amount of time can be consumed determining the optimal syntax,
during which Oleg is obligated to continually update his patch without
any guarantee that it will be accepted in anything like its current
form, or at all.  If people have strong opinions about the syntax, why
were they not ALL expressed when the proposal was originally laid on the
table?  Sure, some people offered opinions, but it doesn't appear that
there was any real consensus, and there certainly wasn't any clear
guidance of the form "this is what you need to do to get your patch
accepted", which leaves everything in limbo and Oleg doing a lot of
extra work to keep updating his patch.

I haven't studied the proposed syntaxes in detail, but ISTM that if
there is no consensus then probably all of the alternatives being
advocated are reasonable.  Again, if that's not the case, then let's
eliminate the unreasonable ones and pick one of the remaining choices.
But if committer A thinks that X is the only reasonable options and
committer B thinks that Y is the only reasonable option, does that mean
that the patch just sits there forever, or do we find a way to move
forward?

...Robert


Re: Not ready for 8.3

From
Dave Page
Date:
Jim C. Nasby wrote:
> On Wed, May 16, 2007 at 04:34:56PM +0100, Dave Page wrote:
>>> How much visibility do we have into the mhonarc database? We should be
>>> able to come up with a simple redirector that would point the old
>>> mhonarc URLs to URLs for the new system...
>> <cough>database?</cough>
>  
> And here I thought the reason we used tho POS was because it was the
> only archiver that used PostgreSQL as it's backend...

You're thinking of the old search engine.

>> It's a file system. It simply generates HTML (or in our case) PHP files 
>> from each message in an mbox. That's one of the reasons for the monthly 
>> break - without it, the directories would be unusably full of files.
>>
>> I the current URLs represent the month, and the ID of the message as it 
>> comes out of the mbox I believe. We could probably write a script to 
>> dump a list of message IDs, directories and mbox positions I imagine, 
>> and then import that into a new database.
>  
> Yeah, if the files still resemble real emails then we can probably come
> up with a way to pull the data in.

We have all the mbox files, so we can import them from there as raw 
messages.

>> It's been on my list to rewrite the whole archive system for a while for 
>> various reasons. There is quite a bit of crossover with the patch 
>> tracker I proposed so I was hoping to look at both together.
> 
> Let me know when you start on that...

Roger.

/D


Re: Not ready for 8.3

From
Magnus Hagander
Date:
Dave Page wrote:
>>> I the current URLs represent the month, and the ID of the message as
>>> it comes out of the mbox I believe. We could probably write a script
>>> to dump a list of message IDs, directories and mbox positions I
>>> imagine, and then import that into a new database.
>>  
>> Yeah, if the files still resemble real emails then we can probably come
>> up with a way to pull the data in.
> 
> We have all the mbox files, so we can import them from there as raw
> messages.

yeah, that's clearly the best source to work from. It's *possible* work
from the mhonarc files (I've done it before), but it's more work.


>>> It's been on my list to rewrite the whole archive system for a while
>>> for various reasons. There is quite a bit of crossover with the patch
>>> tracker I proposed so I was hoping to look at both together.
>>
>> Let me know when you start on that...
> 
> Roger.

Same here - I've done something similar (off mhonarc files and in much
smaller scale) before, and I'm definitely interested in helping out.

//Magnus


Re: Not ready for 8.3

From
Ron Mayer
Date:
Andrew Dunstan wrote:
> Josh Berkus wrote:
>> I think that may be where we're heading.  In that case, we may need to
>> talk about branching earlier so that developers can work on the new
>> version who are frozen out of the in-process one.
> 
> I've argued this in the past. But be aware that it will make more work
> for committers. It is very far from cost-free.

I hate to bring up the CMS flamewar again, but IMHO the one advantage
the distributed systems (Git, Monotone, Darcs, etc) have over CVS &
subversion is that they push this branch management cost down to the
developer who wants a branch - rather from the guys maintaining the
official CMS.


Re: Not ready for 8.3

From
"Joshua D. Drake"
Date:
Ron Mayer wrote:
> Andrew Dunstan wrote:
>> Josh Berkus wrote:
>>> I think that may be where we're heading.  In that case, we may need to
>>> talk about branching earlier so that developers can work on the new
>>> version who are frozen out of the in-process one.
>> I've argued this in the past. But be aware that it will make more work
>> for committers. It is very far from cost-free.
> 
> I hate to bring up the CMS flamewar again, but IMHO the one advantage
> the distributed systems (Git, Monotone, Darcs, etc) have over CVS &
> subversion is that they push this branch management cost down to the
> developer who wants a branch - rather from the guys maintaining the
> official CMS.

One step at a time :). We are now married, and have just begun to have 
have children. We need to buy a house in the next year, and figure out 
how to pay off some debt without going into hock up to our ears with a 
long term low interest loan.

Joshua D. Drake



> 
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
> 
>                http://archives.postgresql.org
> 



Re: Not ready for 8.3

From
"Jonah H. Harris"
Date:
On 5/16/07, Joshua D. Drake <jd@commandprompt.com> wrote:
> I care. I want a professional easy to understand and easy to maintain
> that doesn't cause potential conflict with future and past development
> syntax.

You've just tempted me to create embedded SQL in assembly :)

-- 
Jonah H. Harris, Software Architect | phone: 732.331.1324
EnterpriseDB Corporation            | fax: 732.331.1301
33 Wood Ave S, 3rd Floor            | jharris@enterprisedb.com
Iselin, New Jersey 08830            | http://www.enterprisedb.com/


Re: Not ready for 8.3

From
Richard Huxton
Date:
Magnus Hagander wrote:
>>>> It's been on my list to rewrite the whole archive system for a while
>>>> for various reasons. There is quite a bit of crossover with the patch
>>>> tracker I proposed so I was hoping to look at both together.
>>> Let me know when you start on that...
>> Roger.
> 
> Same here - I've done something similar (off mhonarc files and in much
> smaller scale) before, and I'm definitely interested in helping out.

Is everyone aware of this system that runs on a well-known open-source 
database?  http://www.archiveopteryx.org/
I've used it in a small way, and while I don't claim to have looked at 
it in detail it seems to pretty much do what it claims to.

--   Richard Huxton  Archonet Ltd


Re: Not ready for 8.3

From
Dave Page
Date:
Richard Huxton wrote:
> Magnus Hagander wrote:
>>>>> It's been on my list to rewrite the whole archive system for a while
>>>>> for various reasons. There is quite a bit of crossover with the patch
>>>>> tracker I proposed so I was hoping to look at both together.
>>>> Let me know when you start on that...
>>> Roger.
>>
>> Same here - I've done something similar (off mhonarc files and in much
>> smaller scale) before, and I'm definitely interested in helping out.
> 
> Is everyone aware of this system that runs on a well-known open-source
> database?
>   http://www.archiveopteryx.org/
> I've used it in a small way, and while I don't claim to have looked at
> it in detail it seems to pretty much do what it claims to.
> 

Yeah, I looked at it in the past. The database storage part is actually
pretty simple - it's the web front end that's going to take more effort,
and thats what that product doesn't do (or if it does, it's a secondary
function they don't shout about).

Regards, Dave.


Re: Not ready for 8.3

From
Richard Huxton
Date:
Dave Page wrote:
> Richard Huxton wrote:
>> Magnus Hagander wrote:
>>>>>> It's been on my list to rewrite the whole archive system for a while
>>>>>> for various reasons. There is quite a bit of crossover with the patch
>>>>>> tracker I proposed so I was hoping to look at both together.
>>>>> Let me know when you start on that...
>>>> Roger.
>>> Same here - I've done something similar (off mhonarc files and in much
>>> smaller scale) before, and I'm definitely interested in helping out.
>> Is everyone aware of this system that runs on a well-known open-source
>> database?
>>   http://www.archiveopteryx.org/
>> I've used it in a small way, and while I don't claim to have looked at
>> it in detail it seems to pretty much do what it claims to.
>>
> 
> Yeah, I looked at it in the past. The database storage part is actually
> pretty simple - it's the web front end that's going to take more effort,
> and thats what that product doesn't do (or if it does, it's a secondary
> function they don't shout about).

It's supposed to have something in the latest version, I think. I used 
it as backing store for a small workflow app, so I've got some simple 
views/functions I added and PHP code (cake-php framework) for displaying 
messages if it'll be of any use.

My one concern with the schema was that there didn't seem to be a way to 
partition archives (e.g. by year) to make maintenance a little simpler 
for large databases.

--   Richard Huxton  Archonet Ltd


Re: Not ready for 8.3

From
"Jim C. Nasby"
Date:
On Wed, May 16, 2007 at 07:48:10PM +0200, Magnus Hagander wrote:
> Dave Page wrote:
> >>> I the current URLs represent the month, and the ID of the message as
> >>> it comes out of the mbox I believe. We could probably write a script
> >>> to dump a list of message IDs, directories and mbox positions I
> >>> imagine, and then import that into a new database.
> >>  
> >> Yeah, if the files still resemble real emails then we can probably come
> >> up with a way to pull the data in.
> > 
> > We have all the mbox files, so we can import them from there as raw
> > messages.
> 
> yeah, that's clearly the best source to work from. It's *possible* work
> from the mhonarc files (I've done it before), but it's more work.

We'd want the old URLs to be redirected too, so at some point we'll have
to deal with mhonarc.
-- 
Jim Nasby                                      decibel@decibel.org
EnterpriseDB      http://enterprisedb.com      512.569.9461 (cell)


Re: Not ready for 8.3

From
"Jim C. Nasby"
Date:
On Wed, May 16, 2007 at 09:32:44PM +0100, Richard Huxton wrote:
> Dave Page wrote:
> >Richard Huxton wrote:
> >>Magnus Hagander wrote:
> >>>>>>It's been on my list to rewrite the whole archive system for a while
> >>>>>>for various reasons. There is quite a bit of crossover with the patch
> >>>>>>tracker I proposed so I was hoping to look at both together.
> >>>>>Let me know when you start on that...
> >>>>Roger.
> >>>Same here - I've done something similar (off mhonarc files and in much
> >>>smaller scale) before, and I'm definitely interested in helping out.
> >>Is everyone aware of this system that runs on a well-known open-source
> >>database?
> >>  http://www.archiveopteryx.org/
> >>I've used it in a small way, and while I don't claim to have looked at
> >>it in detail it seems to pretty much do what it claims to.
> >>
> >
> >Yeah, I looked at it in the past. The database storage part is actually
> >pretty simple - it's the web front end that's going to take more effort,
> >and thats what that product doesn't do (or if it does, it's a secondary
> >function they don't shout about).
> 
> It's supposed to have something in the latest version, I think. I used 
> it as backing store for a small workflow app, so I've got some simple 
> views/functions I added and PHP code (cake-php framework) for displaying 
> messages if it'll be of any use.
> 
> My one concern with the schema was that there didn't seem to be a way to 
> partition archives (e.g. by year) to make maintenance a little simpler 
> for large databases.

Luckily I happen to know of a database that would make that transparent
to the app...

But I tend to agree with Dave; the storage part is pretty easy. If we've
still got to write our own front-end ISTM it'd be better to just make it
exactly what we want...
-- 
Jim Nasby                                      decibel@decibel.org
EnterpriseDB      http://enterprisedb.com      512.569.9461 (cell)


Re: Not ready for 8.3

From
Greg Smith
Date:
On Tue, 15 May 2007, Jim C. Nasby wrote:

> Speaking of reviewers... should we put some thought into how we can
> increase the number of people who can review code? It seems that's one
> of our biggest bottlenecks...

Having recently dragged myself from never seeing the code before to being 
able to provide an early review to help the committers out, I can tell you 
a few things that would have sped up that process and therefore improve 
the odds more people will navigate the trail.

My main difficulty was figuring out a workable way to build a local mirror 
of the code (so I could get off-line diffs), keep it in sync with the 
development tree, while branching out working areas to evaluate patches or 
do new development in.  A good example walkthrough of how to do that using 
standard tools would be a big help.  Hint:  if you suggest CVsup I'll 
smack you.

Overall, though, I don't think there would have been any substitute for 
what I learned by putting together my own small patches, so in some 
respects thinking about how to lower the bar for doing that would also 
ultimately expand the review pool.  The main issues I had as a newcomer to 
the codebase was getting data in/out of the new code I added that was 
buried inside the system.  Here are some examples of what I kept 
hoping to find documentation on:

-Examples and usage guidelines for eLog and eReport (the stuff in the FAQ 
needs some more meat)

-How to add a GUC variable.  Once I've got that, now I can adjust the code 
in real-time just by updating it.

-How to add to the statistics for some part of the system that track a new 
variable and follow how that moves through the collector process.

If you put people into a position where they easily can poke data in at 
one end, see how it rattles around inside the engine by logging, and get 
some statistics on the result, now you've got someone who can build their 
own simple patches without being so frustrated.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD


Re: Not ready for 8.3

From
Magnus Hagander
Date:
Jim C. Nasby wrote:
> On Wed, May 16, 2007 at 07:48:10PM +0200, Magnus Hagander wrote:
>> Dave Page wrote:
>>>>> I the current URLs represent the month, and the ID of the message as
>>>>> it comes out of the mbox I believe. We could probably write a script
>>>>> to dump a list of message IDs, directories and mbox positions I
>>>>> imagine, and then import that into a new database.
>>>>  
>>>> Yeah, if the files still resemble real emails then we can probably come
>>>> up with a way to pull the data in.
>>> We have all the mbox files, so we can import them from there as raw
>>> messages.
>> yeah, that's clearly the best source to work from. It's *possible* work
>> from the mhonarc files (I've done it before), but it's more work.
> 
> We'd want the old URLs to be redirected too, so at some point we'll have
> to deal with mhonarc.

Right. Grabbing the msgid alone from them shouldn't be too hard though.
It's included in the "meta-headers" mhonarc sticks in each file, so it
should be a simple regex to find it.

//Magnus



Re: Not ready for 8.3

From
David Fetter
Date:
On Wed, May 16, 2007 at 11:50:26PM -0400, Greg Smith wrote:
> On Tue, 15 May 2007, Jim C. Nasby wrote:
> 
> >Speaking of reviewers... should we put some thought into how we can
> >increase the number of people who can review code? It seems that's one
> >of our biggest bottlenecks...
> 
> Having recently dragged myself from never seeing the code before to being 
> able to provide an early review to help the committers out, I can tell you 
> a few things that would have sped up that process and therefore improve 
> the odds more people will navigate the trail.
> 
> My main difficulty was figuring out a workable way to build a local mirror 
> of the code (so I could get off-line diffs), keep it in sync with the 
> development tree, while branching out working areas to evaluate patches or 
> do new development in.  A good example walkthrough of how to do that using 
> standard tools would be a big help.  Hint:  if you suggest CVsup I'll 
> smack you.
> 
> Overall, though, I don't think there would have been any substitute for 
> what I learned by putting together my own small patches, so in some 
> respects thinking about how to lower the bar for doing that would also 
> ultimately expand the review pool.  The main issues I had as a newcomer to 
> the codebase was getting data in/out of the new code I added that was 
> buried inside the system.  Here are some examples of what I kept 
> hoping to find documentation on:
> 
> -Examples and usage guidelines for eLog and eReport (the stuff in the FAQ 
> needs some more meat)
> 
> -How to add a GUC variable.  Once I've got that, now I can adjust the code 
> in real-time just by updating it.
> 
> -How to add to the statistics for some part of the system that track a new 
> variable and follow how that moves through the collector process.
> 
> If you put people into a position where they easily can poke data in at 
> one end, see how it rattles around inside the engine by logging, and get 
> some statistics on the result, now you've got someone who can build their 
> own simple patches without being so frustrated.

Would you be interested in providing this meat?  You're uniquely
qualified because your shins still smart from all the things you
barked them on :)

Cheers,
D
-- 
David Fetter <david@fetter.org> http://fetter.org/
phone: +1 415 235 3778        AIM: dfetter666                             Skype: davidfetter

Remember to vote!
Consider donating to PostgreSQL: http://www.postgresql.org/about/donate


Re: Not ready for 8.3

From
Dave Page
Date:
Magnus Hagander wrote:
> Right. Grabbing the msgid alone from them shouldn't be too hard though.
> It's included in the "meta-headers" mhonarc sticks in each file, so it
> should be a simple regex to find it.

Should be easier than that - when we import the existing messages from
the mbox files we should be able to figure out the current URL for each
message based on the year/month/list of the mbox file, and the message
number within the mbox.

/D


Re: Not ready for 8.3

From
Andrew Dunstan
Date:

David Fetter wrote:
>>
>> My main difficulty was figuring out a workable way to build a local mirror 
>> of the code (so I could get off-line diffs), keep it in sync with the 
>> development tree, while branching out working areas to evaluate patches or 
>> do new development in.  A good example walkthrough of how to do that using 
>> standard tools would be a big help.  Hint:  if you suggest CVsup I'll 
>> smack you.
>>
>>     
> Would you be interested in providing this meat?  You're uniquely
> qualified because your shins still smart from all the things you
> barked them on :)
>
>
>   

For this item at least the work has been done in showing how to set up a 
local mirror using rsync instead of CVSup. It's mentioned in the dev 
version of the docs at 
http://developer.postgresql.org/pgdocs/postgres/rsync.html - although 
frankly it would be better to import the information rather than just 
refer to the buildfarm HOWTO.

cheers

andredw




Re: Not ready for 8.3

From
Robert Treat
Date:
On Wednesday 16 May 2007 13:02, Robert Haas wrote:
> > I care. I want a professional easy to understand and easy to maintain
> > that doesn't cause potential conflict with future and past development
> >
> > syntax.
>
> <snip>
> If people have strong opinions about the syntax, why 
> were they not ALL expressed when the proposal was originally laid on the
> table?
<snip>
> I haven't studied the proposed syntaxes in detail, 
<snip>
>

lol... just thought this was ironic. :-)

-- 
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL


Re: Not ready for 8.3

From
Robert Treat
Date:
On Thursday 17 May 2007 08:16, Andrew Dunstan wrote:
> David Fetter wrote:
> >> My main difficulty was figuring out a workable way to build a local
> >> mirror of the code (so I could get off-line diffs), keep it in sync with
> >> the development tree, while branching out working areas to evaluate
> >> patches or do new development in.  A good example walkthrough of how to
> >> do that using standard tools would be a big help.  Hint:  if you suggest
> >> CVsup I'll smack you.
> >
> > Would you be interested in providing this meat?  You're uniquely
> > qualified because your shins still smart from all the things you
> > barked them on :)
>
> For this item at least the work has been done in showing how to set up a
> local mirror using rsync instead of CVSup. It's mentioned in the dev
> version of the docs at
> http://developer.postgresql.org/pgdocs/postgres/rsync.html - although
> frankly it would be better to import the information rather than just
> refer to the buildfarm HOWTO.
>

And really should probably be written up into FAQ section or a full on guide 
for "how do i get started hacking on postgresql?" 

-- 
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL


Re: Not ready for 8.3

From
Greg Smith
Date:
On Thu, 17 May 2007, Andrew Dunstan wrote:

> For this item at least the work has been done in showing how to set up a 
> local mirror using rsync instead of CVSup...although frankly it would be 
> better to import the information rather than just refer to the buildfarm 
> HOWTO.

The information in the buildfarm HOWTO has maybe 1/2 of what you need to 
know--you get a local mirror out of it but no idea how to then use that on 
the client side to branch and generate patches.  And what it does provide 
is less helpful than it might be because it includes diversions specific 
to the buildfarm application I found confusing.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD


Re: Not ready for 8.3

From
Greg Smith
Date:
On Thu, 17 May 2007, Robert Treat wrote:

> And really should probably be written up into FAQ section or a full on guide
> for "how do i get started hacking on postgresql?"

To be clear here:  while there are guides for new hackers out there, they 
focus on the big picture.  I walked from those with a good idea of how the 
major pieces fit together and was able to navigate the source code.  It 
was the little details of how to actually work with the code and the 
repository that were the biggest drag on my progress.

I used to wonder if it was just me, but watching the discussion that went 
along with the "OS/X startup scripts" thread recently made me realize how 
many other people struggle(d) with this as well.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD


Re: Not ready for 8.3

From
Andrew Dunstan
Date:

Greg Smith wrote:
> On Thu, 17 May 2007, Andrew Dunstan wrote:
>
>> For this item at least the work has been done in showing how to set 
>> up a local mirror using rsync instead of CVSup...although frankly it 
>> would be better to import the information rather than just refer to 
>> the buildfarm HOWTO.
>
> The information in the buildfarm HOWTO has maybe 1/2 of what you need 
> to know--you get a local mirror out of it but no idea how to then use 
> that on the client side to branch and generate patches.  And what it 
> does provide is less helpful than it might be because it includes 
> diversions specific to the buildfarm application I found confusing.
>
>

Sure. And of course the creation of branches and generation of patches 
is irrelevant to the buildfarm. All this just reinforces my point that 
we should remove that reference and fill in the blanks on the docs, FAQs 
etc.

Incidentally, I don't use a repo mirror for online or offline work, 
although I do for buildfarm work. What I do is to have a pristine 
checkout of each live stable branch plus HEAD. I have a cron job that 
keeps these moderately up to date. When I'm working on a feature or 
patch, I make a copy (using cp -a) of the relevant branch and then go to 
work on that. If the work lasts a while, every so often I run 'cvs 
update' in that copy. And of course I run 'cvs update' before cutting a 
patch or doing a commit. When the work is done I blow away the work 
directory. If the work is large and I need to checkpoint certain files 
locally so I can roll back, I occasionally use RCS.

There's no right answer on how to work - everyone uses tools they feel 
comfortable with.

cheers

andrew


Re: Not ready for 8.3

From
Andrew Sullivan
Date:
On Tue, May 15, 2007 at 04:52:16PM -0400, Alvaro Herrera wrote:

> This is what happens with the Linux kernel.  They have hundreds of
> developers getting their hands dirty during a previous period.  Then
> 2.6.20 is released; the 2.6.21 "merge window" opens, and all sort of
> patches are flooded in.  

I hasten to point out that the Linux kernel has also had several
"stable" releases with huge bugs -- things like massive filesystem
corruption, bizarre failure cases, and nasty compatibility problems
with modules across versions.  I am not entirely sure that the Linux
model is the one to ape.  PostgreSQL has a history with remarkably
few of those blunders, and I'd hate to give that up.

A

-- 
Andrew Sullivan  | ajs@crankycanuck.ca
If they don't do anything, we don't need their acronym.    --Josh Hamilton, on the US FEMA


Re: Not ready for 8.3

From
"Joshua D. Drake"
Date:
Andrew Sullivan wrote:
> On Tue, May 15, 2007 at 04:52:16PM -0400, Alvaro Herrera wrote:
> 
>> This is what happens with the Linux kernel.  They have hundreds of
>> developers getting their hands dirty during a previous period.  Then
>> 2.6.20 is released; the 2.6.21 "merge window" opens, and all sort of
>> patches are flooded in.  
> 
> I hasten to point out that the Linux kernel has also had several
> "stable" releases with huge bugs -- 

/me fondly remembers kernel 2.4.

things like massive filesystem
> corruption, bizarre failure cases, and nasty compatibility problems
> with modules across versions.  I am not entirely sure that the Linux
> model is the one to ape.  PostgreSQL has a history with remarkably
> few of those blunders, and I'd hate to give that up.
> 
> A
> 


-- 
      === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive  PostgreSQL solutions since 1997             http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/



Re: Not ready for 8.3

From
Andrew Dunstan
Date:

Joshua D. Drake wrote:
> Andrew Sullivan wrote:
>> On Tue, May 15, 2007 at 04:52:16PM -0400, Alvaro Herrera wrote:
>>
>>> This is what happens with the Linux kernel.  They have hundreds of
>>> developers getting their hands dirty during a previous period.  Then
>>> 2.6.20 is released; the 2.6.21 "merge window" opens, and all sort of
>>> patches are flooded in.  
>>
>> I hasten to point out that the Linux kernel has also had several
>> "stable" releases with huge bugs -- 
>
> /me fondly remembers kernel 2.4.
>
>

We keep focusing on process. I am on record as saying we can improve our 
processes, but the fact is our major immediate problem is person-power, 
not process. We need more qualified reviewers. Qualified means (to me, 
at least) you have to have done enough visible PostgreSQL hacking that a 
committer can reasonably place some level of trust in your review, 
thereby saving some time. That's not to say that others can't or 
shouldn't do reviews - every little bit helps, but if Freda Bloggs comes 
along with a review of some new, large, feature, she isn't helping to 
make the process shorter, although she might be helping to make it more 
robust.

cheers

andrew


Re: Not ready for 8.3

From
Alvaro Herrera
Date:
Andrew Sullivan wrote:
> On Tue, May 15, 2007 at 04:52:16PM -0400, Alvaro Herrera wrote:
> 
> > This is what happens with the Linux kernel.  They have hundreds of
> > developers getting their hands dirty during a previous period.  Then
> > 2.6.20 is released; the 2.6.21 "merge window" opens, and all sort of
> > patches are flooded in.  
> 
> I hasten to point out that the Linux kernel has also had several
> "stable" releases with huge bugs -- things like massive filesystem
> corruption, bizarre failure cases, and nasty compatibility problems
> with modules across versions.  I am not entirely sure that the Linux
> model is the one to ape.  PostgreSQL has a history with remarkably
> few of those blunders, and I'd hate to give that up.

Sorry, I wasn't trying to say that we should follow the Linux model all
that closely.  I know there are regressions in the "point zero"
releases, and that there are bugs.

But then, we're nowhere near the scale of Linux.  In the press release
for 8.0 we mentioned something about "hundreds of developers".  This was
true -- but it was not in the same league as Linux's "hundreds of
developers".  We're nowhere near the manpower that Linux has, nowhere
near the amount of code these guys change.  I think the figure is around
9000 lines of code changed per day, _every day_[1].  They have new
drivers all the time, internal interfaces are cleaned up, new facilities
are invented to support new kinds of hardware, performance improvements
are made all over the place.  So there are a hundred of strange machines
where the thing does not boot; yes, but those bugs are fixed in 2.6.2x.1
or subsequent "stable branch" releases.

I dare not think of the 2.2 or 2.4 disasters, where distributions were
fond of backporting huge patches from the 2.3 or 2.5 development
branches.  I compiled my own kernel from Linus' tree back then, which
worked without the strange behavior the other kernels had (maybe the
distro kernels stabilized at some point, but I didn't try later in the
game -- I was too used to compiling my own).  Now with 2.6 I don't do
that anymore (of course I don't run 2.6.22 as soon as it is out either).

In my opinion Linux 2.6 is much better than Linux 2.4.

I don't think we should follow either model though.  We have different
problems and different people.

[1] http://lwn.net/Articles/232379/

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


Re: Not ready for 8.3

From
Greg Smith
Date:
On Thu, 17 May 2007, David Fetter wrote:

> Would you be interested in providing this meat?  You're uniquely
> qualified because your shins still smart from all the things you
> barked them on :)

Unfortunately I'm temporarily on the other side of this problem; all the 
time I have to spare right now is going to into reviewing other people's 
patches.

Even more unfortunately, I'm not 100% happy with what I'm doing, and 
certainly wouldn't want to present the shoddy techniques I'm using as 
recommended.  If anyone else has useful info on this subject (keeping a 
local repository in sync with HEAD via rsync while working on branches) 
they'd like to pass along, please drop me a message with whatever level of 
documentation you might have--even rough notes would be a help.  Once I'm 
finished with the reviews I'm trying to contribute, I will sort through 
any suggestions I get and turn that into formal documentation.  This has 
left enough shin damage that I'd enjoy completely slaying the cause.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD


Working with PostgreSQL source tree (was Re: Not ready for 8.3)

From
Heikki Linnakangas
Date:
Greg Smith wrote:
> On Thu, 17 May 2007, David Fetter wrote:
> 
>> Would you be interested in providing this meat?  You're uniquely
>> qualified because your shins still smart from all the things you
>> barked them on :)
> 
> Unfortunately I'm temporarily on the other side of this problem; all the 
> time I have to spare right now is going to into reviewing other people's 
> patches.
> 
> Even more unfortunately, I'm not 100% happy with what I'm doing, and 
> certainly wouldn't want to present the shoddy techniques I'm using as 
> recommended.  If anyone else has useful info on this subject (keeping a 
> local repository in sync with HEAD via rsync while working on branches) 
> they'd like to pass along, please drop me a message with whatever level 
> of documentation you might have--even rough notes would be a help.  Once 
> I'm finished with the reviews I'm trying to contribute, I will sort 
> through any suggestions I get and turn that into formal documentation.  
> This has left enough shin damage that I'd enjoy completely slaying the 
> cause.

Let me describe my method. It's not ideal by any means, but I've been 
satisfied:

First of all, having a local CVS repository is a must. I used to use 
cvsup until a couple of months ago, but it suddenly stopped working, so 
I switched to rsync and I haven't looked back. I have a one line shell 
script to update it:

rsync --progress -avzCH --delete anoncvs.postgresql.org::pgsql-cvs 
/home/hlinnaka/pgcvsrepository

When I start working on a subject, I make a new checkout from the local 
repository to a designated directory under "pg_sandbox"-directory. For 
example, when I started working on the mvcc-safe cluster patch:

~/pg_sandbox$ cvs -d /home/hlinnaka/pgcvsrepository cvs co pgsql
~/pg_sandbox$ mv pgsql pgsql.cluster
~/pg_sandbox$ cd pgsql.cluster
~/pg_sandbox/pgsql.cluster$ cvs update -dP   # Remove empty dirs. 
There's a checkout-flag for that as well, but I never remember it...
~/pg_sandbox/pgsql.cluster$ ./configure --enable-depend --enable-cassert 
--enable-debug --prefix=/home/hlinnaka/pgsql.cluster

For primitive version control, I make a diff after any significant changes:

~/pg_sandbox/pgsql.cluster$ cvs diff -cN > cluster-mvcc-1.patch

The last number is a revision number, increment by one after each diff.

That's it!

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com


Re: Working with PostgreSQL source tree (was Re: Not ready for 8.3)

From
"Pavan Deolasee"
Date:

On 5/18/07, Heikki Linnakangas <heikki@enterprisedb.com> wrote:


When I start working on a subject, I make a new checkout from the local
repository to a designated directory under "pg_sandbox"-directory. For
example, when I started working on the mvcc-safe cluster patch:

~/pg_sandbox$ cvs -d /home/hlinnaka/pgcvsrepository cvs co pgsql
~/pg_sandbox$ mv pgsql pgsql.cluster


You can use cvs co -d pgsql.cluster to save the above step.
 

~/pg_sandbox$ cd pgsql.cluster
~/pg_sandbox/pgsql.cluster$ cvs update -dP   # Remove empty dirs.
There's a checkout-flag for that as well, but I never remember it...
~/pg_sandbox/pgsql.cluster$ ./configure --enable-depend --enable-cassert
--enable-debug --prefix=/home/hlinnaka/pgsql.cluster

I find configuring with CFLAGS="-g -O0" more useful.


For primitive version control, I make a diff after any significant changes:

~/pg_sandbox/pgsql.cluster$ cvs diff -cN > cluster-mvcc-1.patch


I usually commit each version and tag the tree. That helps me to get
diff between two versions as well.
 

Thanks,
Pavan


--
Pavan Deolasee
EnterpriseDB     http://www.enterprisedb.com

Re: Working with PostgreSQL source tree (was Re: Not ready for 8.3)

From
Heikki Linnakangas
Date:
Pavan Deolasee wrote:
> For primitive version control, I make a diff after any significant changes:
>>
>> ~/pg_sandbox/pgsql.cluster$ cvs diff -cN > cluster-mvcc-1.patch
> 
> I usually commit each version and tag the tree. That helps me to get
> diff between two versions as well.

Doesn't that confuse rsync?

I use "diff patch-1 patch-2" for diffs between versions, but it's really 
hard to read.

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com


Re: Working with PostgreSQL source tree (was Re: Not ready for 8.3)

From
"Pavan Deolasee"
Date:

On 5/18/07, Heikki Linnakangas <heikki@enterprisedb.com> wrote:
Pavan Deolasee wrote:
> For primitive version control, I make a diff after any significant changes:
>>
>> ~/pg_sandbox/pgsql.cluster$ cvs diff -cN > cluster-mvcc-1.patch
>
> I usually commit each version and tag the tree. That helps me to get
> diff between two versions as well.

Doesn't that confuse rsync?

I use cvsup. But thats not relevant here. Yes, cvsup would delete any
local tags/branches and sync the tree. But then I don't sync the tree very
often. And when I need to sync my work with CVS HEAD, I create
all the required diffs, sync the tree and reapply the patches (and commit
and tag) the tree again. Thats some work, but is I am OK with that
because as I said we don't need to sync with head that often.
 
Thanks

--
Pavan Deolasee
EnterpriseDB     http://www.enterprisedb.com

Re: Not ready for 8.3

From
"Jim C. Nasby"
Date:
On Thu, May 17, 2007 at 11:02:31PM -0400, Greg Smith wrote:
> On Thu, 17 May 2007, David Fetter wrote:
> 
> >Would you be interested in providing this meat?  You're uniquely
> >qualified because your shins still smart from all the things you
> >barked them on :)
> 
> Unfortunately I'm temporarily on the other side of this problem; all the 
> time I have to spare right now is going to into reviewing other people's 
> patches.
> 
> Even more unfortunately, I'm not 100% happy with what I'm doing, and 
> certainly wouldn't want to present the shoddy techniques I'm using as 
> recommended.  If anyone else has useful info on this subject (keeping a 
> local repository in sync with HEAD via rsync while working on branches) 
> they'd like to pass along, please drop me a message with whatever level of 
> documentation you might have--even rough notes would be a help.  Once I'm 
> finished with the reviews I'm trying to contribute, I will sort through 
> any suggestions I get and turn that into formal documentation.  This has 
> left enough shin damage that I'd enjoy completely slaying the cause.

Rather than trying to get something "FAQ-perfect" right now, can you at
least brain-dump what your current process is so that others can learn?
I've created a wiki page for this and added Heikki's tips:
http://developer.postgresql.org/index.php/Working_with_CVS
-- 
Jim Nasby                                      decibel@decibel.org
EnterpriseDB      http://enterprisedb.com      512.569.9461 (cell)


Re: Not ready for 8.3

From
Greg Smith
Date:
On Fri, 18 May 2007, Jim C. Nasby wrote:

> Rather than trying to get something "FAQ-perfect" right now, can you at
> least brain-dump what your current process is so that others can learn?
> I've created a wiki page for this and added Heikki's tips:
> http://developer.postgresql.org/index.php/Working_with_CVS

Heikki's great summary helps (I think the one piece I was screwing up is 
covered there), and Pavan's comments adds some useful bits.  The still 
missing part is how to make a real branch to work in, which is much easier 
to work with once you figure out how to do it than either using just diffs 
or tagging.

I agree the wiki is the right place to hash this out; when that's 
converged to something useful, then take a snapshot and convert for the 
official manual.  I'll work on it.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD


Re: Not ready for 8.3

From
Andrew Dunstan
Date:

Greg Smith wrote:
>
> Heikki's great summary helps (I think the one piece I was screwing up 
> is covered there), and Pavan's comments adds some useful bits.  The 
> still missing part is how to make a real branch to work in, which is 
> much easier to work with once you figure out how to do it than either 
> using just diffs or tagging.
>
>

What would making a branch actually do for you? The only advantage I can 
see is that it will give you a way of checkpointing your files. As I 
remarked upthread, I occasionally use RCS for that. But mostly I don't 
actually bother. I don't see how you can do it reasonably off a local 
cvs mirror - rsync will just blow away any changes you have checked in 
next time you sync with the master.

I don't think we can make CVS behave like a distributed SCM system, and 
ability to create local branches seems to me one of the fundamental 
points of such systems. If that's what the demand is for, then we should 
look again at moving to something like Mercurial.

cheers

andrew


Re: Not ready for 8.3

From
"Florian G. Pflug"
Date:
Andrew Dunstan wrote:
> What would making a branch actually do for you? The only advantage I can 
> see is that it will give you a way of checkpointing your files. As I 
> remarked upthread, I occasionally use RCS for that. But mostly I don't 
> actually bother. I don't see how you can do it reasonably off a local 
> cvs mirror - rsync will just blow away any changes you have checked in 
> next time you sync with the master.
> 
> I don't think we can make CVS behave like a distributed SCM system, and 
> ability to create local branches seems to me one of the fundamental 
> points of such systems. If that's what the demand is for, then we should 
> look again at moving to something like Mercurial.

I think the great thing about DCVS systems is that not everybody
necessarily needs to use the *same* system. And it doesn't really
matter what the central repository runs on - I think they are
gateway from/to nearly everything available...

I currently use GIT for my SoC project, and it works quite well -
I can create an abitrary number of local branches, and syncing
the currently active branch with CVS is archived by just doing
"cg-update pgsql-head".

greetings, Florian Pflug


Re: Not ready for 8.3

From
Greg Smith
Date:
On Sat, 19 May 2007, Andrew Dunstan wrote:

> What would making a branch actually do for you? The only advantage I can see 
> is that it will give you a way of checkpointing your files.

Exactly.  It's not as bad now, but when I was getting started there were 
lots of times I got something working and I wanted a clean way to save it 
in that state before I started messing with anything else--such that I 
could backtrack what I did if I later broke it.  This style of work has 
some advantages for working on other people's patches as well, especially 
if you're trying to review two at once and it takes you a while to 
finish--the situation I always find myself in.

> I don't see how you can do it reasonably off a local cvs mirror - rsync 
> will just blow away any changes you have checked in next time you sync 
> with the master.

It's certainly not easy.  I try not to let the fact that what I'd like to 
do may actually be impossible ever discourage me.

There are at least three ways to approach this general problem just 
counting the rsync/CVS variations, each with some obvious and some subtle 
advantages/disadvantages.  I'm still in the stage where I'm mapping out 
the options.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD