Thread: RC1 blocker issues

RC1 blocker issues

From
Tom Lane
Date:
Here's what I see as the remaining things we need to resolve before
wrapping 8.2RC1:

* Brendan Jurd's recent report about from_char/to_char not being
consistent about 'D':
http://archives.postgresql.org/pgsql-hackers/2006-11/msg00226.php
Is this a bug?  If so I think we should fix it now.

* the to_char month/day name localization issue.  Consensus at the
moment seems to be that we should revert the last-minute patch.

* possible rearrangement of pg_stat column order:
http://archives.postgresql.org/pgsql-hackers/2006-11/msg00643.php
Should we do this, and if so should we force initdb (via a catversion
change)?  I'm currently leaning to the thought that if we change it
we should force initdb, else we'll risk having a noticeable user-visible
difference between different "8.2" installations.  Personally I'd vote
for the change, but I don't have any big beta databases to reload ;-)

BTW, if we do change system_views.sql I'm inclined to remove the hack
for pg_stat_file() at the bottom, and instead fix pg_proc.h --- initdb
can handle this properly now.
        regards, tom lane


Re: RC1 blocker issues

From
Tom Lane
Date:
I wrote:
> * possible rearrangement of pg_stat column order:
> http://archives.postgresql.org/pgsql-hackers/2006-11/msg00643.php
> Should we do this, and if so should we force initdb (via a catversion
> change)?  I'm currently leaning to the thought that if we change it
> we should force initdb, else we'll risk having a noticeable user-visible
> difference between different "8.2" installations.

Actually, on looking closer, we *must* force initdb because this changes
the expected output for the rules regression test.

So, yea or nay?  I'm working up the patch right now, but will hold off
applying until I hear some comments.
        regards, tom lane


Re: [CORE] RC1 blocker issues

From
Peter Eisentraut
Date:
Tom Lane wrote:
> * Brendan Jurd's recent report about from_char/to_char not being
> consistent about 'D':
> http://archives.postgresql.org/pgsql-hackers/2006-11/msg00226.php
> Is this a bug?  If so I think we should fix it now.

Should be fixed, unless someone with Oracle can show that it is 
intentional.

> * the to_char month/day name localization issue.  Consensus at the
> moment seems to be that we should revert the last-minute patch.

Leave for 8.3.

> * possible rearrangement of pg_stat column order:
> http://archives.postgresql.org/pgsql-hackers/2006-11/msg00643.php
> Should we do this, and if so should we force initdb (via a catversion
> change)?  I'm currently leaning to the thought that if we change it
> we should force initdb, else we'll risk having a noticeable
> user-visible difference between different "8.2" installations. 

Fix and initdb.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/


Re: RC1 blocker issues

From
David Fetter
Date:
On Fri, Nov 24, 2006 at 02:48:22PM -0500, Tom Lane wrote:
> I wrote:
> > * possible rearrangement of pg_stat column order:
> > http://archives.postgresql.org/pgsql-hackers/2006-11/msg00643.php
> > Should we do this, and if so should we force initdb (via a
> > catversion change)?  I'm currently leaning to the thought that if
> > we change it we should force initdb, else we'll risk having a
> > noticeable user-visible difference between different "8.2"
> > installations.
> 
> Actually, on looking closer, we *must* force initdb because this
> changes the expected output for the rules regression test.
> 
> So, yea or nay?  I'm working up the patch right now, but will hold
> off applying until I hear some comments.

+1 on applying it.  Beta testers should understand what "beta" and
"test" mean. :)

Cheers,
D
-- 
David Fetter <david@fetter.org> http://fetter.org/
phone: +1 415 235 3778        AIM: dfetter666                             Skype: davidfetter

Remember to vote!


Re: [CORE] RC1 blocker issues

From
Tom Lane
Date:
Peter Eisentraut <peter_e@gmx.net> writes:
> Tom Lane wrote:
>> * the to_char month/day name localization issue.  Consensus at the
>> moment seems to be that we should revert the last-minute patch.

> Leave for 8.3.

Uh, you mean leave the patch in?  I thought you wanted it out.
        regards, tom lane


Re: [CORE] RC1 blocker issues

From
Bruce Momjian
Date:
Tom Lane wrote:
> Peter Eisentraut <peter_e@gmx.net> writes:
> > Tom Lane wrote:
> >> * the to_char month/day name localization issue.  Consensus at the
> >> moment seems to be that we should revert the last-minute patch.
> 
> > Leave for 8.3.
> 
> Uh, you mean leave the patch in?  I thought you wanted it out.

I assume he means leave the change for 8.3, not now.

--  Bruce Momjian   bruce@momjian.us EnterpriseDB    http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: [CORE] RC1 blocker issues

From
Peter Eisentraut
Date:
Tom Lane wrote:
> Peter Eisentraut <peter_e@gmx.net> writes:
> > Tom Lane wrote:
> >> * the to_char month/day name localization issue.  Consensus at the
> >> moment seems to be that we should revert the last-minute patch.
> >
> > Leave for 8.3.
>
> Uh, you mean leave the patch in?  I thought you wanted it out.

Sorry, take the patch out but leave the issue pending for 8.3.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/


Re: RC1 blocker issues

From
"Simon Riggs"
Date:
On Fri, 2006-11-24 at 12:37 -0800, David Fetter wrote:
> On Fri, Nov 24, 2006 at 02:48:22PM -0500, Tom Lane wrote:
> > I wrote:
> > > * possible rearrangement of pg_stat column order:
> > > http://archives.postgresql.org/pgsql-hackers/2006-11/msg00643.php
> > > Should we do this, and if so should we force initdb (via a
> > > catversion change)?  I'm currently leaning to the thought that if
> > > we change it we should force initdb, else we'll risk having a
> > > noticeable user-visible difference between different "8.2"
> > > installations.
> > 
> > Actually, on looking closer, we *must* force initdb because this
> > changes the expected output for the rules regression test.
> > 
> > So, yea or nay?  I'm working up the patch right now, but will hold
> > off applying until I hear some comments.
> 
> +1 on applying it.  Beta testers should understand what "beta" and
> "test" mean. :)

Yes, I would like to rearrange the columns.

--  Simon Riggs              EnterpriseDB   http://www.enterprisedb.com




Re: RC1 blocker issues

From
"Andrew Dunstan"
Date:
Tom Lane wrote:
> I wrote:
>> * possible rearrangement of pg_stat column order:
>> http://archives.postgresql.org/pgsql-hackers/2006-11/msg00643.php
>> Should we do this, and if so should we force initdb (via a catversion
>> change)?  I'm currently leaning to the thought that if we change it
>> we should force initdb, else we'll risk having a noticeable user-visible
>> difference between different "8.2" installations.
>
> Actually, on looking closer, we *must* force initdb because this changes
> the expected output for the rules regression test.
>
> So, yea or nay?  I'm working up the patch right now, but will hold off
> applying until I hear some comments.
>

Fixing it later would be nastier, or impossible. I think we should fix it
now. We don't have an absolute promise not to require an initdb during
beta, do we?

cheers

andrew



Re: [CORE] RC1 blocker issues

From
Bruce Momjian
Date:
Peter Eisentraut wrote:
> Tom Lane wrote:
> > Peter Eisentraut <peter_e@gmx.net> writes:
> > > Tom Lane wrote:
> > >> * the to_char month/day name localization issue.  Consensus at the
> > >> moment seems to be that we should revert the last-minute patch.
> > >
> > > Leave for 8.3.
> >
> > Uh, you mean leave the patch in?  I thought you wanted it out.
> 
> Sorry, take the patch out but leave the issue pending for 8.3.

OK, patch reverted.

--  Bruce Momjian   bruce@momjian.us EnterpriseDB    http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: RC1 blocker issues

From
Stefan Kaltenbrunner
Date:
Tom Lane wrote:
> I wrote:
>> * possible rearrangement of pg_stat column order:
>> http://archives.postgresql.org/pgsql-hackers/2006-11/msg00643.php
>> Should we do this, and if so should we force initdb (via a catversion
>> change)?  I'm currently leaning to the thought that if we change it
>> we should force initdb, else we'll risk having a noticeable user-visible
>> difference between different "8.2" installations.
> 
> Actually, on looking closer, we *must* force initdb because this changes
> the expected output for the rules regression test.
> 
> So, yea or nay?  I'm working up the patch right now, but will hold off
> applying until I hear some comments.

I would like to see it changed - even late into the beta cycle ...



Stefan


Re: RC1 blocker issues

From
"Joshua D. Drake"
Date:
> BTW, if we do change system_views.sql I'm inclined to remove the hack
> for pg_stat_file() at the bottom, and instead fix pg_proc.h --- initdb
> can handle this properly now.

There is no reason to rush the release of 8.2, if there are last minute 
-- known items. We should fix them. Being a month late, but having a 
better product, is always a better solution.

Sincerely,


Joshua D. Drake



> 
>             regards, tom lane
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 7: You can help support the PostgreSQL project by donating at
> 
>                 http://www.postgresql.org/about/donate
> 



Re: [CORE] RC1 blocker issues

From
"Marc G. Fournier"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



- --On Saturday, November 25, 2006 11:00:04 -0800 "Joshua D. Drake" 
<jd@commandprompt.com> wrote:

>
>> BTW, if we do change system_views.sql I'm inclined to remove the hack
>> for pg_stat_file() at the bottom, and instead fix pg_proc.h --- initdb
>> can handle this properly now.
>
> There is no reason to rush the release of 8.2, if there are last minute --
> known items. We should fix them. Being a month late, but having a better
> product, is always a better solution.

As rare as he and I agree ... I agree ... LISA is nice and all, but end product 
*is* more important then timing, unless you are Microsoft, of course :)

- ----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email . scrappy@hub.org                              MSN . scrappy@hub.org
Yahoo . yscrappy               Skype: hub.org        ICQ . 7615664
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (FreeBSD)

iD8DBQFFaJlE4QvfyHIvDvMRAhatAKCeumY3DPTmULzSplGWlXR+00qEBwCfUFUz
rI3THgwtRBzvh/JEBBGlrtk=
=/m9d
-----END PGP SIGNATURE-----



Re: [CORE] RC1 blocker issues

From
Dave Page
Date:
Marc G. Fournier wrote:

> - --On Saturday, November 25, 2006 11:00:04 -0800 "Joshua D. Drake" 
> <jd@commandprompt.com> wrote:
> 
> As rare as he and I agree ... I agree 

Wow - you wanna go lie down for a while?

:-p

Regards, Dave.


Re: [CORE] RC1 blocker issues

From
"Joshua D. Drake"
Date:
Dave Page wrote:
> Marc G. Fournier wrote:
> 
>> - --On Saturday, November 25, 2006 11:00:04 -0800 "Joshua D. Drake" 
>> <jd@commandprompt.com> wrote:
>>
>> As rare as he and I agree ... I agree 
> 
> Wow - you wanna go lie down for a while?

Oh come on... it isn't *that* rare ;)

Joshua D. Drake

> 
> :-p
> 
> Regards, Dave.
> 



Re: [CORE] RC1 blocker issues

From
Tom Lane
Date:
"Marc G. Fournier" <scrappy@hub.org> writes:
> As rare as he and I agree ... I agree ... LISA is nice and all, but
> end product *is* more important then timing, unless you are Microsoft,
> of course :)

I'm not concerned about announcing at LISA (though I know Josh is) ...
but at the same time I want to get the thing out the door.  I'm not sure
that anything will be gained by further delay.

This has been sort of a weird beta period, because it's the first one
we've had where cleaning up portability problems has been a complete
non-issue.  The buildfarm has changed the rules of the game that way.
Previously we've always had to allocate a fair amount of time for
getting and dealing with port reports, but I don't think we need to
do that anymore.  So really the release decision is down to "at what
point do we think it's stable enough to call it a release?".

I think we're at that point already: I just looked through the CVS logs,
and by my count there were 40 non-cosmetic patches applied to the main
source tree (not doc/ or contrib/) during the past month.  Of these
all but three either were or could have been applied to 8.1 as well,
ie, they fixed problems that exist in 8.1 or before.  The three actual
new-in-8.2 bugs found in the last thirty days are:

2006-11-10 13:10  tgl
* backend/catalog/information_schema.sql: Fix errors inkey_column_usage.position_in_unique_constraint column
recentlyaddedto information_schema (per a SQL2003 addition).  The originalcoding failed if a referenced column
participatedin more than onepg_constraint entry.  Also, it did not work if an FK relieddirectly on a unique index
withoutany constraint syntactic sugar--- this case is outside the SQL spec, but PG has always supportedit, so it's
reasonablefor our information_schema to handle it too.Per bug#2750 from Stephen Haberman.
 

2006-11-10 17:59  tgl
* backend/utils/adt/ruleutils.c: Fix pg_get_serial_sequence(),which could incorrectly return the name of an index on a
serialcolumn,rather than the name of the associated sequence.  Falloutfrom recent changes in dependency setup for
serials. Per bug #2732from Basil Evseenko.
 

2006-11-24 16:18  tgl
* backend/catalog/system_views.sql, backend/utils/adt/genfile.c,include/catalog/catversion.h,
include/catalog/pg_proc.h,test/regress/expected/rules.out:Change pg_stat_all_tables andsister views to put the
recently-addedvacuum/analyze timestampcolumns at the end, rather than at a random spot in the middle asin the original
patch.

That last one barely even qualifies as a bug; it's a cosmetic
improvement.  So it's now a solid two weeks since we found a real new bug.

So if you ask me, we are past the point of diminishing returns for the
current level of testing.  Putting out an RC might draw the attention of
a few more people, but really the only way we are going to find more
bugs is to put out a release.  Or perhaps knock heads together a little
harder on pgsql-hackers to make people think less about 8.3 development
and more about 8.2 testing, but how successful is that likely to be?
        regards, tom lane


Re: [CORE] RC1 blocker issues

From
"Marc G. Fournier"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



- --On Saturday, November 25, 2006 15:58:41 -0500 Tom Lane <tgl@sss.pgh.pa.us> 
wrote:

> So if you ask me, we are past the point of diminishing returns for the
> current level of testing.  Putting out an RC might draw the attention of
> a few more people, but really the only way we are going to find more
> bugs is to put out a release.  Or perhaps knock heads together a little
> harder on pgsql-hackers to make people think less about 8.3 development
> and more about 8.2 testing, but how successful is that likely to be?

Based on experience ... not very.  And I'm not voicing against LISA, I'm just 
agreeing with Joshua that it shouldn't the "requirement" ... if we are ready, 
no arguments from this end ...


Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email . scrappy@hub.org                              MSN . scrappy@hub.org
Yahoo . yscrappy               Skype: hub.org        ICQ . 7615664
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (FreeBSD)

iD8DBQFFaLC44QvfyHIvDvMRAnO3AKCJy+ghmE4dLk1/jQ0jWEw+ySbWJgCfeFlB
928t3jXtaKIgwE+GSSigU1g=
=mVks
-----END PGP SIGNATURE-----



Re: [CORE] RC1 blocker issues

From
Andrew Sullivan
Date:
On Sat, Nov 25, 2006 at 03:58:41PM -0500, Tom Lane wrote:
> source tree (not doc/ or contrib/) during the past month.  Of these
> all but three either were or could have been applied to 8.1 as well,
> ie, they fixed problems that exist in 8.1 or before.  The three actual

It strikes me that this sort of analysis might be a useful future
rule of thumb test for releases, given the buildfarm, no?  At what
point does the reported-bug testing stop, by some overwhelming
margin, revealing new bugs?  That's the proof that a release
candidate is reached.

Andrew "process high horse is about to throw me" Sullivan

-- 
Andrew Sullivan  | ajs@crankycanuck.ca
A certain description of men are for getting out of debt, yet are
against all taxes for raising money to pay it off.    --Alexander Hamilton


Re: [CORE] RC1 blocker issues

From
Andrew Dunstan
Date:

Tom Lane wrote:
> This has been sort of a weird beta period, because it's the first one
> we've had where cleaning up portability problems has been a complete
> non-issue.  The buildfarm has changed the rules of the game that way.
> Previously we've always had to allocate a fair amount of time for
> getting and dealing with port reports, but I don't think we need to
> do that anymore.  
>   

Isn't it nice when one's children outperform the intentions, 
expectations and hopes one held for them? ;-)

cheers

andrew




Re: [CORE] RC1 blocker issues

From
Josh Berkus
Date:
Marc, Josh:

> > There is no reason to rush the release of 8.2, if there are last minute
> > -- known items. We should fix them. Being a month late, but having a
> > better product, is always a better solution.
>
> As rare as he and I agree ... I agree ... LISA is nice and all, but end
> product *is* more important then timing, unless you are Microsoft, of
> course :)

Just so you all know: I gave a green flag for two magazines (one in Middle 
East, one in Dutch) to carry articles about 8.2 for their January issues, 
which will come out in December.   So there's a distinct possibility that 
those magazines will come out and 8.2 still won't be available.  I hadn't 
anticipated us slipping the schedule by more than a month.  Not that this 
should be our deciding criteria, but just so people know that I'm not just 
pushing the date for no reason.

Also note that if we release between December 18th and January 3rd, we can 
pretty much expect no coverage from the press.

Overall, I submit that our release process is broken, and that we're having 
trouble getting this release out because nobody is paying attention to it (of 
which I too have been guilty off-and-on).  I suggest that we need something 
different for 8.3, or we can forget about even trying to do a short release 
schedule. 

-- 
Josh Berkus
PostgreSQL @ Sun
San Francisco


Re: [CORE] RC1 blocker issues

From
Peter Eisentraut
Date:
Josh Berkus wrote:
> Just so you all know: I gave a green flag for two magazines (one in
> Middle East, one in Dutch) to carry articles about 8.2 for their
> January issues, which will come out in December.   So there's a
> distinct possibility that those magazines will come out and 8.2 still
> won't be available.  I hadn't anticipated us slipping the schedule by
> more than a month.  Not that this should be our deciding criteria,
> but just so people know that I'm not just pushing the date for no
> reason.

We have this issue every year and it's never ended up being a problem.  
It's the author's or publisher's fault if they can't write their 
articles ambiguous enough.

> Also note that if we release between December 18th and January 3rd,
> we can pretty much expect no coverage from the press.

We also have this issue every year and I still don't believe it.  
Curiously, there is always excellent press coverage of everything else 
between December 18th and January 3rd.

> Overall, I submit that our release process is broken, and that we're
> having trouble getting this release out because nobody is paying
> attention to it (of which I too have been guilty off-and-on).  I
> suggest that we need something different for 8.3, or we can forget
> about even trying to do a short release schedule.

We also have this issue every year, but the time from beta to release 
has always been about three months.  With the buildfarm helping out, 
we're a couple of weeks early this time.  Rejoice.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/


Re: [CORE] RC1 blocker issues

From
Tom Lane
Date:
Peter Eisentraut <peter_e@gmx.net> writes:
> Josh Berkus wrote:
>> Overall, I submit that our release process is broken, and that we're
>> having trouble getting this release out because nobody is paying
>> attention to it (of which I too have been guilty off-and-on).

> We also have this issue every year, but the time from beta to release 
> has always been about three months.  With the buildfarm helping out, 
> we're a couple of weeks early this time.  Rejoice.

I'm not sure that the process is broken, but I agree that there's been
way too little focus on testing this time around; it seems that most of
the chatter on pgsql-hackers since beta started has been about ideas for
8.3 development.  Have we caused that by deciding to have a short 8.3
devel cycle, ie, do people feel they needed a head start?  If so, it's
bad, but the damage is already done, and won't be repeated as long as we
go back to a more normal schedule after 8.3.  If there's another force
at work, what is it?

I am a bit worried about this, because we're predicating the decision
to release 8.2 now on the lack of bug reports; if that's due to lack of
testing rather than lack of bugs, we might have a disaster in the
making.  But there's no way to know that now, and really I see no value
in being fearful at this point.  If we delayed a month, we'd be in
pretty much just the same situation a month from now.
        regards, tom lane


Re: [CORE] RC1 blocker issues

From
Theo Schlossnagle
Date:
On Nov 26, 2006, at 6:50 PM, Peter Eisentraut wrote:

> Josh Berkus wrote:
>> Just so you all know: I gave a green flag for two magazines (one in
>> Middle East, one in Dutch) to carry articles about 8.2 for their
>> January issues, which will come out in December.   So there's a
>> distinct possibility that those magazines will come out and 8.2 still
>> won't be available.  I hadn't anticipated us slipping the schedule by
>> more than a month.  Not that this should be our deciding criteria,
>> but just so people know that I'm not just pushing the date for no
>> reason.
>
> We have this issue every year and it's never ended up being a problem.
> It's the author's or publisher's fault if they can't write their
> articles ambiguous enough.

If the article is about 8.2 (which is likely should be to pique  
interest) then there is only so much ambiguity that can be afforded.   
Additionally, matching releases of articles, books and other media  
coverage around the time of a release is common practice to and  
publishers like it when they align.  The above statement sounds like  
it has a "world-revolves around PostgreSQL" sentiment.  While it does  
on this list, it doesn't elsewhere.  If it is the intention to  
leverage the press coverage and articles for PostgreSQL's public  
face, then it would be good to respect that even if you can't  
appreciate it.

>> Also note that if we release between December 18th and January 3rd,
>> we can pretty much expect no coverage from the press.
>
> We also have this issue every year and I still don't believe it.
> Curiously, there is always excellent press coverage of everything else
> between December 18th and January 3rd.

Retail and consumer press is hot then.  Business press is dead.   
Don't both doing a press release then, it'd be better to wait.

// Theo Schlossnagle
// CTO -- http://www.omniti.com/~jesus/
// OmniTI Computer Consulting, Inc. -- http://www.omniti.com/




Re: [CORE] RC1 blocker issues

From
Josh Berkus
Date:
Peter,

> We have this issue every year and it's never ended up being a problem.
> It's the author's or publisher's fault if they can't write their
> articles ambiguous enough.

Well, the problem with that attitude is that it's pretty much a statement that 
we don't want magazine coverage.  Magazine editors will not keep covering us 
if they can't target an issue towards an article 1-2 months *after* the 
release date, as predicted 3 months in advance due to print lead times.  I've 
already had an article proposal rejected by one magazine due to not being 
able to promise this.  I suspect that the two magazines cited will not cover 
us a second time.

And last I checked, it was still our goal to seek "equal airtime" with other 
DBMSes in the press.   Or have we given up competing?

I'm not saying that our release schedules should be determined by PR concerns.  
PR/Marketing has always been our lowest priority.  There's a difference 
between "lowest priority" and "ignoring completely".  If we are callous and 
ignorant of the needs of the press when making releases, then we will get no 
press coverage, which will then make it even harder for our users to get 
their bosses to approve using PostgreSQL officially.

> We also have this issue every year and I still don't believe it.
> Curiously, there is always excellent press coverage of everything else
> between December 18th and January 3rd.

Not in the United States, which still dominates the tech press.  Maybe it's 
different in Germany.  We've never done a release between those dates; in 
fact, we held back 7.4 for that reason.

> We also have this issue every year, but the time from beta to release
> has always been about three months.  With the buildfarm helping out,
> we're a couple of weeks early this time.  Rejoice.

I got spoiled by 8.1, which released in an 8-week beta with no slippage.  
Surely we can do that again?

-- 
Josh Berkus
PostgreSQL @ Sun
San Francisco


Re: [CORE] RC1 blocker issues

From
"Marc G. Fournier"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



- --On Sunday, November 26, 2006 17:41:28 -0800 Josh Berkus <josh@agliodbs.com> 
wrote:

> I got spoiled by 8.1, which released in an 8-week beta with no slippage.
> Surely we can do that again?

Except that at least three ppl so far have stated that we are on time ... Tom 
Lane, PeterE (as he stated, thanks to the buildfarm, we are actually a bit 
ahead of our normal scheduale) and myself ...

Not sure why this thread is continuing, when the problem that was brouht up by 
the trhead didn't actually exist in the first place :(


Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email . scrappy@hub.org                              MSN . scrappy@hub.org
Yahoo . yscrappy               Skype: hub.org        ICQ . 7615664
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (FreeBSD)

iD8DBQFFakZ14QvfyHIvDvMRAoFFAJ41hvU4dPTg2rV8VUTKpjFijB0tXACfbROn
Rh888LXX0wxIYCJI9mRp7bU=
=+WOh
-----END PGP SIGNATURE-----



Re: [CORE] RC1 blocker issues

From
"Joshua D. Drake"
Date:
Tom Lane wrote:
> Peter Eisentraut <peter_e@gmx.net> writes:
>> Josh Berkus wrote:
>>> Overall, I submit that our release process is broken, and that we're
>>> having trouble getting this release out because nobody is paying
>>> attention to it (of which I too have been guilty off-and-on).
> 
>> We also have this issue every year, but the time from beta to release 
>> has always been about three months.  With the buildfarm helping out, 
>> we're a couple of weeks early this time.  Rejoice.
> 
> I'm not sure that the process is broken, but I agree that there's been
> way too little focus on testing this time around; it seems that most of
> the chatter on pgsql-hackers since beta started has been about ideas for
> 8.3 development.  Have we caused that by deciding to have a short 8.3
> devel cycle, ie, do people feel they needed a head start?  If so, it's
> bad, but the damage is already done, and won't be repeated as long as we
> go back to a more normal schedule after 8.3.  If there's another force
> at work, what is it?

A couple of thoughts.

1. 8.1 is good enough ;) To be perfectly honest, I haven't looked at 8.2 
*at all* except for the few extremely minor things I did for contrib. 
There is nothing in it that my customers *need*. There are things that 
would be nice (specifically the constraint exclusion updates) but for 
the most part.. meh, my customers will skip the release anyway for 8.3.

That is not to say there isn't other good stuff (GIN), just that my 
customers are happy with 8.1.

2. I actually don't mind if 8.3 is a short or normal cycle. My main 
concern is *when* the cycle hits. Having feature freeze right before 
everyone and their mother is going to go on vacation, not to mention 
into the middle of show season seems to be a problem. Thus one option 
would be instead of a short 8.3 cycle, we have a long one. Stretch the 
cycle out by 3 months or so, instead of shrinking.

However I know that a lot of people are trying to do *alot* of work for 
8.3. I have had conversations with several individuals who want:

Recursive queries
Multi table indexes
GROUP BY/WITH
Further HOT Standby Work

These all seem like pretty big projects to do with a short lifecycle?

Sincerely,

Joshua D. Drake




Re: [CORE] RC1 blocker issues

From
"Joshua D. Drake"
Date:
>> We have this issue every year and it's never ended up being a problem.
>> It's the author's or publisher's fault if they can't write their
>> articles ambiguous enough.
> 
> If the article is about 8.2 (which is likely should be to pique 
> interest) then there is only so much ambiguity that can be afforded.  
> Additionally, matching releases of articles, books and other media 
> coverage around the time of a release is common practice to and 
> publishers like it when they align.  The above statement sounds like it 
> has a "world-revolves around PostgreSQL" sentiment.  While it does on 
> this list, it doesn't elsewhere.  If it is the intention to leverage the 
> press coverage and articles for PostgreSQL's public face, then it would 
> be good to respect that even if you can't appreciate it.

+1

> 
>>> Also note that if we release between December 18th and January 3rd,
>>> we can pretty much expect no coverage from the press.
>>
>> We also have this issue every year and I still don't believe it.
>> Curiously, there is always excellent press coverage of everything else
>> between December 18th and January 3rd.
> 
> Retail and consumer press is hot then.  Business press is dead.  Don't 
> both doing a press release then, it'd be better to wait.

+1

Joshua D. Drake


> 
> // Theo Schlossnagle
> // CTO -- http://www.omniti.com/~jesus/
> // OmniTI Computer Consulting, Inc. -- http://www.omniti.com/
> 
> 



Re: [CORE] RC1 blocker issues

From
Tom Lane
Date:
Josh Berkus <josh@agliodbs.com> writes:
> I got spoiled by 8.1, which released in an 8-week beta with no slippage.  
> Surely we can do that again?

The reason we slipped from November 15 was the discovery of a serious
bug that required a complex fix; we concluded we needed more testing
time to have any confidence in the fix.  If another such bug comes up
in the next week, I'll vote to slip again.  If not, we should release
per schedule.

8.1 was fortunate enough not to have any such bugs found during beta,
but that isn't a situation that we can promise every time out.
        regards, tom lane


Re: [CORE] RC1 blocker issues

From
Tom Lane
Date:
"Joshua D. Drake" <jd@commandprompt.com> writes:
> 1. 8.1 is good enough ;) To be perfectly honest, I haven't looked at 8.2 
> *at all* except for the few extremely minor things I did for contrib. 
> There is nothing in it that my customers *need*.

None of your customers do multiple outer joins?  Nobody has a use-case
for INSERT RETURNING, such as wanting to fetch the value assigned to a
serial column?  Nobody has a use for CREATE INDEX CONCURRENTLY?
Nobody needs an order-of-magnitude speedup in large sorts?  Nobody's
hit a context swap storm that might be fixed by 8.2?  I could go on
like this for awhile.

If you are unexcited by 8.2, I'm not entirely sure what we might
accomplish in 8.3 that *would* draw your attention.

> However I know that a lot of people are trying to do *alot* of work for 
> 8.3. I have had conversations with several individuals who want:

> Recursive queries
> Multi table indexes
> GROUP BY/WITH
> Further HOT Standby Work

> These all seem like pretty big projects to do with a short lifecycle?

Indeed, and if not one of them appears in 8.3, I won't be very surprised
nor shed any tear.  The point of the short 8.3 dev cycle is (a) to try
to align ourselves with a better time of year for beta/release cycle,
and (b) to push out several big improvements that are already nearly done
but missed 8.2, such as bitmap indexes.  Any other big projects that can
be done by March will be nice gravy, but they aren't going to get to
dictate the schedule.
        regards, tom lane


Re: [CORE] RC1 blocker issues

From
"Joshua D. Drake"
Date:
Tom Lane wrote:
> "Joshua D. Drake" <jd@commandprompt.com> writes:
>> 1. 8.1 is good enough ;) To be perfectly honest, I haven't looked at 8.2 
>> *at all* except for the few extremely minor things I did for contrib. 
>> There is nothing in it that my customers *need*.

Note, I said need, not want.

> 
> None of your customers do multiple outer joins?

Of course they do but refer to the above.

>  Nobody has a use-case
> for INSERT RETURNING, such as wanting to fetch the value assigned to a
> serial column?

currval()? lastval()?

>  Nobody has a use for CREATE INDEX CONCURRENTLY?'

Of course they do, again need not want. CREATE INDEX CONCURRENTLY is a 
great feature but it isn't something that is whiz, bang, pow (such as 
the enormous performance increase between 7.4/8.0 and 8.1).

Our most active customers, even those with many hundreds of millions of 
rows per table, can create an index reasonably quick based on the 
hardware they run. They just schedule it to run after hours or on off peak.

> Nobody needs an order-of-magnitude speedup in large sorts?  Nobody's
> hit a context swap storm that might be fixed by 8.2?  I could go on
> like this for awhile.

Don't take it personally Tom, I wasn't knocking the hard work. I was 
simply stating what I see, which is 8.1 is pretty darn good. It should 
be considered a compliment.

Of course every feature in 8.2 is appreciated, but that doesn't mean I 
have customers clamoring for them. I am just now getting most of our 
customers to move to 8.1. I still have many customers on 7.3.

Just because something *can* do something, doesn't mean that customers 
*need* it to do so. There are certainly many users/customers that will 
benefit from 8.2 but many of my customers will never even install it.

If I tell a customer 8.2 is out and we get these great features and then 
I saw, but 8.3 is less than 9 months away. You can kiss the upgrade to 
8.2 goodbye.

Especially since many of my customers are now running multi-hundred 
gigabyte databases. They need a serious reason to upgrade because it 
will be a long outage.

> 
> If you are unexcited by 8.2, I'm not entirely sure what we might
> accomplish in 8.3 that *would* draw your attention.
> 

Not unexcited, just happy with 8.1 :).

>> However I know that a lot of people are trying to do *alot* of work for 
>> 8.3. I have had conversations with several individuals who want:
> 
>> Recursive queries
>> Multi table indexes
>> GROUP BY/WITH
>> Further HOT Standby Work
> 
>> These all seem like pretty big projects to do with a short lifecycle?
> 
> Indeed, and if not one of them appears in 8.3, I won't be very surprised
> nor shed any tear.  The point of the short 8.3 dev cycle is (a) to try
> to align ourselves with a better time of year for beta/release cycle,
> and (b) to push out several big improvements that are already nearly done
> but missed 8.2, such as bitmap indexes.  Any other big projects that can
> be done by March will be nice gravy, but they aren't going to get to
> dictate the schedule.

Which pushes them to 8.4 potentially, which makes things even more 
interesting because what I list above, is what *my* customers want and 
have wanted for a long time (and yes, I tell them the same thing 
everytime... any time you want to cough up some money, I will put 
developers on it :)).

Sincerely,

Joshua D. Drake



> 
>             regards, tom lane
> 



Re: [CORE] RC1 blocker issues

From
"Marc G. Fournier"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



- --On Sunday, November 26, 2006 20:04:02 -0500 Tom Lane <tgl@sss.pgh.pa.us> 
wrote:

> I am a bit worried about this, because we're predicating the decision
> to release 8.2 now on the lack of bug reports; if that's due to lack of
> testing rather than lack of bugs, we might have a disaster in the
> making.  But there's no way to know that now, and really I see no value
> in being fearful at this point.  If we delayed a month, we'd be in
> pretty much just the same situation a month from now.

Just my $0.02, but in a way, I think you nail'd it on the head earlier in this 
thread (or a recent one) ...

The buildfarm tends to provide us with ongoing testing ... anything that the 
buildfarm *can't* test for most likely won't get tested / found until we hit a 
real world situation, and real world situations don't generally happen until 
after the release ...

So, one question that I have is ... is there some way of extending the build 
farm testing that would reduce "post-release" bugs?  Do we have any known holes 
in our regression tests that would have found any bugs reported by "humans"?

You have to remember, in the past, alot of testing revolved around ppl 
installing and running the regression tests on their platform ... buildfarm is 
automatically doing that now, which is why we haven't done a 'call for port 
reports' ...

And I'm not saying that 'human testing' isn't required anymore, only that we've 
reduced the reliance on it through the buildfarm, since any bugs that do creep 
in (that are a result of the regression tests) tend to be found before most ppl 
would be able to report them ...

- ----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email . scrappy@hub.org                              MSN . scrappy@hub.org
Yahoo . yscrappy               Skype: hub.org        ICQ . 7615664
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (FreeBSD)

iD8DBQFFan464QvfyHIvDvMRAmtgAKCKOKgibz06owFTiaRFG21i0tnfXgCfcagv
Lst5n6/iwcqzHGZ/06NezfM=
=eB9Z
-----END PGP SIGNATURE-----



Re: [CORE] RC1 blocker issues

From
Tom Lane
Date:
"Joshua D. Drake" <jd@commandprompt.com> writes:
>> Nobody has a use for CREATE INDEX CONCURRENTLY?

> Of course they do, again need not want. CREATE INDEX CONCURRENTLY is a 
> great feature but it isn't something that is whiz, bang, pow (such as 
> the enormous performance increase between 7.4/8.0 and 8.1).

Hm, are you sure there aren't equivalent performance increases between
8.1 and 8.2?  I don't know of any single performance fix we've done in
the last ten years that's not a niche feature for some value of "niche".
There are certainly things in 8.2 that are killer improvements for some
classes of applications, just as 7.4, 8.0, and 8.1 could say the same
for some other classes.  Considering that 8.2 has been focused on
performance fixes to a much greater degree than any prior release,
I find it surprising that you're dismissing it as uninteresting on
that dimension.

> Of course every feature in 8.2 is appreciated, but that doesn't mean I 
> have customers clamoring for them. I am just now getting most of our 
> customers to move to 8.1. I still have many customers on 7.3.

[blink...]  You are doing those customers a disservice.  Yes, I know
that Red Hat is still nominally supporting 7.3 and even 7.1, but that's
only because of an overly conservative corporate policy --- there are
way too many unfixable problems in those ancient releases.

> Which pushes them to 8.4 potentially, which makes things even more 
> interesting because what I list above, is what *my* customers want and 
> have wanted for a long time

AFAICS, we can do 8.3 next spring with the stuff that's already on the
table, then 8.4 in spring 2008 with the stuff you propose; or we can
do 8.3 in spring 2008 with all the above.  Assuming that anyone comes
through with the stuff you propose by 2008, a fact not in evidence.
Ultimately, this project is not driven by complainers, it's driven by
people who get things done.  Step up to the plate.
        regards, tom lane


Re: [CORE] RC1 blocker issues

From
Stefan Kaltenbrunner
Date:
Tom Lane wrote:
> "Joshua D. Drake" <jd@commandprompt.com> writes:
>>> Nobody has a use for CREATE INDEX CONCURRENTLY?
> 
>> Of course they do, again need not want. CREATE INDEX CONCURRENTLY is a 
>> great feature but it isn't something that is whiz, bang, pow (such as 
>> the enormous performance increase between 7.4/8.0 and 8.1).
> 
> Hm, are you sure there aren't equivalent performance increases between
> 8.1 and 8.2?  I don't know of any single performance fix we've done in
> the last ten years that's not a niche feature for some value of "niche".
> There are certainly things in 8.2 that are killer improvements for some
> classes of applications, just as 7.4, 8.0, and 8.1 could say the same
> for some other classes.  Considering that 8.2 has been focused on
> performance fixes to a much greater degree than any prior release,
> I find it surprising that you're dismissing it as uninteresting on
> that dimension.

from the testing i have done with some of our production databases - 8.2
gives a tremendous performance boost (nearly on a similiar scale that
7.4->8.1 gave us!). Some of those gains are planner improvments (like
the out-joins enhancements others seem to come from the improved sorting
and concurrency. Last but least 8.2 seems to use a bit less of memory
for some queries too which helps concurrency.
So I'm a bit surprised that Josh is not considering those as very
interesting either ...


Stefan


Re: [CORE] RC1 blocker issues

From
"Joshua D. Drake"
Date:
>> Of course every feature in 8.2 is appreciated, but that doesn't mean I 
>> have customers clamoring for them. I am just now getting most of our 
>> customers to move to 8.1. I still have many customers on 7.3.
> 
> [blink...]  You are doing those customers a disservice.  Yes, I know
> that Red Hat is still nominally supporting 7.3 and even 7.1, but that's
> only because of an overly conservative corporate policy --- there are
> way too many unfixable problems in those ancient releases.

You are preaching to the choir here :) but money talks, and if the app 
works.... I have an insurance company that is on 7.3.15, why? Because it 
works for them. Their app is in a mature and stable life cycle with very 
few changes.

If it makes you feel any better, the majority are moving away and toward 
8.1 but (new business excluded) customers don't like to change. They 
want to rely on their platform.

We have finally determined that Replicator is EOF for 7.3 and 7.4 and 
will not be supported *period* past 12/31/06, which has forced some 
migrations for customers. However for customers that are not using 
replicator, there isn't a whole lot we can do...

Now if PostgreSQL.Org were to officially announce EOF of certain 
versions, it would definitely lend weight to my argument to customers.


>> Which pushes them to 8.4 potentially, which makes things even more 
>> interesting because what I list above, is what *my* customers want and 
>> have wanted for a long time
> 
> AFAICS, we can do 8.3 next spring with the stuff that's already on the
> table, then 8.4 in spring 2008 with the stuff you propose; or we can
> do 8.3 in spring 2008 with all the above.  Assuming that anyone comes
> through with the stuff you propose by 2008, a fact not in evidence.
> Ultimately, this project is not driven by complainers, it's driven by
> people who get things done.  Step up to the plate.

Well I am certainly not complaining and I will be the first to tell 
people to put up or shut up. Well, second in this case :). I am just 
stating what reality is for my customers and that reality is that 8.2 is 
a no-op release with 8.3 only 6 - 9 months away.

I am also not saying that pushing 8.3 to spring 08 is a good idea 
either, that is an awful long time without a feature release.

Hmmm, thoughts on an Ubuntu style release structure (every 6 months, 
with a single release during a cycle considered LTS?)

Sincerely,

Joshua D. Drake



> 
>             regards, tom lane
> 



Re: [CORE] RC1 blocker issues

From
"Joshua D. Drake"
Date:
> from the testing i have done with some of our production databases - 8.2
> gives a tremendous performance boost (nearly on a similiar scale that
> 7.4->8.1 gave us!). Some of those gains are planner improvments (like
> the out-joins enhancements others seem to come from the improved sorting
> and concurrency. Last but least 8.2 seems to use a bit less of memory
> for some queries too which helps concurrency.
> So I'm a bit surprised that Josh is not considering those as very
> interesting either ...

O.k. hold on... let's be realistic. If I have a 500 Gig database, and I 
know that 8.3 is coming in 6-9 months... why would I migrate to 8.2 with 
8.3 literally right around the corner?

Now take into account that 8.1 works just fine for the customer?

What is my argument?

It's faster? The customer isn't having performance issues... So what's 
the argument?

I can build indexes without an exclusive lock? I am running 75-150k in 
hardware... I build indexes fast anyway.

Constraint exclusion works for updates and deletes? Well that is 
certainly useful, but our major issue was SELECTS and you already built 
out a complete partitioning system.

And frankly, CMD has a standing policy to not push a .0 release. Ever. 
If a customer comes to me and says I have a mission critical system that 
is currently making me *n* amount of dollars an hour, what version of 
PostgreSQL would you suggest? That version will be 8.1.5 until at least 
Feb/March depending on what happens as early adopters pick up.

Again, I am not complaining, nor being negative about 8.2 but I don't 
get the leisure of playing with software. I work with software. That 
means measured, timed and slow migrations to stable releases.

Unfortunately in this case it means that the software may get skipped 
because I know (although the customer likely doesn't) that a new major 
release is due around my birthday.

Sincerely,

Joshua D. Drake




Re: [CORE] RC1 blocker issues

From
Stefan Kaltenbrunner
Date:
Joshua D. Drake wrote:
> 
>> from the testing i have done with some of our production databases - 8.2
>> gives a tremendous performance boost (nearly on a similiar scale that
>> 7.4->8.1 gave us!). Some of those gains are planner improvments (like
>> the out-joins enhancements others seem to come from the improved sorting
>> and concurrency. Last but least 8.2 seems to use a bit less of memory
>> for some queries too which helps concurrency.
>> So I'm a bit surprised that Josh is not considering those as very
>> interesting either ...
> 
> O.k. hold on... let's be realistic. If I have a 500 Gig database, and I 
> know that 8.3 is coming in 6-9 months... why would I migrate to 8.2 with 
> 8.3 literally right around the corner?

so what ? there is not guarantee that 8.3 _WILL_ be there in that 
timeframe and given that you might not want to jump onto 8.3.0 that 
timeframe could be much longer.

> 
> Now take into account that 8.1 works just fine for the customer?

well then why even thinking about upgrading to 8.3 or 8.4 or 9.0 or 
whatever ?

> 
> What is my argument?
> 
> It's faster? The customer isn't having performance issues... So what's 
> the argument?
> 
> I can build indexes without an exclusive lock? I am running 75-150k in 
> hardware... I build indexes fast anyway.

That is probably not directly dependent on the hardware - creating 
indexes in PostgreSQL is directly dependent on the speed of the single 
CPU doing the building (given you more than a single IDE disk).
If you look back to the thread that caused the massive improvements in 
external sort speed you will see that we are talking something like 12 
hours vs. a bit more then 2 hours for a 1.8B row table on what i think 
was the fastest single core available at that point (2.6Ghz Opteron).
While most tables are much smaller in reality that improvment might 
result in the difference between a "hardly noticable outage" and "real 
downtime" due to the exclusive looking create index requires(and it 
speeds up dump & restore cycles considerably so making upgrades less 
problematic).

> 
> Constraint exclusion works for updates and deletes? Well that is 
> certainly useful, but our major issue was SELECTS and you already built 
> out a complete partitioning system.

joins against UNION ALL'd tables are one of the things where 8.2 has 
improved considerably

> 
> And frankly, CMD has a standing policy to not push a .0 release. Ever. 
> If a customer comes to me and says I have a mission critical system that 
> is currently making me *n* amount of dollars an hour, what version of 
> PostgreSQL would you suggest? That version will be 8.1.5 until at least 
> Feb/March depending on what happens as early adopters pick up.

see above - per that logic 8.3 is more then 6 months away for you - and 
there is a year of development in 8.2 fixing a lot of small things that 
might bite(or rather annoy) you (or not *g*) sometime.

> 
> Again, I am not complaining, nor being negative about 8.2 but I don't 
> get the leisure of playing with software. I work with software. That 
> means measured, timed and slow migrations to stable releases.

fair point (and in fact we do the same here) - but your original mail 
made the impression that you are underimpressed by 8.2 which might be 
true for your use cases but as a general statement it is not in my opinion.


Stefan


Re: [CORE] RC1 blocker issues

From
"Heikki Linnakangas"
Date:
Tom Lane wrote:
> I'm not sure that the process is broken, but I agree that there's been
> way too little focus on testing this time around; it seems that most of
> the chatter on pgsql-hackers since beta started has been about ideas for
> 8.3 development.  Have we caused that by deciding to have a short 8.3
> devel cycle, ie, do people feel they needed a head start?  If so, it's
> bad, but the damage is already done, and won't be repeated as long as we
> go back to a more normal schedule after 8.3.  If there's another force
> at work, what is it?

I can't speak for others, but I'd really like to be of more help in 
testing. I'd like to see 8.2 released as soon as possible, so that we 
can start discussing 8.3 items.

I don't have a real-world application to test, but if there's anything I 
can do, please let me know. Any gray areas that haven't received enough 
attention yet?

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com


Re: [CORE] RC1 blocker issues

From
Andrew Sullivan
Date:
On Sun, Nov 26, 2006 at 11:31:28PM -0800, Joshua D. Drake wrote:
> I can build indexes without an exclusive lock? I am running 75-150k in 
> hardware... I build indexes fast anyway.

"Build fast" sure isn't the same as "don't block", though.  But
whatever.

One reason to use free software is precisely that you don't have to
get on the upgrade treadmill.  If a customer does or does not want to
upgrade, presumably that is their choice.

A

-- 
Andrew Sullivan  | ajs@crankycanuck.ca
Everything that happens in the world happens at some place.    --Jane Jacobs 


Re: [CORE] RC1 blocker issues

From
"Joshua D. Drake"
Date:
On Mon, 2006-11-27 at 09:35 -0500, Andrew Sullivan wrote:
> On Sun, Nov 26, 2006 at 11:31:28PM -0800, Joshua D. Drake wrote:
> > I can build indexes without an exclusive lock? I am running 75-150k in 
> > hardware... I build indexes fast anyway.
> 
> "Build fast" sure isn't the same as "don't block", though.  But
> whatever.

It is when the customer doesn't mind that it blocks, because it builds
fast :)

> 
> One reason to use free software is precisely that you don't have to
> get on the upgrade treadmill.  If a customer does or does not want to
> upgrade, presumably that is their choice.

Agreed.

Sincerely,

Joshua D. Drake


> 
> A
> 
-- 
     === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive  PostgreSQL solutions since 1997            http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate





Re: [CORE] RC1 blocker issues

From
"Jim C. Nasby"
Date:
On Mon, Nov 27, 2006 at 12:10:11PM +0000, Heikki Linnakangas wrote:
> Tom Lane wrote:
> >I'm not sure that the process is broken, but I agree that there's been
> >way too little focus on testing this time around; it seems that most of
> >the chatter on pgsql-hackers since beta started has been about ideas for
> >8.3 development.  Have we caused that by deciding to have a short 8.3
> >devel cycle, ie, do people feel they needed a head start?  If so, it's
> >bad, but the damage is already done, and won't be repeated as long as we
> >go back to a more normal schedule after 8.3.  If there's another force
> >at work, what is it?
> 
> I can't speak for others, but I'd really like to be of more help in 
> testing. I'd like to see 8.2 released as soon as possible, so that we 
> can start discussing 8.3 items.
> 
> I don't have a real-world application to test, but if there's anything I 
> can do, please let me know. Any gray areas that haven't received enough 
> attention yet?

Since testing in the past had a semi-defined goal (make sure it builds
and passes make check on your platform), perhaps that should be done in
the future; point out to users what features they should pay special
attention to during the beta. Of course, that will tend to mirror what's
in the change log, but people will pay better attention to a
plain-language description of what to test than trying to decipher it
from the change log.
-- 
Jim Nasby                                            jim@nasby.net
EnterpriseDB      http://enterprisedb.com      512.569.9461 (cell)


Re: [CORE] RC1 blocker issues

From
Bruce Momjian
Date:
Tom Lane wrote:
> Peter Eisentraut <peter_e@gmx.net> writes:
> > Josh Berkus wrote:
> >> Overall, I submit that our release process is broken, and that we're
> >> having trouble getting this release out because nobody is paying
> >> attention to it (of which I too have been guilty off-and-on).
> 
> > We also have this issue every year, but the time from beta to release 
> > has always been about three months.  With the buildfarm helping out, 
> > we're a couple of weeks early this time.  Rejoice.
> 
> I'm not sure that the process is broken, but I agree that there's been
> way too little focus on testing this time around; it seems that most of
> the chatter on pgsql-hackers since beta started has been about ideas for
> 8.3 development.  Have we caused that by deciding to have a short 8.3
> devel cycle, ie, do people feel they needed a head start?  If so, it's
> bad, but the damage is already done, and won't be repeated as long as we
> go back to a more normal schedule after 8.3.  If there's another force
> at work, what is it?

I have been pushing people to announce and ask for feedback on 8.3 work,
particularly because many didn't do that for 8.2, and because 8.3 will
be a short cycle.  Seeing the problems it caused, I have backed off from
doing this anymore.

> I am a bit worried about this, because we're predicating the decision
> to release 8.2 now on the lack of bug reports; if that's due to lack of
> testing rather than lack of bugs, we might have a disaster in the
> making.  But there's no way to know that now, and really I see no value
> in being fearful at this point.  If we delayed a month, we'd be in
> pretty much just the same situation a month from now.

My feeling is that the community sees there are beta and RC candidates. 
If they don't want to test them and report, there isn't much more we can
do.  Fundamentally it is the community's release.

--  Bruce Momjian   bruce@momjian.us EnterpriseDB    http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: [CORE] RC1 blocker issues

From
"Kevin Grittner"
Date:
>>> On Sun, Nov 26, 2006 at  7:04 PM, in message
<659.1164589442@sss.pgh.pa.us>,
Tom Lane <tgl@sss.pgh.pa.us> wrote: 
> . . . it seems that most of
> the chatter on pgsql- hackers since beta started has been about ideas
for
> 8.3 development.  Have we caused that by deciding to have a short
8.3
> devel cycle, ie, do people feel they needed a head start?  If so,
it's
> bad, but the damage is already done, and won't be repeated as long as
we
> go back to a more normal schedule after 8.3.  If there's another
force
> at work, what is it?
> 
> I am a bit worried about this, because we're predicating the
decision
> to release 8.2 now on the lack of bug reports; if that's due to lack
of
> testing rather than lack of bugs, we might have a disaster in the
> making.  But there's no way to know that now, and really I see no
value
> in being fearful at this point.  If we delayed a month, we'd be in
> pretty much just the same situation a month from now.
For what it's worth, we have not had any problems with the 8.2beta3
release.
We converted one of our databases which contains statewide circuit
court data to beta3 on November 11th, as soon as the beta3 release was
available.  We've been replicating data from all 72 county databases
since then with no trouble.  We did a stress simulating our public web
traffic (http://wcca.wicourts.gov) by using HTTP requests from our log
files and multiple renderers and got fantastic performance.  (It settled
in at 120 to 150 web requests per second, with an average of 5 to 10
queries run per request.  The Java middle tiers through which all data
pass were running on the same box, and the replication was active at the
time.)  
We have also been using beta3 for our development and first stage
testing of new software, and have recently converted on copy of our
largest database (a 400 GB searchable transaction repository) with no
problems.
While we have avoided serving up data to the pubic or to the court
system end users from the beta databases, I would feel comfortable doing
that with this release right now should our production copies all fail. 
That is based on both our experience and the fact that they only change
which is a new "bug" in 8.2 was (in my eyes at least) strictly a
cosmetic issue -- nobody should rely on the order of columns when using
"SELECT *".  It's unfortunate that we have to go through another initdb
/ pg_dump on these, but I understand the arguments for the change, and
we did know it was still beta when we decided to go that way.
-Kevin



Re: [CORE] RC1 blocker issues

From
Bruce Momjian
Date:
Jim C. Nasby wrote:
> On Mon, Nov 27, 2006 at 12:10:11PM +0000, Heikki Linnakangas wrote:
> > Tom Lane wrote:
> > >I'm not sure that the process is broken, but I agree that there's been
> > >way too little focus on testing this time around; it seems that most of
> > >the chatter on pgsql-hackers since beta started has been about ideas for
> > >8.3 development.  Have we caused that by deciding to have a short 8.3
> > >devel cycle, ie, do people feel they needed a head start?  If so, it's
> > >bad, but the damage is already done, and won't be repeated as long as we
> > >go back to a more normal schedule after 8.3.  If there's another force
> > >at work, what is it?
> > 
> > I can't speak for others, but I'd really like to be of more help in 
> > testing. I'd like to see 8.2 released as soon as possible, so that we 
> > can start discussing 8.3 items.
> > 
> > I don't have a real-world application to test, but if there's anything I 
> > can do, please let me know. Any gray areas that haven't received enough 
> > attention yet?
> 
> Since testing in the past had a semi-defined goal (make sure it builds
> and passes make check on your platform), perhaps that should be done in
> the future; point out to users what features they should pay special
> attention to during the beta. Of course, that will tend to mirror what's
> in the change log, but people will pay better attention to a
> plain-language description of what to test than trying to decipher it
> from the change log.

The release notes are available on day-1 of beta.

--  Bruce Momjian   bruce@momjian.us EnterpriseDB    http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: [CORE] RC1 blocker issues

From
"Jim C. Nasby"
Date:
On Mon, Nov 27, 2006 at 01:17:26PM -0500, Bruce Momjian wrote:
> Jim C. Nasby wrote:
> > Since testing in the past had a semi-defined goal (make sure it builds
> > and passes make check on your platform), perhaps that should be done in
> > the future; point out to users what features they should pay special
> > attention to during the beta. Of course, that will tend to mirror what's
> > in the change log, but people will pay better attention to a
> > plain-language description of what to test than trying to decipher it
> > from the change log.
> 
> The release notes are available on day-1 of beta.

Of course, but hoping that users will read the release notes and start
testing everything in there isn't the same as putting out a call for
testing on X, Y, and Z. The release notes also don't pay any attention
to what's already handled by regression tests, and more important,
what's not. Presumably doing things like testing syntax doesn't do
nearly as much good as performance or scalability testing.
-- 
Jim Nasby                                            jim@nasby.net
EnterpriseDB      http://enterprisedb.com      512.569.9461 (cell)


Re: [CORE] RC1 blocker issues

From
Tom Lane
Date:
"Jim C. Nasby" <jim@nasby.net> writes:
> On Mon, Nov 27, 2006 at 01:17:26PM -0500, Bruce Momjian wrote:
>> The release notes are available on day-1 of beta.

> Of course, but hoping that users will read the release notes and start
> testing everything in there isn't the same as putting out a call for
> testing on X, Y, and Z. The release notes also don't pay any attention
> to what's already handled by regression tests, and more important,
> what's not. Presumably doing things like testing syntax doesn't do
> nearly as much good as performance or scalability testing.

I think that trying to put forward any detailed testing guidelines could
be counterproductive.  Testing is valuable to the extent that testers
try things the code authors did not think of.  ISTM that publishing
"test this" recommendations would tend to slot testers' thoughts into
the same mindset the authors had.
        regards, tom lane


Re: [CORE] RC1 blocker issues

From
David Fetter
Date:
On Sun, Nov 26, 2006 at 08:42:26PM -0800, Joshua D. Drake wrote:
> Tom Lane wrote:
> >"Joshua D. Drake" <jd@commandprompt.com> writes:
> >Nobody has a use-case for INSERT RETURNING, such as wanting to
> >fetch the value assigned to a serial column?
> 
> currval()? lastval()?

INSERT ... RETURNING can return a rowset, not just one particular part
of one particular row.

> > Nobody has a use for CREATE INDEX CONCURRENTLY?'
> 
> Of course they do, again need not want. CREATE INDEX CONCURRENTLY is
> a great feature but it isn't something that is whiz, bang, pow (such
> as the enormous performance increase between 7.4/8.0 and 8.1).
> 
> Our most active customers, even those with many hundreds of millions
> of rows per table, can create an index reasonably quick based on the
> hardware they run. They just schedule it to run after hours or on
> off peak.

All that needs to happen is for this to take 49 hours.  Suddenly,
there's a use case ;)

> >Nobody needs an order-of-magnitude speedup in large sorts?
> >Nobody's hit a context swap storm that might be fixed by 8.2?  I
> >could go on like this for awhile.
> 
> Don't take it personally Tom, I wasn't knocking the hard work. I was
> simply stating what I see, which is 8.1 is pretty darn good. It
> should be considered a compliment.
> 
> Of course every feature in 8.2 is appreciated, but that doesn't mean
> I have customers clamoring for them. I am just now getting most of
> our customers to move to 8.1. I still have many customers on 7.3.

That's a big problem for both you and your customers.  At some point
in the not too distant future, 7.3 will get EOLed.

> Just because something *can* do something, doesn't mean that
> customers *need* it to do so. There are certainly many
> users/customers that will benefit from 8.2 but many of my customers
> will never even install it.
> 
> If I tell a customer 8.2 is out and we get these great features and
> then I saw, but 8.3 is less than 9 months away. You can kiss the
> upgrade to 8.2 goodbye.
> 
> Especially since many of my customers are now running multi-hundred
> gigabyte databases. They need a serious reason to upgrade because it
> will be a long outage.

The performance and feature gains from 8.1 to 8.2 are fairly easy to
justify on this scale.

> >>However I know that a lot of people are trying to do *alot* of work for 
> >>8.3. I have had conversations with several individuals who want:
> >
> >>Recursive queries
> >>Multi table indexes
> >>GROUP BY/WITH
> >>Further HOT Standby Work
> >
> >>These all seem like pretty big projects to do with a short
> >>lifecycle?
> >
> >Indeed, and if not one of them appears in 8.3, I won't be very
> >surprised nor shed any tear.  The point of the short 8.3 dev cycle
> >is (a) to try to align ourselves with a better time of year for
> >beta/release cycle, and (b) to push out several big improvements
> >that are already nearly done but missed 8.2, such as bitmap
> >indexes.  Any other big projects that can be done by March will be
> >nice gravy, but they aren't going to get to dictate the schedule.
> 
> Which pushes them to 8.4 potentially, which makes things even more
> interesting because what I list above, is what *my* customers want
> and have wanted for a long time (and yes, I tell them the same thing
> everytime... any time you want to cough up some money, I will put
> developers on it :)).

Have any of them gotten close to doing this?  What approaches have you
tried?

Cheers,
D
-- 
David Fetter <david@fetter.org> http://fetter.org/
phone: +1 415 235 3778        AIM: dfetter666                             Skype: davidfetter

Remember to vote!


Re: [CORE] RC1 blocker issues

From
"Joshua D. Drake"
Date:
> > Of course they do, again need not want. CREATE INDEX CONCURRENTLY is
> > a great feature but it isn't something that is whiz, bang, pow (such
> > as the enormous performance increase between 7.4/8.0 and 8.1).
> > 
> > Our most active customers, even those with many hundreds of millions
> > of rows per table, can create an index reasonably quick based on the
> > hardware they run. They just schedule it to run after hours or on
> > off peak.
> 
> All that needs to happen is for this to take 49 hours.  Suddenly,
> there's a use case ;)

Agreed :) 

> > Of course every feature in 8.2 is appreciated, but that doesn't mean
> > I have customers clamoring for them. I am just now getting most of
> > our customers to move to 8.1. I still have many customers on 7.3.
> 
> That's a big problem for both you and your customers.  At some point
> in the not too distant future, 7.3 will get EOLed.

Not a big problem for me, a big problem for the customer. They have all
been warned. I will smile humbly as I collect the migration check.

>  
> > Especially since many of my customers are now running multi-hundred
> > gigabyte databases. They need a serious reason to upgrade because it
> > will be a long outage.
> 
> The performance and feature gains from 8.1 to 8.2 are fairly easy to
> justify on this scale.

Not if they aren't having performance problems now.

> > Which pushes them to 8.4 potentially, which makes things even more
> > interesting because what I list above, is what *my* customers want
> > and have wanted for a long time (and yes, I tell them the same thing
> > everytime... any time you want to cough up some money, I will put
> > developers on it :)).
> 
> Have any of them gotten close to doing this?  What approaches have you
> tried?

ODBCng was a successful project that was a direct correlation between
need and solutions available.

Joshua D. Drake

> 
> Cheers,
> D
-- 
     === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive  PostgreSQL solutions since 1997            http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate





Re: RC1 blocker issues

From
"Karen Hill"
Date:
"Joshua D. Drake" wrote:

> >> However I know that a lot of people are trying to do *alot* of work for
> >> 8.3. I have had conversations with several individuals who want:
> >
> >> Recursive queries
> >> Multi table indexes
> >> GROUP BY/WITH
> >> Further HOT Standby Work
> >
> >> These all seem like pretty big projects to do with a short lifecycle?
> >
> > Indeed, and if not one of them appears in 8.3, I won't be very surprised
> > nor shed any tear.  The point of the short 8.3 dev cycle is (a) to try
> > to align ourselves with a better time of year for beta/release cycle,
> > and (b) to push out several big improvements that are already nearly done
> > but missed 8.2, such as bitmap indexes.  Any other big projects that can
> > be done by March will be nice gravy, but they aren't going to get to
> > dictate the schedule.
>
> Which pushes them to 8.4 potentially, which makes things even more
> interesting because what I list above, is what *my* customers want and
> have wanted for a long time (and yes, I tell them the same thing
> everytime... any time you want to cough up some money, I will put
> developers on it :)).
>

Are any of them asking for SQL:2003 Window functions?  I think having
window functions in PostgreSQL would bridge the gap between PostgreSQL
and DB2/Oracle significantly.  I see DBAs creating window functions all
the time for business users in Oracle. Would Window functions even make
Postgresql viable in the data warehouse/OLAP market?  If so, just
having them would justify a version bump up to 9, wouldn't it?

I do suspect that implementing them would be a challenge, aren't
SQL:2003 Window function operations O(n^2)?  If they are, I don't know
what kind of magic Oracle performs under the hood because when I've
stood over the shoulder of an Oracle DBA as an intern, they were quite
snappy in returning the results.

regards,
karen



Re: RC1 blocker issues

From
Josh Berkus
Date:
Karen,

> Are any of them asking for SQL:2003 Window functions?  I think having
> window functions in PostgreSQL would bridge the gap between PostgreSQL
> and DB2/Oracle significantly.  I see DBAs creating window functions all
> the time for business users in Oracle. Would Window functions even make
> Postgresql viable in the data warehouse/OLAP market?

These are very much wanted, but also very, very difficult.  If you know
anyone who wants to devote about 600 hours to working on them, please let
me know ...

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco


Re: RC1 blocker issues

From
Gavin Sherry
Date:
On Tue, 27 Nov 2006, Karen Hill wrote:

> "Joshua D. Drake" wrote:
>
> > >> However I know that a lot of people are trying to do *alot* of work for
> > >> 8.3. I have had conversations with several individuals who want:
> > >
> > >> Recursive queries
> > >> Multi table indexes
> > >> GROUP BY/WITH
> > >> Further HOT Standby Work
> > >
> > >> These all seem like pretty big projects to do with a short lifecycle?
> > >
> > > Indeed, and if not one of them appears in 8.3, I won't be very surprised
> > > nor shed any tear.  The point of the short 8.3 dev cycle is (a) to try
> > > to align ourselves with a better time of year for beta/release cycle,
> > > and (b) to push out several big improvements that are already nearly done
> > > but missed 8.2, such as bitmap indexes.  Any other big projects that can
> > > be done by March will be nice gravy, but they aren't going to get to
> > > dictate the schedule.
> >
> > Which pushes them to 8.4 potentially, which makes things even more
> > interesting because what I list above, is what *my* customers want and
> > have wanted for a long time (and yes, I tell them the same thing
> > everytime... any time you want to cough up some money, I will put
> > developers on it :)).
> >
>
> Are any of them asking for SQL:2003 Window functions?  I think having
> window functions in PostgreSQL would bridge the gap between PostgreSQL
> and DB2/Oracle significantly.  I see DBAs creating window functions all
> the time for business users in Oracle. Would Window functions even make
> Postgresql viable in the data warehouse/OLAP market?  If so, just
> having them would justify a version bump up to 9, wouldn't it?
>
> I do suspect that implementing them would be a challenge, aren't
> SQL:2003 Window function operations O(n^2)?  If they are, I don't know
> what kind of magic Oracle performs under the hood because when I've
> stood over the shoulder of an Oracle DBA as an intern, they were quite
> snappy in returning the results.

I'm putting together a proposal for these. I cannot promise that they will
make it into 8.3 however -- not least because there's a fair bit of work
to be done. As you say, however, they're remarkably cool and very
important to Oracle data warehouse users.

If anyone else is looking at window functions, please raise their hand so
that we can coordinate a proposal to hackers.

Thanks,

Gavin


Re: [CORE] RC1 blocker issues

From
"Zeugswetter Andreas ADI SD"
Date:
> "SELECT *".  It's unfortunate that we have to go through another
initdb
> / pg_dump on these, but I understand the arguments for the change, and
> we did know it was still beta when we decided to go that way.

You can certainly find a workaround to not need an initdb.
drop indexes based on the changed to_char function,
verify other incompatible changes since beta3 ?? anybody think of
something else ?
shutdown the db,
binary edit the new version into pg_control,
start db with new version,
recreate the changed system view,
recreate indexes dropped above.

It is just that this is not automated. But if we make a list of the
initdb forcing
changes, it should be possible.

Andreas


Re: [CORE] RC1 blocker issues

From
"Kevin Grittner"
Date:
>>> On Tue, Nov 28, 2006 at  5:44 AM, in message
<E1539E0ED7043848906A8FF995BDA579018FE15F@m0143.s-mxs.net>,
"Zeugswetter
Andreas ADI SD" <ZeugswetterA@spardat.at> wrote: 

>> "SELECT *".  It's unfortunate that we have to go through another
>> initdb
> 
> You can certainly find a workaround to not need an initdb.
That would save some time.
> drop indexes based on the changed to_char function,
There are none.
> verify other incompatible changes since beta3 ?? anybody think of
> something else ?
Anybody?
> shutdown the db, 
> binary edit the new version into pg_control,
Anything besides offset 000C to 000F?
> start db with new version,
> recreate the changed system view,
Are these all of them?:
pg_stat_all_tables
pg_stat_sys_tables
pg_stat_user_tables
Any special tricks to replacing these in a database that's been in
use?
Thanks,
-Kevin