Thread: beta5 ...

beta5 ...

From
The Hermit Hacker
Date:
things appear to have quieted off nicely ... so would like to put out a
Beta5 for testing ...

Tom, I saw/read your proposal about the JOIN syntax, but haven't seen any
commit on it yet, nor any arguments against the changes ... so just
wondering where those stand right now?

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org





Re: beta5 ...

From
Tom Lane
Date:
The Hermit Hacker <scrappy@hub.org> writes:
> things appear to have quieted off nicely ... so would like to put out a
> Beta5 for testing ...

> Tom, I saw/read your proposal about the JOIN syntax, but haven't seen any
> commit on it yet, nor any arguments against the changes ... so just
> wondering where those stand right now?

You must have been looking the other way ;-) ... it's committed.

What I'm currently thinking about is the discussion from last week where
Vadim reported that he could get "stuck spinlock" errors during btree
index crash recovery, because the backend fixing the index might hold
disk-buffer locks longer than the ~70 second timeout for spinlocks
(see "Btree runtime recovery. Stuck spins" thread on 2/8 and 2/9).

Vadim says (and I agree) that we really ought to implement a new
lightweight lock manager that would fall between spinlocks and regular
locks in terms of overhead and functionality.  But it's not reasonable
to try to do that for 7.1 at this late date.  So I was trying to pick a
stopgap solution for 7.1.  Unfortunately Vadim's off to Siberia and I
can't consult with him...

I'm currently thinking of modifying the buffer manager so that disk
buffer spinlocks use an alternate version of s_lock() with no timeout,
and perhaps longer sleeps (no zero-delay selects anyway).  This was one
of the ideas we kicked around last week, and I think it's about the best
we can do for now.  Comments anyone?

Other than that, I have nothing to hold up a beta5.  Anyone else?
        regards, tom lane


Re: beta5 ...

From
Bruce Momjian
Date:
I am GO.  SET DIAGNOSTICS is my only open item left.


> The Hermit Hacker <scrappy@hub.org> writes:
> > things appear to have quieted off nicely ... so would like to put out a
> > Beta5 for testing ...
> 
> > Tom, I saw/read your proposal about the JOIN syntax, but haven't seen any
> > commit on it yet, nor any arguments against the changes ... so just
> > wondering where those stand right now?
> 
> You must have been looking the other way ;-) ... it's committed.
> 
> What I'm currently thinking about is the discussion from last week where
> Vadim reported that he could get "stuck spinlock" errors during btree
> index crash recovery, because the backend fixing the index might hold
> disk-buffer locks longer than the ~70 second timeout for spinlocks
> (see "Btree runtime recovery. Stuck spins" thread on 2/8 and 2/9).
> 
> Vadim says (and I agree) that we really ought to implement a new
> lightweight lock manager that would fall between spinlocks and regular
> locks in terms of overhead and functionality.  But it's not reasonable
> to try to do that for 7.1 at this late date.  So I was trying to pick a
> stopgap solution for 7.1.  Unfortunately Vadim's off to Siberia and I
> can't consult with him...
> 
> I'm currently thinking of modifying the buffer manager so that disk
> buffer spinlocks use an alternate version of s_lock() with no timeout,
> and perhaps longer sleeps (no zero-delay selects anyway).  This was one
> of the ideas we kicked around last week, and I think it's about the best
> we can do for now.  Comments anyone?
> 
> Other than that, I have nothing to hold up a beta5.  Anyone else?
> 
>             regards, tom lane
> 


--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: beta5 ...

From
Lincoln Yeoh
Date:
At 04:17 PM 2/16/01 -0500, Tom Lane wrote:
>
>Vadim says (and I agree) that we really ought to implement a new
>lightweight lock manager that would fall between spinlocks and regular
>locks in terms of overhead and functionality.  But it's not reasonable

Will there be an arbitrary user locking feature? E.g. lock on arbitrary
text string. That would be great :). 

BTW, is 7.1 going to be a bit slower than 7.0? Or just Beta 5? Just
curious. Don't mind waiting for 7.2 for the speed-up if necessary.

Cheerio,
Link.




Re: Re: beta5 ...

From
Bruce Momjian
Date:
> BTW, is 7.1 going to be a bit slower than 7.0? Or just Beta 5? Just
> curious. Don't mind waiting for 7.2 for the speed-up if necessary.
> 

We expect 7.1 to be faster than 7.0.X.  We may have a small problem that
we may have to address.  Not sure yet.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: beta5 ...

From
The Hermit Hacker
Date:
On Sat, 17 Feb 2001, Lincoln Yeoh wrote:

> At 04:17 PM 2/16/01 -0500, Tom Lane wrote:
> >
> >Vadim says (and I agree) that we really ought to implement a new
> >lightweight lock manager that would fall between spinlocks and regular
> >locks in terms of overhead and functionality.  But it's not reasonable
>
> Will there be an arbitrary user locking feature? E.g. lock on arbitrary
> text string. That would be great :).
>
> BTW, is 7.1 going to be a bit slower than 7.0? Or just Beta 5? Just
> curious. Don't mind waiting for 7.2 for the speed-up if necessary.

It is possible that it will be ... the question is whether the slow down
is unbearable or not, as to whether we'll let it hold things up or not ...

From reading one of Tom's email's, it looks like the changes to 'fix' the
slowdown are drastic/large enough that it might not be safe (or desirable)
to fix it at this late of a stage in beta ...

Depending on what is involved, we might put out a v7.1 for March 1st, so
that ppl can feel confident about using the various features, but have a
v7.1.1 that follows relatively closely on its heels that addresses the
performance problem ...






Re: beta5 ...

From
Tatsuo Ishii
Date:
> Other than that, I have nothing to hold up a beta5.  Anyone else?
> 
>             regards, tom lane

I see a small problem with the regression test. If PL/pgSQL has been
already to template1, the regression scripts will fail because
createlang fails. Probably we should create the regression database
using template0?
--
Tatsuo Ishii


Re: beta5 ...

From
Tom Lane
Date:
Tatsuo Ishii <t-ishii@sra.co.jp> writes:
> Probably we should create the regression database
> using template0?

Seems like a good idea.
        regards, tom lane


Re: Re: beta5 ...

From
Bruce Momjian
Date:
> >
> > BTW, is 7.1 going to be a bit slower than 7.0? Or just Beta 5? Just
> > curious. Don't mind waiting for 7.2 for the speed-up if necessary.
> 
> It is possible that it will be ... the question is whether the slow down
> is unbearable or not, as to whether we'll let it hold things up or not ...
> 
> >From reading one of Tom's email's, it looks like the changes to 'fix' the
> slowdown are drastic/large enough that it might not be safe (or desirable)
> to fix it at this late of a stage in beta ...
> 
> Depending on what is involved, we might put out a v7.1 for March 1st, so
> that ppl can feel confident about using the various features, but have a
> v7.1.1 that follows relatively closely on its heels that addresses the
> performance problem ...

The easy fix is to just set the delay to zero.  Looks like that will fix
most of the problem.  The near-committers thing may indeed be overkill,
and certainly is not worth holding beta.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: Re: beta5 ...

From
The Hermit Hacker
Date:
On Sat, 17 Feb 2001, Bruce Momjian wrote:

> > >
> > > BTW, is 7.1 going to be a bit slower than 7.0? Or just Beta 5? Just
> > > curious. Don't mind waiting for 7.2 for the speed-up if necessary.
> >
> > It is possible that it will be ... the question is whether the slow down
> > is unbearable or not, as to whether we'll let it hold things up or not ...
> >
> > >From reading one of Tom's email's, it looks like the changes to 'fix' the
> > slowdown are drastic/large enough that it might not be safe (or desirable)
> > to fix it at this late of a stage in beta ...
> >
> > Depending on what is involved, we might put out a v7.1 for March 1st, so
> > that ppl can feel confident about using the various features, but have a
> > v7.1.1 that follows relatively closely on its heels that addresses the
> > performance problem ...
>
> The easy fix is to just set the delay to zero.  Looks like that will fix
> most of the problem.

Except that Vadim had a reason for setting it to 5, and I'm loath to see
that changed unless someone actaully understands the ramifications other
then increasing performance ...

> The near-committers thing may indeed be overkill, and certainly is not
> worth holding beta.

What is this 'near-committers thing'??




Re: Re: beta5 ...

From
Tom Lane
Date:
The Hermit Hacker <scrappy@hub.org> writes:
>> The easy fix is to just set the delay to zero.  Looks like that will fix
>> most of the problem.

> Except that Vadim had a reason for setting it to 5,

He claimed to have seen better performance with a nonzero delay.
So far none of the rest of us have been able to duplicate that.
Perhaps he was using a machine where a 5-microsecond select() delay
actually is 5 microseconds?  If so, he's the outlier, not the
rest of us ...
        regards, tom lane


Re: Re: beta5 ...

From
Bruce Momjian
Date:
> > The easy fix is to just set the delay to zero.  Looks like that will fix
> > most of the problem.
> 
> Except that Vadim had a reason for setting it to 5, and I'm loath to see
> that changed unless someone actaully understands the ramifications other
> then increasing performance ...

See post from a few minutes ago with analysis of purpose and actual
affect of Vadim's parameter.  I objected to the delay when it was
introduced because of my analysis, but Vadim's argument is that 5
microseconds is very small delay, just enough to yield the CPU.  We now
see that is much longer than that.

> 
> > The near-committers thing may indeed be overkill, and certainly is not
> > worth holding beta.
> 
> What is this 'near-committers thing'??

Other backends about to commit.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: beta5 ...

From
Tom Lane
Date:
Tatsuo Ishii <t-ishii@sra.co.jp> writes:
> I see a small problem with the regression test. If PL/pgSQL has been
> already to template1, the regression scripts will fail because
> createlang fails. Probably we should create the regression database
> using template0?

Done ...
        regards, tom lane


Re: beta5 ...

From
Tom Lane
Date:
The Hermit Hacker <scrappy@hub.org> writes:
> things appear to have quieted off nicely ... so would like to put out a
> Beta5 for testing ...

Unless Peter E. has some more commits up his sleeve, I think we're
good to go.
        regards, tom lane


Re: beta5 ...

From
Peter Eisentraut
Date:
Tom Lane writes:

> The Hermit Hacker <scrappy@hub.org> writes:
> > things appear to have quieted off nicely ... so would like to put out a
> > Beta5 for testing ...
>
> Unless Peter E. has some more commits up his sleeve, I think we're
> good to go.

Just uploaded freshly baked man pages, including your createdb change.
That'd be it.

-- 
Peter Eisentraut      peter_e@gmx.net       http://yi.org/peter-e/



Re: beta5 ...

From
The Hermit Hacker
Date:
On Sun, 18 Feb 2001, Tom Lane wrote:

> The Hermit Hacker <scrappy@hub.org> writes:
> > things appear to have quieted off nicely ... so would like to put out a
> > Beta5 for testing ...
>
> Unless Peter E. has some more commits up his sleeve, I think we're
> good to go.

okay, I'll put one out Mon aft, just in case of any strays that come up
tonight, or any final commits from our overseas committers ...

thanks ;:)



Re: beta5 ...

From
Peter T Mount
Date:
Quoting The Hermit Hacker <scrappy@hub.org>:

> On Sun, 18 Feb 2001, Tom Lane wrote:
> 
> > The Hermit Hacker <scrappy@hub.org> writes:
> > > things appear to have quieted off nicely ... so would like to put
> out a
> > > Beta5 for testing ...
> >
> > Unless Peter E. has some more commits up his sleeve, I think we're
> > good to go.
> 
> okay, I'll put one out Mon aft, just in case of any strays that come up
> tonight, or any final commits from our overseas committers ...

I'm not planning on doing any until after Beta5 is out ;-)

Peter

> 
> thanks ;:)
> 
> 



-- 
Peter Mount peter@retep.org.uk
PostgreSQL JDBC Driver: http://www.retep.org.uk/postgres/
RetepPDF PDF library for Java: http://www.retep.org.uk/pdf/


RE: beta5 ...

From
Michael Ansley
Date:
<p><font size="2">Would there be any value in setting up a project on sourceforge to make use of their compile farm?  I
knowthat it doesn't cover all platforms, but it would perhaps be a start to mechanical compile and regression
testing.</font><p><fontsize="2">Just a thought...</font><br /><p><font size="2">MikeA</font><br /><br /><p><font
size="2">-----OriginalMessage-----</font><br /><font size="2">From: Tom Lane [<a
href="mailto:tgl@sss.pgh.pa.us">mailto:tgl@sss.pgh.pa.us</a>]</font><br/><font size="2">Sent: 20 February 2001
15:51</font><br/><font size="2">To: Justin Clift</font><br /><font size="2">Cc: pgsql-hackers@postgresql.org</font><br
/><fontsize="2">Subject: Re: [HACKERS] beta5 ... </font><br /><p><font size="2">>> As a matter of curiosity, is
eachbeta compiled and then regression</font><br /><font size="2">>> tested against *every* one of the known
"supported"platforms before</font><br /><font size="2">>> release?</font><p><font size="2">Who are you expecting
todo that, exactly?</font><p><font size="2">One of the differences between Postgres and a proprietary commercial
</font><br/><font size="2">database is that there is no vast support machinery behind the scenes.</font><br /><font
size="2">Whatyou see going on on this list is what you get: beta testing</font><br /><font size="2">consists of the
activitiesperformed and reported by list members.</font><p><font size="2">Normally, if we are about to push out a beta
thentwo or three people</font><br /><font size="2">will double-check that the current CVS tip builds and passes
regression</font><br/><font size="2">on their personal machines.  But the "supported platforms" coverage</font><br
/><fontsize="2">depicted in the docs consists of all the platforms that are reported to</font><br /><font size="2">us
asworking during the entire beta test period, including many that</font><br /><font size="2">the key developers have no
directaccess to.  There's no way that we</font><br /><font size="2">could reverse the process and cause that to happen
beforea beta release</font><br /><font size="2">instead of after; certainly no way that we could cause all that
effort</font><br/><font size="2">to be repeated for each beta version.</font><p><font size="2">If you are using a beta
versionthen you are part of that testing</font><br /><font size="2">process, not a beneficiary of something that's
happenedbehind closed</font><br /><font size="2">doors.</font><p>                        <font size="2">regards, tom
lane</font><code><fontsize="3"><br /><br /> **********************************************************************<br
/>This email and any files transmitted with it are confidential and<br /> intended solely for the use of the individual
orentity to whom they<br /> are addressed. If you have received this email in error please notify<br /> Nick West -
GlobalInfrastructure Manager.<br /><br /> This footnote also confirms that this email message has been swept by<br />
MIMEsweeperfor the presence of computer viruses.<br /><br /> www.mimesweeper.com<br />
**********************************************************************<br/></font></code> 

Re: beta5 ...

From
Thomas Lockhart
Date:
> Would there be any value in setting up a project on sourceforge to
> make use of their compile farm?  I know that it doesn't cover all
> platforms, but it would perhaps be a start to mechanical compile and
> regression testing.

I haven't looked at the platforms available in the compile farm
recently, but afaik regression coverage for over half a dozen platforms
already happens without (extra) effort: Tom Lane has three or more
platforms, I've got Linux, Bruce has BSDI, Marc has FreeBSD, we have
some active W32 developers, etc etc.

What would SF add to this mix?
                          - Thomas


Re: Re: beta5 ...

From
Larry Rosenman
Date:
* Thomas Lockhart <lockhart@alumni.caltech.edu> [010220 10:51]:
> > Would there be any value in setting up a project on sourceforge to
> > make use of their compile farm?  I know that it doesn't cover all
> > platforms, but it would perhaps be a start to mechanical compile and
> > regression testing.
> 
> I haven't looked at the platforms available in the compile farm
> recently, but afaik regression coverage for over half a dozen platforms
> already happens without (extra) effort: Tom Lane has three or more
> platforms, I've got Linux, Bruce has BSDI, Marc has FreeBSD, we have
> some active W32 developers, etc etc.
I have a UnixWare 7.1.1 box I run PG on....
> 
> What would SF add to this mix?
> 
>                            - Thomas
-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 972-414-9812                 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749


Re: Re: beta5 ...

From
Tom Lane
Date:
>> Would there be any value in setting up a project on sourceforge to
>> make use of their compile farm?  I know that it doesn't cover all
>> platforms, but it would perhaps be a start to mechanical compile and
>> regression testing.

> I haven't looked at the platforms available in the compile farm
> recently, but afaik regression coverage for over half a dozen platforms
> already happens without (extra) effort: Tom Lane has three or more
> platforms, I've got Linux, Bruce has BSDI, Marc has FreeBSD, we have
> some active W32 developers, etc etc.

I run HPUX and Linux/PPC routinely, so that's only two here.  Still, we
have reasonable coverage among the core team and a bunch more platforms
used by active pgsql-hackers people.  Also, the project does have an
Alpha in-house at hub.org (if Marc ever gets it back into commission
after that failed OS reinstall...)

> What would SF add to this mix?

The current list of machines at cf.sourceforge.net seems to be
                lqqqqqqqChoose compile farm server...qqqqqqqk                x A. [x86] Linux 2.2 (Debian 2.2)
x                x C. [x86] FreeBSD (4.2-stable)             x                x
 x                x G. [Alpha] Compaq Tru64 (5.1)             x                x H. [Alpha] Linux 2.2 (RedHat 7.0)
  x                x                                           x                x L. [Sparc - E240] Linux 2.2 (Debian
2.2) x                x M. [Sparc - E240] Sun Solaris (8)         x                x
      x                x Exit                                      x
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj

I think I'll go try a build on that Solaris 8 machine, since we've heard
some reports of problems on Solaris.  However, I'm not sure that we need
any organized use of their compilefarm.  If they made it easy to
*automatically* run build/install/regress test on multiple machines,
I could see the facility being useful (especially so once a few more
platforms are offered).  But right now it looks like it's just shell
access to platforms other than your own, which is not going to help us
all that much.
        regards, tom lane


Re: Re: beta5 ...

From
Justin Clift
Date:
As well as Linux I run Solaris 8 SPARC (32-bit not 64), Solaris 7 SPARC
(SMP, 32-bit not 64), Solaris 7 Intel (both SMP and uni-processor) and
Solaris 8 Intel (both SMP and uni-processor).

I can be counted on to do testing of these as required in about 2 weeks
from now, after I get a new permanent connection here.

With luck I'll additionally have the finances to buy some SPARC 64-bit
machines in a few months.

Regards and best wishes,

Justin Clift
Database Administrator

Tom Lane wrote:
> > What would SF add to this mix?
> 
> The current list of machines at cf.sourceforge.net seems to be
> 
>                  lqqqqqqqChoose compile farm server...qqqqqqqk
>                  x A. [x86] Linux 2.2 (Debian 2.2)           x
>                  x C. [x86] FreeBSD (4.2-stable)             x
>                  x                                           x
>                  x G. [Alpha] Compaq Tru64 (5.1)             x
>                  x H. [Alpha] Linux 2.2 (RedHat 7.0)         x
>                  x                                           x
>                  x L. [Sparc - E240] Linux 2.2 (Debian 2.2)  x
>                  x M. [Sparc - E240] Sun Solaris (8)         x
>                  x                                           x
>                  x Exit                                      x
>                  mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj
> 
> I think I'll go try a build on that Solaris 8 machine, since we've heard
> some reports of problems on Solaris.  However, I'm not sure that we need
> any organized use of their compilefarm.  If they made it easy to
> *automatically* run build/install/regress test on multiple machines,
> I could see the facility being useful (especially so once a few more
> platforms are offered).  But right now it looks like it's just shell
> access to platforms other than your own, which is not going to help us
> all that much.
> 
>                         regards, tom lane


RE: beta5 ...

From
Matthew
Date:
I think UP or SMP should be an option to check, perhaps just a box for the
number of processors.  Also something to capture the compile flags.  I have
a dual Ppro, and it compiles fine unless I use the -j3 or -j4 commands,
then I get an error.

Matt

> -----Original Message-----
> From:    Vince Vielhaber [SMTP:vev@michvhf.com]
> Sent:    Thursday, February 22, 2001 10:57 AM
> To:    Pete Forman
> Cc:    pgsql-hackers@postgresql.org
> Subject:    RE: [HACKERS] beta5 ...
> 
> On Thu, 22 Feb 2001, Pete Forman wrote:
> 
> > Vince Vielhaber writes:
> >  > On Thu, 22 Feb 2001, Christopher Kings-Lynne wrote:
> >  >
> >  > > What about adding a field where they paste the output of 'uname
> >  > > -a' on their system...?
> >  >
> >  > Got this and Justin's changes along with compiler version.  Anyone
> >  > think of anything else?
> >
> > Architecture.  IRIX, Solaris and AIX allow applications and libraries
> > to be built 32 or 64 bit.
> 
> Added.
> 
> > You may also like to add a field for configure options used.  Or is
> > this just for results OOTB?
> 
> That comes later.  This part is just for identifying the system itself.
> 
> Vince.
> -- 
> ==========================================================================
> Vince Vielhaber -- KA8CSH    email: vev@michvhf.com    http://www.pop4.net
>  128K ISDN from $22.00/mo - 56K Dialup from $16.00/mo at Pop4 Networking
>         Online Campground Directory    http://www.camping-usa.com
>        Online Giftshop Superstore    http://www.cloudninegifts.com
> ==========================================================================
> 
> 


RE: beta5 ...

From
Peter Eisentraut
Date:
Matthew writes:

> I think UP or SMP should be an option to check, perhaps just a box for the
> number of processors.  Also something to capture the compile flags.  I have
> a dual Ppro, and it compiles fine unless I use the -j3 or -j4 commands,
> then I get an error.

Which error?

Parallel make doesn't work when you build from a CVS tree, but it should
work with a distribution tarball.

-- 
Peter Eisentraut      peter_e@gmx.net       http://yi.org/peter-e/