Thread: AW: Re: GiST for 7.1 !!
> > > we've almost totally rewrite gist.c because old code and algorithm > > > were not suitable for variable size keys. I think it might be > > > submitted into 7.1 beta source tree. > > > > Urgh. Dropping in a total rewrite when we're already past > beta3 doesn't > > strike me as good project management practice --- especially if the > > rewrite was done to add features (ie variable-size keys) not merely > > fix bugs. I think it might be more prudent to hold this for 7.2. > > OK. If our changes will not go to 7.1, is't possible to create > feature archive and announce it somewhere. It would be nice if > people could test it. Anyway, I'll create web page with all > docs and patches. I'm afraid one more year to 7.2 is enough for > GiST to die :-) I think featureism is the the most prominent argument for PostgreSQL. Thus standing before a decision to eighter fix GiST bugs and risc a new bug (limited to GiST) because of an added feature or shipping a known broken GiST, my vote would definitely be to add Oleg's patch. Andreas
On Thu, 11 Jan 2001, Zeugswetter Andreas SB wrote:
>
> > > > we've almost totally rewrite gist.c because old code and algorithm
> > > > were not suitable for variable size keys. I think it might be
> > > > submitted into 7.1 beta source tree.
> > >
> > > Urgh. Dropping in a total rewrite when we're already past
> > beta3 doesn't
> > > strike me as good project management practice --- especially if the
> > > rewrite was done to add features (ie variable-size keys) not merely
> > > fix bugs. I think it might be more prudent to hold this for 7.2.
> >
> > OK. If our changes will not go to 7.1, is't possible to create
> > feature archive and announce it somewhere. It would be nice if
> > people could test it. Anyway, I'll create web page with all
> > docs and patches. I'm afraid one more year to 7.2 is enough for
> > GiST to die :-)
>
> I think featureism is the the most prominent argument for PostgreSQL.
> Thus standing before a decision to eighter fix GiST bugs and risc a new
> bug (limited to GiST) because of an added feature or shipping a known
> broken GiST, my vote would definitely be to add Oleg's patch.
Definetely, our changes limited to GiST insert algorithm only.
Other changes are bugfixes. I encourage people interested in GiST
to test my submission. Our implementation of RD-Tree which we used
to support of indexing of int4 arrays will works only with our
version of gist.c (actually our interest to GiST was motivated by
index support of int4 arrays).
Regards,
Oleg
>
> Andreas
>
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Oleg Bartunov wrote: > > On Thu, 11 Jan 2001, Zeugswetter Andreas SB wrote: > > > I think featureism is the the most prominent argument for PostgreSQL. Exactly! Altough it has already lost much of it ;( > > Thus standing before a decision to eighter fix GiST bugs and risc a new > > bug (limited to GiST) because of an added feature or shipping a known > > broken GiST, my vote would definitely be to add Oleg's patch. That's my vote too, specially if there will be some regression tests accompanying the patches. The current (pre-patch) state of affairs with GiST could probably be described as security-by-obscurity anyhow i.e. "we have't tried it so we think it probably works" ;-) Also I suspect there are still only a few users, most of them capable of fixing inside gist.c if something nasti turns up. They will be much more motivated to do so if it is in "official" sources and not in the sources from postgresql-gist.org or somesuch ;) > Definetely, our changes limited to GiST insert algorithm only. > Other changes are bugfixes. So applying _only_ bugfixes may also not be an option as they are not well tested without the changed gist.c ----------------- Hannu
Hannu Krosing <hannu@tm.ee> writes:
> That's my vote too, specially if there will be some regression tests
> accompanying the patches. The current (pre-patch) state of affairs with
> GiST could probably be described as security-by-obscurity anyhow i.e.
> "we have't tried it so we think it probably works" ;-)
Au contraire, there *are* a few users of GiST out there now, Gene Selkov
to name one. So there is a definite risk of breaking things that worked
in 7.0 and before, in the name of adding new features.
If I thought that we had adequate ability to test the new GiST
implementation during the remaining beta period, I wouldn't be
so worried. But at this point, Oleg's changes could not appear
in the beta series before beta4, and between the late date, the
lack of regression test, and the few interested people to test it,
I doubt that we'll get any useful coverage.
I would recommend that Oleg do like Ryan K. did for awhile with the
Alpha patches: make them available as a set of diffs to be applied
to the official distribution. We'll be happy to merge them in for
7.2, but the calendar says it's too late for 7.1.
regards, tom lane
Tom Lane wrote: > > Hannu Krosing <hannu@tm.ee> writes: > > That's my vote too, specially if there will be some regression tests > > accompanying the patches. The current (pre-patch) state of affairs with > > GiST could probably be described as security-by-obscurity anyhow i.e. > > "we have't tried it so we think it probably works" ;-) > > Au contraire, there *are* a few users of GiST out there now, Gene Selkov > to name one. Yes, he is the only one (except Oleg) whom I know to use it too ;) > So there is a definite risk of breaking things that worked > in 7.0 and before, in the name of adding new features. True. Could we ask Gene to test 7.1 with Oleg's patches ? > If I thought that we had adequate ability to test the new GiST > implementation during the remaining beta period, I wouldn't be > so worried. But at this point, Oleg's changes could not appear > in the beta series before beta4, and between the late date, the > lack of regression test, and the few interested people to test it, > I doubt that we'll get any useful coverage. Or if in fact there _are_ only a few people using it now we could get _all_ the coverage to be sufficiently sure we don't break anyones code. GiST being such an obscure and underused feature I'm pretty sure that most (all?) active users are on Hackers list and read everything that has GiST in subject. > I would recommend that Oleg do like Ryan K. did for awhile with the > Alpha patches: make them available as a set of diffs to be applied > to the official distribution. We'll be happy to merge them in for > 7.2, but the calendar says it's too late for 7.1. Even for the _real_ bugfixes in gist.c ? ---------------- Hannu
Hannu Krosing <hannu@tm.ee> writes:
>> ... the calendar says it's too late for 7.1.
> Even for the _real_ bugfixes in gist.c ?
If he were submitting only bugfixes, we wouldn't be having this
discussion.
Look, I don't like postponing improvements either. But if we don't
adhere to project management discipline, we are never going to get
releases out the door at all --- or if we do, they'll be too buggy
to be reliable. It's not like "no new features during beta" is such
a draconian or difficult-to-understand rule.
The RelFileNodeEquals() bug we found on Monday proves that no one had
yet done enough stress-testing on 7.1 to discover that multiple
databases were broken. Think about that for awhile before you campaign
for inserting untested new features at this point. We need to focus on
TESTING, people, not new features.
regards, tom lane
> The RelFileNodeEquals() bug we found on Monday proves that no one had > yet done enough stress-testing on 7.1 to discover that multiple > databases were broken. Think about that for awhile before > you campaign for inserting untested new features at this point. > We need to focus on TESTING, people, not new features. I mostly sure that Oleg' changes touch *only* gist subdir (Oleg?) so *nothing* will be broken in other areas. That's why I don't object new gist in 7.1. RelFileNodeEquals is quite another thing, thanks for fix again -:) Vadim
Tom Lane wrote: > > Hannu Krosing <hannu@tm.ee> writes: > >> ... the calendar says it's too late for 7.1. > > > Even for the _real_ bugfixes in gist.c ? > > If he were submitting only bugfixes, we wouldn't be having this > discussion. But he had very little incentive to fix bugs in the version he would not use. > Look, I don't like postponing improvements either. But if we don't > adhere to project management discipline, But should we do that _blindly_? I'd think that improving/fixing things in seldom-visited corners of postgres should be a little more tolerable than messing around in core. > we are never going to get > releases out the door at all --- or if we do, they'll be too buggy > to be reliable. It's not like "no new features during beta" is such > a draconian or difficult-to-understand rule. I'd rather describe his changes as "a (bug)fix that required a major rewrite" ;) > The RelFileNodeEquals() bug we found on Monday proves that no one had > yet done enough stress-testing on 7.1 to discover that multiple > databases were broken. BTW, What do people use for stress-testing ? > Think about that for awhile before you campaign for inserting untested > new features at this point. Rather new variants of little-tested features ;) > We need to focus on TESTING, people, not new features. I make a personal promise to spend at least 5 hours of testing new GiST functionality during this weekend if it is commited to 7.1 CVS. (ok, I do it anyhow, just that currently I'm testing it using the patches ;) ------------------------- Hannu
On Thu, 11 Jan 2001, Mikheev, Vadim wrote: > > The RelFileNodeEquals() bug we found on Monday proves that no one had > > yet done enough stress-testing on 7.1 to discover that multiple > > databases were broken. Think about that for awhile before > > you campaign for inserting untested new features at this point. > > We need to focus on TESTING, people, not new features. > > I mostly sure that Oleg' changes touch *only* gist subdir (Oleg?) Yes, and only one file - gist.c > so *nothing* will be broken in other areas. That's why I don't > object new gist in 7.1. > We prepare regression test for RD-Tree in the same way as Gene does for his contribution. I put all files on http://www.sai.msu.su/~megera/postgres/gist/. btw, all Gene's test for seg and cube in contrib area are passed. It would be better Gene check his application himself. I'm sorry for trouble with my submission - I hoped we will be ready before beta2,3, but we spent too many time to get old insertion algoritm works with variable size keys until we realized it's just not suitable for this. I understand Tom's arguments and respect his experience, so I think it's possible to put link to my page in 7.1 docs for people interested in GiST features. Also, we found GiST part of postgres documentation is too short, so we'll try to contribute something sometime later. From other side, GiST was too hidden for people, while it's very powerfull feature and many people for sure really needs GiST power. Frankly speaking I discovered GiST power myself by accident :-) Now we have many plans to use GiST in our real life applications such as Web site management system, full text search (killer application !), data mining and others. There are several improvements and new features we plan to add to GiST which could be go to 7.2. Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
On Thu, 11 Jan 2001, Hannu Krosing wrote: > I make a personal promise to spend at least 5 hours of testing new GiST > functionality during this weekend if it is commited to 7.1 CVS. > (ok, I do it anyhow, just that currently I'm testing it using the > patches ;) Hanny, latest version is available at http://www.sai.msu.su/~megera/postgres/gist/ nothing changed in code (in compare with my submission), just added some info and regression test. Let me know if you need some help. Oleg > > ------------------------- > Hannu > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
Oleg Bartunov <oleg@sai.msu.su> writes:
> I understand Tom's arguments and respect his experience, so I think it's
> possible to put link to my page in 7.1 docs for people interested in
> GiST features.
Bear in mind that I only have one core vote ;-). We've already had some
private core discussion about whether to accept this patch now, and so
far I think I'm outvoted.
Did I understand you to say that you'd added some regression tests for
GiST? That would lessen my unhappiness a little bit ...
regards, tom lane
On Thu, 11 Jan 2001, Tom Lane wrote: > Oleg Bartunov <oleg@sai.msu.su> writes: > > I understand Tom's arguments and respect his experience, so I think it's > > possible to put link to my page in 7.1 docs for people interested in > > GiST features. > > Bear in mind that I only have one core vote ;-). We've already had some > private core discussion about whether to accept this patch now, and so > far I think I'm outvoted. > There are several Tom Lane, judge by your activity. You probably need several votes. > Did I understand you to say that you'd added some regression tests for > GiST? That would lessen my unhappiness a little bit ... Yes, we did. Currently all files are available from my page http://www.sai.msu.su/~megera/postgres/gist/ I could submit them to hackers list if CORE people got consensus Regards, Oleg > > regards, tom lane > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
On Fri, 12 Jan 2001, Oleg Bartunov wrote: > On Thu, 11 Jan 2001, Tom Lane wrote: > > > Oleg Bartunov <oleg@sai.msu.su> writes: > > > I understand Tom's arguments and respect his experience, so I think it's > > > possible to put link to my page in 7.1 docs for people interested in > > > GiST features. > > > > Bear in mind that I only have one core vote ;-). We've already had some > > private core discussion about whether to accept this patch now, and so > > far I think I'm outvoted. > > > > There are several Tom Lane, judge by your activity. You probably need > several votes. > > > Did I understand you to say that you'd added some regression tests for > > GiST? That would lessen my unhappiness a little bit ... > > Yes, we did. Currently all files are available from my page > http://www.sai.msu.su/~megera/postgres/gist/ > I could submit them to hackers list if CORE people got consensus Okay, if there are appropriate regression tests, I'm going to say go for it ... Does anyone have any objections to my downloading the tar file (doing that now), committing the changes and wrapping up a quick Beta4 just so that we have a tar ball that is testable right away? Save Lamar and the other packagers a bit of work by avoiding beta3 packages :)
just downloaded it and can't find any regression tests ... ? On Thu, 11 Jan 2001, The Hermit Hacker wrote: > On Fri, 12 Jan 2001, Oleg Bartunov wrote: > > > On Thu, 11 Jan 2001, Tom Lane wrote: > > > > > Oleg Bartunov <oleg@sai.msu.su> writes: > > > > I understand Tom's arguments and respect his experience, so I think it's > > > > possible to put link to my page in 7.1 docs for people interested in > > > > GiST features. > > > > > > Bear in mind that I only have one core vote ;-). We've already had some > > > private core discussion about whether to accept this patch now, and so > > > far I think I'm outvoted. > > > > > > > There are several Tom Lane, judge by your activity. You probably need > > several votes. > > > > > Did I understand you to say that you'd added some regression tests for > > > GiST? That would lessen my unhappiness a little bit ... > > > > Yes, we did. Currently all files are available from my page > > http://www.sai.msu.su/~megera/postgres/gist/ > > I could submit them to hackers list if CORE people got consensus > > Okay, if there are appropriate regression tests, I'm going to say go for > it ... > > Does anyone have any objections to my downloading the tar file (doing that > now), committing the changes and wrapping up a quick Beta4 just so that we > have a tar ball that is testable right away? > > Save Lamar and the other packagers a bit of work by avoiding beta3 > packages :) > > > > Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy Systems Administrator @ hub.org primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
The Hermit Hacker wrote: > Save Lamar and the other packagers a bit of work by avoiding beta3 > packages :) :-) Well, working on those packages now. Fun stuff. I learn more about more different parts of the code each new version, as the hang-ups change from version to version. Although, thanks to Karl DeBisschop, I now have a new secret weapon in my war Bwahaha... :-) There have been more changes since 7.0 than in any previous version I can remember as far as the build goes. Not a bad thing -- just a learning experience. At least the build won't change from beta3 to beta4..... -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
On Thu, 11 Jan 2001, The Hermit Hacker wrote: > > just downloaded it and can't find any regression tests ... ? it's in the contrib-intarray.tar.gz gmake, gmake install, gmake installcheck Oleg > > On Thu, 11 Jan 2001, The Hermit Hacker wrote: > > > On Fri, 12 Jan 2001, Oleg Bartunov wrote: > > > > > On Thu, 11 Jan 2001, Tom Lane wrote: > > > > > > > Oleg Bartunov <oleg@sai.msu.su> writes: > > > > > I understand Tom's arguments and respect his experience, so I think it's > > > > > possible to put link to my page in 7.1 docs for people interested in > > > > > GiST features. > > > > > > > > Bear in mind that I only have one core vote ;-). We've already had some > > > > private core discussion about whether to accept this patch now, and so > > > > far I think I'm outvoted. > > > > > > > > > > There are several Tom Lane, judge by your activity. You probably need > > > several votes. > > > > > > > Did I understand you to say that you'd added some regression tests for > > > > GiST? That would lessen my unhappiness a little bit ... > > > > > > Yes, we did. Currently all files are available from my page > > > http://www.sai.msu.su/~megera/postgres/gist/ > > > I could submit them to hackers list if CORE people got consensus > > > > Okay, if there are appropriate regression tests, I'm going to say go for > > it ... > > > > Does anyone have any objections to my downloading the tar file (doing that > > now), committing the changes and wrapping up a quick Beta4 just so that we > > have a tar ball that is testable right away? > > > > Save Lamar and the other packagers a bit of work by avoiding beta3 > > packages :) > > > > > > > > > > Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy > Systems Administrator @ hub.org > primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
On Fri, 12 Jan 2001, Oleg Bartunov wrote: > On Thu, 11 Jan 2001, The Hermit Hacker wrote: > > > > > just downloaded it and can't find any regression tests ... ? > > it's in the contrib-intarray.tar.gz > gmake, gmake install, gmake installcheck erk, can we get this somehow done in such a way that its part of the *standard* regression tests? so when ppl do 'make test', the GiST stuff is checked also? My worry, as with others, isn't that GiST itself is broken by the changes, its that *somehow* there is an interaction that is with the rest of the system that isn't being tested ... > > > Oleg > > > > On Thu, 11 Jan 2001, The Hermit Hacker wrote: > > > > > On Fri, 12 Jan 2001, Oleg Bartunov wrote: > > > > > > > On Thu, 11 Jan 2001, Tom Lane wrote: > > > > > > > > > Oleg Bartunov <oleg@sai.msu.su> writes: > > > > > > I understand Tom's arguments and respect his experience, so I think it's > > > > > > possible to put link to my page in 7.1 docs for people interested in > > > > > > GiST features. > > > > > > > > > > Bear in mind that I only have one core vote ;-). We've already had some > > > > > private core discussion about whether to accept this patch now, and so > > > > > far I think I'm outvoted. > > > > > > > > > > > > > There are several Tom Lane, judge by your activity. You probably need > > > > several votes. > > > > > > > > > Did I understand you to say that you'd added some regression tests for > > > > > GiST? That would lessen my unhappiness a little bit ... > > > > > > > > Yes, we did. Currently all files are available from my page > > > > http://www.sai.msu.su/~megera/postgres/gist/ > > > > I could submit them to hackers list if CORE people got consensus > > > > > > Okay, if there are appropriate regression tests, I'm going to say go for > > > it ... > > > > > > Does anyone have any objections to my downloading the tar file (doing that > > > now), committing the changes and wrapping up a quick Beta4 just so that we > > > have a tar ball that is testable right away? > > > > > > Save Lamar and the other packagers a bit of work by avoiding beta3 > > > packages :) > > > > > > > > > > > > > > > > Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy > > Systems Administrator @ hub.org > > primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org > > > > Regards, > Oleg > _____________________________________________________________ > Oleg Bartunov, sci.researcher, hostmaster of AstroNet, > Sternberg Astronomical Institute, Moscow University (Russia) > Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ > phone: +007(095)939-16-83, +007(095)939-23-83 > > Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy Systems Administrator @ hub.org primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
On Thu, 11 Jan 2001, The Hermit Hacker wrote: > On Fri, 12 Jan 2001, Oleg Bartunov wrote: > > > On Thu, 11 Jan 2001, The Hermit Hacker wrote: > > > > > > > > just downloaded it and can't find any regression tests ... ? > > > > it's in the contrib-intarray.tar.gz > > gmake, gmake install, gmake installcheck > > erk, can we get this somehow done in such a way that its part of the > *standard* regression tests? so when ppl do 'make test', the GiST stuff > is checked also? My worry, as with others, isn't that GiST itself is > broken by the changes, its that *somehow* there is an interaction that is > with the rest of the system that isn't being tested ... No way, we need to load functions. there are several contributions which depends on loaded functions. If you suggest how to do this in general way, it would fine. To test GiST you need to define some data structure ( in our case - RD-tree) and functions to access it > > > > > > > > > Oleg > > > > > > On Thu, 11 Jan 2001, The Hermit Hacker wrote: > > > > > > > On Fri, 12 Jan 2001, Oleg Bartunov wrote: > > > > > > > > > On Thu, 11 Jan 2001, Tom Lane wrote: > > > > > > > > > > > Oleg Bartunov <oleg@sai.msu.su> writes: > > > > > > > I understand Tom's arguments and respect his experience, so I think it's > > > > > > > possible to put link to my page in 7.1 docs for people interested in > > > > > > > GiST features. > > > > > > > > > > > > Bear in mind that I only have one core vote ;-). We've already had some > > > > > > private core discussion about whether to accept this patch now, and so > > > > > > far I think I'm outvoted. > > > > > > > > > > > > > > > > There are several Tom Lane, judge by your activity. You probably need > > > > > several votes. > > > > > > > > > > > Did I understand you to say that you'd added some regression tests for > > > > > > GiST? That would lessen my unhappiness a little bit ... > > > > > > > > > > Yes, we did. Currently all files are available from my page > > > > > http://www.sai.msu.su/~megera/postgres/gist/ > > > > > I could submit them to hackers list if CORE people got consensus > > > > > > > > Okay, if there are appropriate regression tests, I'm going to say go for > > > > it ... > > > > > > > > Does anyone have any objections to my downloading the tar file (doing that > > > > now), committing the changes and wrapping up a quick Beta4 just so that we > > > > have a tar ball that is testable right away? > > > > > > > > Save Lamar and the other packagers a bit of work by avoiding beta3 > > > > packages :) > > > > > > > > > > > > > > > > > > > > > > Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy > > > Systems Administrator @ hub.org > > > primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org > > > > > > > Regards, > > Oleg > > _____________________________________________________________ > > Oleg Bartunov, sci.researcher, hostmaster of AstroNet, > > Sternberg Astronomical Institute, Moscow University (Russia) > > Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ > > phone: +007(095)939-16-83, +007(095)939-23-83 > > > > > > Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy > Systems Administrator @ hub.org > primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
The Hermit Hacker <scrappy@hub.org> writes:
> Does anyone have any objections to my downloading the tar file (doing that
> now), committing the changes and wrapping up a quick Beta4 just so that we
> have a tar ball that is testable right away?
I think we oughta review the changes at least a little bit before
pushing out a beta4... go ahead and commit though...
regards, tom lane
Lamar Owen <lamar.owen@wgcr.org> writes:
> There have been more changes since 7.0 than in any previous version I
> can remember as far as the build goes.
No surprise, considering the amount of work Peter E. has done on
cleaning up the configure, build, and install process --- exactly
the stuff that would affect RPM building. I trust you're finding
that the changes are for the better.
Unless Peter has plans he hasn't mentioned, future revs shouldn't
see so much activity there.
regards, tom lane
> > erk, can we get this somehow done in such a way that its part of the > > *standard* regression tests? so when ppl do 'make test', > > the GiST stuff is checked also? My worry, as with others, isn't that > > GiST itself is broken by the changes, its that *somehow* there is an > > interaction that is with the rest of the system that isn't being tested ... > > No way, we need to load functions. there are several contributions > which depends on loaded functions. If you suggest how to do this > in general way, it would fine. To test GiST you need to define some > data structure ( in our case - RD-tree) and functions to access it Look at regress/input/create_function_1.source for hints from SPI tests... Vadim
On Thu, 11 Jan 2001, Mikheev, Vadim wrote: > > > erk, can we get this somehow done in such a way that its part of the > > > *standard* regression tests? so when ppl do 'make test', > > > the GiST stuff is checked also? My worry, as with others, isn't that > > > GiST itself is broken by the changes, its that *somehow* there is an > > > interaction that is with the rest of the system that isn't being tested > ... > > > > No way, we need to load functions. there are several contributions > > which depends on loaded functions. If you suggest how to do this > > in general way, it would fine. To test GiST you need to define some > > data structure ( in our case - RD-tree) and functions to access it > > Look at regress/input/create_function_1.source for hints from > SPI tests... > Thanks Vadim for tips. Will do this way, but tommorow. It's 3:19 am already and I have to sleep :-) > Vadim > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
Oleg Bartunov <oleg@sai.msu.su> writes:
>>>> No way, we need to load functions. there are several contributions
>>>> which depends on loaded functions. If you suggest how to do this
>>>> in general way, it would fine. To test GiST you need to define some
>>>> data structure ( in our case - RD-tree) and functions to access it
>>
>> Look at regress/input/create_function_1.source for hints from
>> SPI tests...
Um, you do realize that a contrib module that gets used as part of the
regress tests may as well be mainstream? At least in terms of the
portability requirements it will have to meet?
I'm unhappy again. Bad enough we accepted a new feature during beta;
now we're going to expect an absolutely virgin contrib module to work
everywhere in order to pass regress tests?
regards, tom lane
> Um, you do realize that a contrib module that gets used as part of the > regress tests may as well be mainstream? At least in terms of the > portability requirements it will have to meet? > > I'm unhappy again. Bad enough we accepted a new feature during beta; > now we're going to expect an absolutely virgin contrib module to work > everywhere in order to pass regress tests? Ops, agreed. And I fear that in current code there is no one GiST index implementation -:( Should we worry about regress tests? -:) Vadim
Tom Lane wrote: > Um, you do realize that a contrib module that gets used as part of the > regress tests may as well be mainstream? At least in terms of the > portability requirements it will have to meet? > I'm unhappy again. Bad enough we accepted a new feature during beta; > now we're going to expect an absolutely virgin contrib module to work > everywhere in order to pass regress tests? Last I checked, two contrib modules had to be built for regression testing. But that was 7.0. (autoinc and refint.....). -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Lamar Owen <lamar.owen@wgcr.org> writes:
>> I'm unhappy again. Bad enough we accepted a new feature during beta;
>> now we're going to expect an absolutely virgin contrib module to work
>> everywhere in order to pass regress tests?
> Last I checked, two contrib modules had to be built for regression
> testing.
Sure, but they've been there awhile. All of my concerns here are
schedule-driven: do we really want to be wringing out a new contrib
module, to the point where it will run everywhere, before we can
release 7.1?
regards, tom lane
Tom Lane wrote: > Lamar Owen <lamar.owen@wgcr.org> writes: > >> I'm unhappy again. Bad enough we accepted a new feature during beta; > >> now we're going to expect an absolutely virgin contrib module to work > >> everywhere in order to pass regress tests? > > Last I checked, two contrib modules had to be built for regression > > testing. > Sure, but they've been there awhile. All of my concerns here are > schedule-driven: do we really want to be wringing out a new contrib > module, to the point where it will run everywhere, before we can > release 7.1? Are the benefits worth the effort? Can the current GiST developers pull it off in time? If the answer to either question is not a resounding YES then we really don't need to go down this road. Either leave it in contrib and regression testless (with a test script in the contrib), or make it a feature patch. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
On Thu, 11 Jan 2001, Tom Lane wrote: > Lamar Owen <lamar.owen@wgcr.org> writes: > >> I'm unhappy again. Bad enough we accepted a new feature during beta; > >> now we're going to expect an absolutely virgin contrib module to work > >> everywhere in order to pass regress tests? > > > Last I checked, two contrib modules had to be built for regression > > testing. > > Sure, but they've been there awhile. All of my concerns here are > schedule-driven: do we really want to be wringing out a new contrib > module, to the point where it will run everywhere, before we can > release 7.1? Hrmmm ... just a thought here, but how about a potential 'interactive' regression test, where it asks if you want to run regress on GiST? If so, do it, if not, ignore it ... ?
On Thu, 11 Jan 2001, Mikheev, Vadim wrote: > > Um, you do realize that a contrib module that gets used as part of the > > regress tests may as well be mainstream? At least in terms of the > > portability requirements it will have to meet? > > > > I'm unhappy again. Bad enough we accepted a new feature during beta; > > now we're going to expect an absolutely virgin contrib module to work > > everywhere in order to pass regress tests? > > Ops, agreed. > And I fear that in current code there is no one GiST index > implementation -:( Should we worry about regress tests? -:) Yes, we had to write contrib module even to test GiST. People, I'm really confused after reading all of messages. GiST is just an interface and to test any interface you need 2 sides. In current code there is only one side. old GiST code live untested for years. What's the problem ? It's the problem of current regression test, mostly. Ok. We could rewrite R-Tree to use GiST and make regression test which will not make people nervous. But this certainly not for 7.1 and most probable without us. Author of R-Tree could write this easily. I read Bruce's interview and was really relaxed - how everything is going well. Bruce, we need your opinion. Oleg > > Vadim > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
OK. We found an old implementation of R-Tre using GiST (Pg95)
and we'll try to implement regression test using R-Tree
it's anyway will be a good test.
Regards,
Oleg
On Fri, 12 Jan 2001, Oleg Bartunov wrote:
> On Thu, 11 Jan 2001, Mikheev, Vadim wrote:
>
> > > Um, you do realize that a contrib module that gets used as part of the
> > > regress tests may as well be mainstream? At least in terms of the
> > > portability requirements it will have to meet?
> > >
> > > I'm unhappy again. Bad enough we accepted a new feature during beta;
> > > now we're going to expect an absolutely virgin contrib module to work
> > > everywhere in order to pass regress tests?
> >
> > Ops, agreed.
> > And I fear that in current code there is no one GiST index
> > implementation -:( Should we worry about regress tests? -:)
>
>
> Yes, we had to write contrib module even to test GiST. People,
> I'm really confused after reading all of messages.
> GiST is just an interface and to test any interface you need 2 sides.
> In current code there is only one side. old GiST code live
> untested for years. What's the problem ? It's the problem of
> current regression test, mostly.
> Ok. We could rewrite R-Tree to use GiST and make regression test which
> will not make people nervous. But this certainly not for 7.1 and most
> probable without us. Author of R-Tree could write this easily.
> I read Bruce's interview and was really relaxed -
> how everything is going well. Bruce, we need your opinion.
>
>
> Oleg
> >
> > Vadim
> >
>
> Regards,
> Oleg
> _____________________________________________________________
> Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
> Sternberg Astronomical Institute, Moscow University (Russia)
> Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
> phone: +007(095)939-16-83, +007(095)939-23-83
>
Regards, Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
On Fri, 12 Jan 2001, Oleg Bartunov wrote: > On Thu, 11 Jan 2001, Mikheev, Vadim wrote: > > > > Um, you do realize that a contrib module that gets used as part of the > > > regress tests may as well be mainstream? At least in terms of the > > > portability requirements it will have to meet? > > > > > > I'm unhappy again. Bad enough we accepted a new feature during beta; > > > now we're going to expect an absolutely virgin contrib module to work > > > everywhere in order to pass regress tests? > > > > Ops, agreed. > > And I fear that in current code there is no one GiST index > > implementation -:( Should we worry about regress tests? -:) > > > Yes, we had to write contrib module even to test GiST. People, > I'm really confused after reading all of messages. > GiST is just an interface and to test any interface you need 2 sides. > In current code there is only one side. old GiST code live > untested for years. What's the problem ? It's the problem of > current regression test, mostly. > Ok. We could rewrite R-Tree to use GiST and make regression test which > will not make people nervous. But this certainly not for 7.1 and most > probable without us. Author of R-Tree could write this easily. The problem is that there is inadequate amount of time to fully test the the regression tests among all of the platforms that we support ... therefore, we can't test GiST among all of the platforms we support ... for instance, can you guarantee that _int.c will compile on *every* platform that PostgreSQL supports? That it will operate properly? Have you tested that?
Tom Lane wrote: > > Um, you do realize that a contrib module that gets used as part of the > regress tests may as well be mainstream? At least in terms of the > portability requirements it will have to meet? _If_ we want to have a tested GiST (and not the "probably works but really has some nasty known bugs" one) we need to write _tests_. To test it we need something that makes use of it. As the only things that make use of it are extensions we need to make use of them in tests. So I propose the following : 1. Keep the fixed (new) gist.c in the main codebase 2. make use of the RD-index and/or Gene's tests in contrib in regression tests 3. Tellpeople beforehand that it is not the end of the world if GiST _tests_ fail on their platform > I'm unhappy again. Bad enough we accepted a new feature during beta; > now we're going to expect an absolutely virgin contrib module to work > everywhere in order to pass regress tests? There can be always "expected" discrepancies in regress tests, and failing GiST test just tells people that if they want to use GiST on their platform they must probably fix things in core code as well. Currently they have to find it out the hard way - first lot of work trying to "fix" their own code and only then the bright idea that maybe it is actually broken in the core. IMHO, giving out real test results, even negative, instead of leaving things untested would be a honest thing to do. ----------------- Hannu
Oleg Bartunov wrote: > > OK. We found an old implementation of R-Tre using GiST (Pg95) > and we'll try to implement regression test using R-Tree > it's anyway will be a good test. How is it different than using RD-tree for tests ? Can you do it usin already compiled-in functions and modifying things only at SQL level ? Or is it just much simpler ? --------------------- Hannu
On Fri, 12 Jan 2001, Hannu Krosing wrote: > Oleg Bartunov wrote: > > > > OK. We found an old implementation of R-Tre using GiST (Pg95) > > and we'll try to implement regression test using R-Tree > > it's anyway will be a good test. > > How is it different than using RD-tree for tests ? > No difference at all ! It's just another implemetation of R-Tree. > Can you do it usin already compiled-in functions and modifying > things only at SQL level ? > unfortunately not ! Current postgres code has nothing connected with GiST and this is a problem ! How to test interface code without having two sides ? I understand we don't want to have another reason for complaints about non-working regression test. I never got regression test passed 100% on my Linux box with almost all versions of PostgreSQL but I could live with that. What's wrong with warning message if GiST test not passed ? > Or is it just much simpler ? > I'm interesting to test performance of built-in R-Tree and R-Tree + GiST. Oleg > --------------------- > Hannu > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
Oleg Bartunov <oleg@sai.msu.su> writes:
> What's wrong with
> warning message if GiST test not passed ?
You're being *way* too optimistic. An output discrepancy in a test of
GIST we could live with. But think about other scenarios:
1. GIST test coredumps on some platforms. This corrupts other tests
(at least through the "system is starting up" failure mode), thus
masking problems that we actually care about.
2. GIST test code does not compile on some platforms, causing "make check"
to fail completely.
At this point my vote is to leave the GIST test in contrib for 7.1.
Anyone who actually cares about GIST (to be blunt: all three of you)
can run it as a separate step. I don't want it in the standard regress
tests until 7.2, when we will have a reasonable amount of time to test
and debug the test.
regards, tom lane
> IMHO, giving out real test results, even negative, instead of leaving
> things untested would be a honest thing to do.
afaict there are several concerns or decisions, and we've made a few
already:
Re: gist.c patches...
1) Oleg and Hannu are committed to testing the repaired GiST as soon as
it is in the main tree. They are both testing already with the patched
version.
2) They will try to contact Gene to encourage testing with Gene's
application, though they have no reason to suspect from their own
testing that Gene's stuff will break.
3) There is a consensus that the gist.c patches should appear in the 7.1
release, to allow useful work with GiST and to enable further
development. So it is OK to commit the gist.c patches based on Oleg's
and Hannu's existing and future test plan.
Re: regression tests...
4) We all would like to see some regression tests of GiST. Tom Lane has
(rightly) expressed concern over unforeseen breakage in the regression
flow when done on other platforms.
5) Oleg has some regression-capable test code available for contrib, but
has indicated that fully (re)writing the regression tests will take too
much time.
6) We have at least two committed testers for the 7.1 release for the
GiST features. That is two more than we've ever had before (afacr Gene
didn't participate in the end-stage beta cycle, but I may not be
remembering correctly) so the risks that something is not right are
greatly reduced, to below the risks of same on the day of release for
previous versions.
8) Additional regression testing is required asap, but may not be
allowed into the default 7.1 test sequence.
How about adding an optional test a la "bigtest" for GiST for this
release? It could go mainstream for 7.1.x or for 7.2 as we get more
experience with it. This is just a suggestion and I'm sure there are
other possibilities. I'm pretty sure we agree on most of points 1-8, and
that 1-3 are resolved. Comments?
- Thomas
On Fri, 12 Jan 2001, Oleg Bartunov wrote: > On Fri, 12 Jan 2001, Hannu Krosing wrote: > > > Oleg Bartunov wrote: > > > > > > OK. We found an old implementation of R-Tre using GiST (Pg95) > > > and we'll try to implement regression test using R-Tree > > > it's anyway will be a good test. > > > > How is it different than using RD-tree for tests ? > > > > No difference at all ! It's just another implemetation of R-Tree. > > > Can you do it usin already compiled-in functions and modifying > > things only at SQL level ? > > > > unfortunately not ! Current postgres code has nothing connected with > GiST and this is a problem ! How to test interface code without > having two sides ? I understand we don't want to have another reason > for complaints about non-working regression test. I never got > regression test passed 100% on my Linux box with almost all versions > of PostgreSQL but I could live with that. What's wrong with > warning message if GiST test not passed ? It has *nothing* to do with passing or not, it has to do with timing of hte patches ... had they come in before we went beta, this would all have been a no-brainer ... because they didn't, the problem arises ... GiST changes are included ... testing of GiST changes aren't integrated ... can we *please* drop this whole thing already, as its really detracting from getting *real* work done with very little, to no, benefit ...
On Fri, 12 Jan 2001, Tom Lane wrote: > Oleg Bartunov <oleg@sai.msu.su> writes: > > What's wrong with > > warning message if GiST test not passed ? > > You're being *way* too optimistic. An output discrepancy in a test of > GIST we could live with. But think about other scenarios: > > 1. GIST test coredumps on some platforms. This corrupts other tests > (at least through the "system is starting up" failure mode), thus > masking problems that we actually care about. > > 2. GIST test code does not compile on some platforms, causing "make check" > to fail completely. > > At this point my vote is to leave the GIST test in contrib for 7.1. > Anyone who actually cares about GIST (to be blunt: all three of you) > can run it as a separate step. I don't want it in the standard regress > tests until 7.2, when we will have a reasonable amount of time to test > and debug the test. Agreed ... now let's move onto more important things, cause we've spent much too long on this as it is ... Namely, should we bundle up a beta4 this weeekend, so that the GiST changes are in place for further testing, or hold off for ... ?
On Fri, 12 Jan 2001, Thomas Lockhart wrote: > How about adding an optional test a la "bigtest" for GiST for this > release? It could go mainstream for 7.1.x or for 7.2 as we get more > experience with it. This is just a suggestion and I'm sure there are > other possibilities. I'm pretty sure we agree on most of points 1-8, and > that 1-3 are resolved. Comments? make GIST_TEST=yes to include GiST testing would be cool, if it can be done ... this way Tom's worry about non-GiST users having bad regress tests is appeased ... but I do agree with Tom that mainstreaming the GiST testing would be a bad idea ... if we could somehow include it as an optional test (as you say, ala bigtest), then, if nothing else, it saves having to cd to the contrib directory and run it there ... ala one stop shopping ... *But*, for the 3 ppl we've pointed out as users of GiST, this is definitely not a priority issue ... if we can do it, great, if not, no sweat either ...
Thomas Lockhart <lockhart@alumni.caltech.edu> writes:
> How about adding an optional test a la "bigtest" for GiST for this
> release?
We could do that, but it seems like rather pointless effort, compared
to just telling people "go run the tests in these contrib modules if
you want to test GIST".
I have no objection to fully integrating some GIST test(s) for 7.2.
I just don't want to deal with it at this late stage of the 7.1 cycle.
We have a long list of considerably more mainstream to-do items yet
to deal with ...
regards, tom lane
The Hermit Hacker <scrappy@hub.org> writes:
> Namely, should we bundle up a beta4 this weeekend, so that the GiST
> changes are in place for further testing, or hold off for ... ?
First I'd like to finish a couple of open items I have, like fixing
the CRIT_SECTION code so that SIGTERM response will not occur when
we are holding a spinlock. Should be able to get this stuff done in
a day or two, if I quit arguing about GIST and get back to work...
regards, tom lane
Thomas Lockhart writes: > How about adding an optional test a la "bigtest" for GiST for this > release? An optional test is like no test at all. No one runs optional tests. If the test is supposed to work then it should be mainstream. If the test might not work then you better go back and figure out what you're testing. If the test might not *compile* (which is probably the more severe problem that people are concerned about) then this idea won't help that at all unless you want to rework the regression test driver framework as well. -- Peter Eisentraut peter_e@gmx.net http://yi.org/peter-e/
> An optional test is like no test at all. No one runs optional tests. If
> the test is supposed to work then it should be mainstream. If the test
> might not work then you better go back and figure out what you're testing.
> If the test might not *compile* (which is probably the more severe problem
> that people are concerned about) then this idea won't help that at all
> unless you want to rework the regression test driver framework as well.
I agree completely. This is just a transition phase to get GiST into the
mainstream.
- Thomas
> At this point my vote is to leave the GIST test in contrib for 7.1. > Anyone who actually cares about GIST (to be blunt: all three of you) > can run it as a separate step. I don't want it in the standard regress > tests until 7.2, when we will have a reasonable amount of time to test > and debug the test. Agreed. I want the GIST fixes in 7.1, but adding a new test at this point is too risky. The issue is that only the GIST people will be using the GIST fixes, while adding it to the regression test will affect all users, which is too risky at this point. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
> Agreed ... now let's move onto more important things, cause we've spent > much too long on this as it is ... > > Namely, should we bundle up a beta4 this weeekend, so that the GiST > changes are in place for further testing, or hold off for ... ? I would hold off. GIST people can download the snapshot. Others aren't interested in GIST. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
On Fri, 12 Jan 2001, Tom Lane wrote: > Thomas Lockhart <lockhart@alumni.caltech.edu> writes: > > How about adding an optional test a la "bigtest" for GiST for this > > release? > > We could do that, but it seems like rather pointless effort, compared > to just telling people "go run the tests in these contrib modules if > you want to test GIST". > > I have no objection to fully integrating some GIST test(s) for 7.2. > I just don't want to deal with it at this late stage of the 7.1 cycle. > We have a long list of considerably more mainstream to-do items yet > to deal with ... Not up to us to deal with, its up to Oleg ... Oleg, if you could work on and submit patches for this before the release, that would be appreciated ... it might also serve to increase visibility of GiST if ppl know there is a regress test for it ...
On Fri, 12 Jan 2001, Tom Lane wrote: > The Hermit Hacker <scrappy@hub.org> writes: > > Namely, should we bundle up a beta4 this weeekend, so that the GiST > > changes are in place for further testing, or hold off for ... ? > > First I'd like to finish a couple of open items I have, like fixing > the CRIT_SECTION code so that SIGTERM response will not occur when > we are holding a spinlock. Should be able to get this stuff done in > a day or two, if I quit arguing about GIST and get back to work... Okay, let's scheduale for Monday then if we can ... unless someone comes across something major like we did with the whole beta2/beta3 release :)
I am sorry I wasn't listening -- I may have helped by at least answering the direct questions and by testing. I have, in fact, positively tested both my and Oleg's code in the today's snapshot on a number of linux and FreeBSD systems. I failed on this one: SunOS typhoon 5.7 Generic_106541-10 sun4u sparc SUNW,Ultra-1 on which configure didn't detect the absence of libz.so I don't think my applications are affected by Oleg's changes. But I understand the tension that occurred during the past few days and even though I am now satisfied with the agreement you seem to have achieved, I could have hardly influenced it in any reasonable way. I am as sympathetic with the need for a smooth an solid code control as I am with promoting great features (or, in this case, just keeping a feature alive). So, if I were around at the time I was asked to vote, I wouldn't know how. I usually find it difficult to take sides in "Motherhood vs. Clean Air" debates. It is true that throwing a core during a regression test does gives one a black eye. It is also true that there are probably hundreds of possible users, ignorant of the GiST, trying to invent surrogate solutions. As far as I am concerned, I will be satisfied with whatever solution you arrive at. I am pleased that in this neighborhood, reason prevails over faith. --Gene
selkovjr@mcs.anl.gov writes:
> I am sorry I wasn't listening -- I may have helped by at least
> answering the direct questions and by testing. I have, in fact,
> positively tested both my and Oleg's code in the today's snapshot on a
> number of linux and FreeBSD systems. I failed on this one:
> SunOS typhoon 5.7 Generic_106541-10 sun4u sparc SUNW,Ultra-1
> on which configure didn't detect the absence of libz.so
Really? Details please. It's hard to see how it could have messed
up on that.
regards, tom lane
Hi, I've put R-Tree realization using GiST (yet another test of our changes in gist code )on my gist page http://www.sai.msu.su/~megera/postgres/gist/ Also, I've put some GiST related papers for interested readers. The package( contrib-rtree_box_gist.tar.gz ) is built for 7.1. If you find it's interesting you may include it into contrib area for 7.1 from README.rtree_box_gist: 1. One interesting thing is that insertion time for built-in R-Tree is about 8 times more than ones for GiST implementationof R-Tree !!! 2. Postmaster requires much more memory for built-in R-Tree 3. Search time depends on dataset. In our case we got: +------------+-----------+--------------+ |Number boxes|R-tree,sec|R-tree using | | | | GiST, sec | +------------+-----------+--------------+ | 10| 0.002| 0.002| +------------+-----------+--------------+ | 100| 0.002| 0.002| +------------+-----------+--------------+ | 1000| 0.002| 0.002| +------------+-----------+--------------+ | 10000| 0.015| 0.025| +------------+-----------+--------------+ | 20000| 0.029| 0.048| +------------+-----------+--------------+ | 40000| 0.055| 0.092| +------------+-----------+--------------+ | 80000| 0.113| 0.178| +------------+-----------+--------------+ | 160000| 0.338| 0.337| +------------+-----------+--------------+ | 320000| 0.674| 0.673| +------------+-----------+--------------+ Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
> Tom Lane writes: > > selkovjr@mcs.anl.gov writes: ... > > SunOS typhoon 5.7 Generic_106541-10 sun4u sparc SUNW,Ultra-1 > > > on which configure didn't detect the absence of libz.so > > Really? Details please. It's hard to see how it could have messed > up on that. Tom, I didn't look well enough -- I apologize. The library is there, but ld.so believes it is not: typhoon> postmaster ld.so.1: postmaster: fatal: libz.so: open failed: No such file or directory Killed This may very well be just my ISP's problem. Anyway, the details are: 1. My (relevant) environment: LD_LIBRARY_PATH=/usr/openwin/lib:/usr/lib:/usr/ucblib:/usr/ccs/lib PGLIB=/home/customer/selkovjr/pgsql/lib PGDATA=/home/customer/selkovjr/pgsql/data PATH=/usr/local/vendor/SUNWspro/bin:/usr/local/bin:/usr/local/gnu/bin:/usr/local/GNU/bin:/usr/sbin:/usr/bin:/usr/ccs/bin:/usr/ucb:/etc:/usr/etc:/usr/openwin/bin:/home/customer/selkovjr/bin:./usr/local/bin::/home/customer/selkovjr/pgsql/bin 2. I built postgres (from the snapshot of Jan 13) with: ./configure --prefix=/home/customer/selkovjr/pgsql make make install 3. initdb worked. 4. The library in question is in /usr/openwin/lib: typhoon> ls -l /usr/openwin/lib | grep libz -rwxr-xr-x 1 root bin 97836 Sep 23 1999 libz.a -rwxr-xr-x 1 root bin 70452 Sep 23 1999 libz.so.1 I can't think of anything else. Is there a one-liner to test libz? I believe I have successfully tested and run 6.5.3 in the same environment. --Gene
selkovjr@mcs.anl.gov writes:
>>>> on which configure didn't detect the absence of libz.so
>>
>> Really? Details please. It's hard to see how it could have messed
>> up on that.
> I didn't look well enough -- I apologize. The library is there, but
> ld.so believes it is not:
> typhoon> postmaster
> ld.so.1: postmaster: fatal: libz.so: open failed: No such file or directory
> Killed
Odd. Can you show us the part of config.log that relates to zlib?
It's strange that configure's check to see if zlib is linkable should
succeed, only to have the live startup fail. Is it possible that
you ran configure with a different library search path (LD_LIBRARY_PATH
or local equivalent) than you are using now?
It's suspicious that the error message mentions libz.so when the actual
file name is libz.so.1, but I still don't see how that could result in
configure's link test succeeding but the executable not running.
regards, tom lane
Tom Lane writes:
> selkovjr@mcs.anl.gov writes:
> >>>> on which configure didn't detect the absence of libz.so
> >>
> >> Really? Details please. It's hard to see how it could have messed
> >> up on that.
>
> > I didn't look well enough -- I apologize. The library is there, but
> > ld.so believes it is not:
>
> > typhoon> postmaster
> > ld.so.1: postmaster: fatal: libz.so: open failed: No such file or directory
> > Killed
>
> Odd. Can you show us the part of config.log that relates to zlib?
configure:4179: checking for zlib.h
configure:4189: gcc -E conftest.c >/dev/null 2>conftest.out
configure:4207: checking for inflate in -lz
configure:4226: gcc -o conftest conftest.c -lz -lgen -lnsl -lsocket -ldl -lm -lreadline -ltermcap -lcurses 1>&5
configure:4660: checking for crypt.h
This doesn't tell me much. But I modified configure to exit right
after this, without removing conftest*, and when I ran conftest it came
back with the same message:
typhoon> ./conftest
ld.so.1: ./conftest: fatal: libz.so: open failed: No such file or directory
Killed
> It's strange that configure's check to see if zlib is linkable should
> succeed, only to have the live startup fail.
It is. In this line:
if { (eval echo configure:4226: \"$ac_link\") 1>&5; (eval $ac_link) 2>&5; } && test -s conftest${ac_exeext}; then
why is conftest tested for size instead of being executed?
> Is it possible that
> you ran configure with a different library search path (LD_LIBRARY_PATH
> or local equivalent) than you are using now?
No, I didn't alter it. I am using the system-wide settings.
> It's suspicious that the error message mentions libz.so when the actual
> file name is libz.so.1, but I still don't see how that could result in
> configure's link test succeeding but the executable not running.
That puzzles me as well. It seems to be because there is no libz.so on
the system. For if I do this:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/customer/selkovjr/lib
ln -s /usr/openwin/lib/libz.so.1 ~/lib/libz.so
the libz problem is gone, only to be followed by the next one:
typhoon> ./conftest
ld.so.1: ./conftest: fatal: libreadline.so: open failed: No such file or directory
The odd thing is, there is no libreadline.so* on this system. Here's the corresponding part of config.log:
configure:3287: checking for library containing readline
configure:3305: gcc -o conftest conftest.c -ltermcap -lcurses 1>&5
Undefined first referencedsymbol in file
readline /var/tmp/ccxxiW3R.o
ld: fatal: Symbol referencing errors. No output written to conftest
collect2: ld returned 1 exit status
configure: failed program was:
#line 3294 "configure"
#include "confdefs.h"
/* Override any gcc2 internal prototype to avoid an error. */
/* We use char because int might match the return type of a gcc2 builtin and then its argument prototype would still
apply. */
char readline();
int main() {
readline()
; return 0; }
configure:3327: gcc -o conftest conftest.c -lreadline -ltermcap -lcurses
1>&5
This system is probaly badly misconfigured, but it would be great if
configure could see that. By the way, would you mind if I asked you to
log in and take a look? Is there a phone number where I can get you
with the password? I am not sure whether such tests could be of any
value, but it's the only Sun machine available to me for testing.
Thank you,
--Gene
selkovjr@mcs.anl.gov writes:
> configure:4207: checking for inflate in -lz
> configure:4226: gcc -o conftest conftest.c -lz -lgen -lnsl -lsocket -ldl -lm -lreadline -ltermcap -lcurses
1>&5
> configure:4660: checking for crypt.h
> This doesn't tell me much. But I modified configure to exit right
> after this, without removing conftest*, and when I ran conftest it came
> back with the same message:
> typhoon> ./conftest
> ld.so.1: ./conftest: fatal: libz.so: open failed: No such file or directory
> Killed
>> It's strange that configure's check to see if zlib is linkable should
>> succeed, only to have the live startup fail.
> This system is probaly badly misconfigured, but it would be great if
> configure could see that.
Gene and I looked into this, and the cause of the misbehavior is this:
gcc on this installation is set to search /usr/local/lib (along with the
usual system library directories). libz.so and libreadline.so are
indeed in /usr/local/lib, so configure's tests to see if they can be
linked against will succeed. But he had LD_LIBRARY_PATH set to a list
that did *not* include /usr/local/lib, so actually firing up the
executable would fail.
As he says, it'd be nice if configure could either prevent this or at
least detect it. Not sure about a good way to do that --- any ideas?
regards, tom lane
Tom Lane writes: > Gene and I looked into this, and the cause of the misbehavior is this: > gcc on this installation is set to search /usr/local/lib (along with the > usual system library directories). libz.so and libreadline.so are > indeed in /usr/local/lib, so configure's tests to see if they can be > linked against will succeed. But he had LD_LIBRARY_PATH set to a list > that did *not* include /usr/local/lib, so actually firing up the > executable would fail. You get what you pay for. If you're running executables from configure you're asking for it. This setup is a poor man's cross-compilation situation because the system you're compiling on is not identically configured to the system you're going to run on. (Strictly speaking, the behaviour of a test program might even vary with different LD_LIBRARY_PATH settings.) So a) PostgreSQL does not support cross-compilation (yet). Too bad. b) We could get rid of all executition time checks in configure (to remedy (a)). This is one of my plans for the future. c) You could move the execution time checks up before the suspicious library checks, but I'm afraid that this will onlycure a particular symptom and might introduce other problems. I'd say, you're stuck. -- Peter Eisentraut peter_e@gmx.net http://yi.org/peter-e/
Patrick Welche <prlw1@newn.cam.ac.uk> writes:
> Wouldn't a -Wl,-R/usr/local/lib have helped?
Well, yeah, but how would we know to do that? The fact that gcc is
searching /usr/local/lib is completely unknown to configure.
regards, tom lane
On Fri, Jan 19, 2001 at 12:46:53AM +0100, Peter Eisentraut wrote: > Tom Lane writes: > > > Gene and I looked into this, and the cause of the misbehavior is this: > > gcc on this installation is set to search /usr/local/lib (along with the > > usual system library directories). libz.so and libreadline.so are > > indeed in /usr/local/lib, so configure's tests to see if they can be > > linked against will succeed. But he had LD_LIBRARY_PATH set to a list > > that did *not* include /usr/local/lib, so actually firing up the > > executable would fail. > > You get what you pay for. If you're running executables from configure > you're asking for it. > > This setup is a poor man's cross-compilation situation because the system > you're compiling on is not identically configured to the system you're > going to run on. (Strictly speaking, the behaviour of a test program > might even vary with different LD_LIBRARY_PATH settings.) > > So > > a) PostgreSQL does not support cross-compilation (yet). Too bad. > > b) We could get rid of all executition time checks in configure (to > remedy (a)). This is one of my plans for the future. > > c) You could move the execution time checks up before the suspicious > library checks, but I'm afraid that this will only cure a particular > symptom and might introduce other problems. > > I'd say, you're stuck. > > -- > Peter Eisentraut peter_e@gmx.net http://yi.org/peter-e/ > Wouldn't a -Wl,-R/usr/local/lib have helped? Cheers, Patrick
I have added the URL to the GIST SGML docs. > Hi, > > I've put R-Tree realization using GiST (yet another test of our changes in > gist code )on my gist page http://www.sai.msu.su/~megera/postgres/gist/ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > Also, I've put some GiST related papers for interested readers. > The package( contrib-rtree_box_gist.tar.gz ) is built for 7.1. > If you find it's interesting you may include it into contrib area for 7.1 > > from README.rtree_box_gist: > > > 1. One interesting thing is that insertion time for built-in R-Tree is > about 8 times more than ones for GiST implementation of R-Tree !!! > 2. Postmaster requires much more memory for built-in R-Tree > 3. Search time depends on dataset. In our case we got: > +------------+-----------+--------------+ > |Number boxes|R-tree, sec|R-tree using | > | | | GiST, sec | > +------------+-----------+--------------+ > | 10| 0.002| 0.002| > +------------+-----------+--------------+ > | 100| 0.002| 0.002| > +------------+-----------+--------------+ > | 1000| 0.002| 0.002| > +------------+-----------+--------------+ > | 10000| 0.015| 0.025| > +------------+-----------+--------------+ > | 20000| 0.029| 0.048| > +------------+-----------+--------------+ > | 40000| 0.055| 0.092| > +------------+-----------+--------------+ > | 80000| 0.113| 0.178| > +------------+-----------+--------------+ > | 160000| 0.338| 0.337| > +------------+-----------+--------------+ > | 320000| 0.674| 0.673| > +------------+-----------+--------------+ > > > Regards, > Oleg > _____________________________________________________________ > Oleg Bartunov, sci.researcher, hostmaster of AstroNet, > Sternberg Astronomical Institute, Moscow University (Russia) > Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ > phone: +007(095)939-16-83, +007(095)939-23-83 > > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026