Thread: 8.2 beta blockers
I see the following items standing between us and putting out 8.2beta1: * Set client encoding based on OS environment - Peter E. I'm not sure whether Peter is intending to complete this item for 8.2 or not, but if it's to be done it ought to be done before we start beta. * The contrib/userlock replacement issue We have three possible choices for this: do nothing, install a bug-compatible, allegedly-clean-room implementation in contrib: http://archives.postgresql.org/pgsql-patches/2006-09/msg00077.php or put a hopefully-cleaner design into core, eg per my suggestions here: http://archives.postgresql.org/pgsql-hackers/2006-09/msg00467.php I favor the third alternative, mainly because by changing the API we remove all doubt as to whether any "intellectual property" remains from the original GPL'd code. However, we've got to make up our minds and get on with it. * AIX linking issues This isn't necessarily a beta-stopper, but it'd be nice to get it done so we can be sure that any beta testing done on AIX will test the revisions. * Documentation: VALUES-list docs and release notes Must-do so that beta testers will know what there is to test. There are other open items on Bruce's list but none of them seem to me to be must-fix-before-beta. I do not know the status of the client-encoding change, but I think we could get all the other items done tomorrow. I propose that we try to wrap these things and ship beta1 Tuesday or Wednesday. Sitting around waiting is not a productive use of time, especially when it seems that many people's attention has already moved on to 8.3 development ideas. Personally I'm willing to commit to making the VALUES-list docs and userlock replacement code happen tomorrow. Bruce seems to be close on the release notes, and if the other two items aren't ready, well, beta1 can ship without 'em. Comments? regards, tom lane
A long time ago, in a galaxy far, far away, tgl@sss.pgh.pa.us (Tom Lane) wrote: > I see the following items standing between us and putting out 8.2beta1: > * AIX linking issues > > This isn't necessarily a beta-stopper, but it'd be nice to get it done > so we can be sure that any beta testing done on AIX will test the > revisions. This has to do with the discussion about shared vs static libs? If a patch is put out, it'll get build-farm results on AIX within a couple hours. If there's something worth trying, then try away... -- select 'cbbrowne' || '@' || 'gmail.com'; http://cbbrowne.com/info/wp.html When I met th'POPE back in '58, I scrubbed him with a MILD SOAP or DETERGENT for 15 minutes. He seemed to enjoy it ...
On Sun, Sep 17, 2006 at 07:38:38PM -0400, Tom Lane wrote: > We have three possible choices for this: do nothing, install a > bug-compatible, allegedly-clean-room implementation in contrib: > http://archives.postgresql.org/pgsql-patches/2006-09/msg00077.php > or put a hopefully-cleaner design into core, eg per my suggestions here: > http://archives.postgresql.org/pgsql-hackers/2006-09/msg00467.php > I favor the third alternative, mainly because by changing the API > we remove all doubt as to whether any "intellectual property" > remains from the original GPL'd code. However, we've got to make up > our minds and get on with it. FWIW, I'm +1 on the cleaner design you suggested. While I understand the concerns of adding features/API this late; as a user, I'd rather not wait another year to have these available in core(yes, I know alternative measures would exist if it did not make it into core, but the convenience of having it there would certainly be nice). That is, I really like the waiting variant. It is something that I would use. The lack thereof(IIRC) in the current contrib implementation is something that I have recently lamented about. I understand that "want" is not a reason to compromise the feature freeze, so I hope the legal concerns Tom mentions will be enough. =)
On 2006-09-18, James William Pye <pgsql@jwp.name> wrote: > FWIW, I'm +1 on the cleaner design you suggested. While I understand the > concerns of adding features/API this late; Adding features is one thing, breaking existing users of the code is another. -- Andrew, Supernews http://www.supernews.com - individual and corporate NNTP services
Christopher Browne <cbbrowne@acm.org> writes:
> A long time ago, in a galaxy far, far away, tgl@sss.pgh.pa.us (Tom Lane) wrote:
>> I see the following items standing between us and putting out 8.2beta1:
>> * AIX linking issues
> This has to do with the discussion about shared vs static libs?
> If a patch is put out, it'll get build-farm results on AIX within a
> couple hours.
Yeah, I know, which is why I don't find it absolutely critical that
this make it to beta1. But one of the concerns mentioned in the thread
is that the changes might break things for older AIX versions. If we
get it into beta1, we have a better chance of finding out before release
whether there are any issues with AIX versions that aren't represented
in buildfarm.
regards, tom lane
Andrew - Supernews <andrew+nonews@supernews.com> writes:
> On 2006-09-18, James William Pye <pgsql@jwp.name> wrote:
>> FWIW, I'm +1 on the cleaner design you suggested. While I understand the
>> concerns of adding features/API this late;
> Adding features is one thing, breaking existing users of the code is another.
We already have an established solution for people who need backwards
compatibility: download the old userlock code from pgfoundry. I'm
unclear why we should duplicate that project within core.
regards, tom lane
On Sun, Sep 17, 2006 at 07:38:38PM -0400, Tom Lane wrote: > * The contrib/userlock replacement issue > > We have three possible choices for this: do nothing, install a > bug-compatible, allegedly-clean-room implementation in contrib: > http://archives.postgresql.org/pgsql-patches/2006-09/msg00077.php > or put a hopefully-cleaner design into core, eg per my suggestions here: > http://archives.postgresql.org/pgsql-hackers/2006-09/msg00467.php > I favor the third alternative, mainly because by changing the API > we remove all doubt as to whether any "intellectual property" > remains from the original GPL'd code. However, we've got to make up > our minds and get on with it. One problem I see with userlock is that you're asking for lock ID conflicts unless you control everything on the system that's using userlocks. Right now, doing so probably isn't that difficult, but my hope is that as PostgreSQL becomes more acceptable we'll see more software written to use it... software that might need to play well with other software within the same database. I think the best way to fix this is to set aside some range of the lock ID space for a naming scheme that is based on schemas, or some other kind of text identifier. I think this really just amounts to putting some recommended practices into the docs; perhaps not using any int4,int4 IDs where the first int is larger than 16k or whatever the maximum system OID is. That would allow for segmenting locks based on schema OIDs. Down the road we might want a nicer, schema-based interface, but just setting aside the space should be fine for now. -- Jim Nasby jim@nasby.net EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
Tom Lane wrote: > I see the following items standing between us and putting out 8.2beta1: > > * Set client encoding based on OS environment - Peter E. [snip] > Personally I'm willing to commit to making the VALUES-list docs and > userlock replacement code happen tomorrow. Bruce seems to be close > on the release notes, and if the other two items aren't ready, well, > beta1 can ship without 'em. If you talk about the "client encoding thing" here... Since the client encoding change is a feature addition and a behavioral change, I think this should really be done before going to beta, IMHO. Btw. was there any mailing list discussion on this item? Other than this: http://archives.postgresql.org/pgsql-hackers/2003-05/msg00737.php. I really hope that this change will only affect psql, not pg_dump, as Peter wrote in 2003. I would strongly object to such a change (as much as my voice counts). The current behavior of dumping with the database encoding is exactly the right thing to do. I have a database in UTF-8 here. Using any LATIN based encoding for dumping the database will simply break, because there are always characters that don't map into the encoding. Even with psql there could be issues with existing scripts, but I see a benefit at least. Best Regards Michael Paesold
Tom Lane wrote: > I see the following items standing between us and putting out 8.2beta1: > > * Set client encoding based on OS environment - Peter E. > > I'm not sure whether Peter is intending to complete this item for 8.2 > or not, but if it's to be done it ought to be done before we start beta. This just reminds me, are there plans to take into account multibyte server encodings inside the client quote function? regards, Lukas
Am Montag, 18. September 2006 09:20 schrieb Lukas Kahwe Smith: > This just reminds me, are there plans to take into account multibyte > server encodings inside the client quote function? Huh? -- Peter Eisentraut http://developer.postgresql.org/~petere/
Am Montag, 18. September 2006 01:38 schrieb Tom Lane: > * Set client encoding based on OS environment - Peter E. This is not an item for 8.2 in my mind. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Quoth tgl@sss.pgh.pa.us (Tom Lane): > Christopher Browne <cbbrowne@acm.org> writes: >> A long time ago, in a galaxy far, far away, tgl@sss.pgh.pa.us (Tom Lane) wrote: >>> I see the following items standing between us and putting out 8.2beta1: >>> * AIX linking issues > >> This has to do with the discussion about shared vs static libs? > >> If a patch is put out, it'll get build-farm results on AIX within a >> couple hours. > > Yeah, I know, which is why I don't find it absolutely critical that > this make it to beta1. But one of the concerns mentioned in the thread > is that the changes might break things for older AIX versions. If we > get it into beta1, we have a better chance of finding out before release > whether there are any issues with AIX versions that aren't represented > in buildfarm. Well, systems running elder versions of AIX are doing so without any IBM support. AIX 5.1 is no longer supported... I'd have a problem with running any of those elder versions due to the absence of support, myself... -- (format nil "~S@~S" "cbbrowne" "gmail.com") http://linuxdatabases.info/info/nonrdbms.html "Microsoft is a cross between the Borg and the Ferengi. Unfortunately, they use Borg to do their marketing and Ferengi to do their programming." -- Simon Slavin in asr
Peter Eisentraut wrote: > Am Montag, 18. September 2006 09:20 schrieb Lukas Kahwe Smith: >> This just reminds me, are there plans to take into account multibyte >> server encodings inside the client quote function? > > Huh? Ah, I just checked the libpq docs and there seems to be a PQescapeStringConn. Not sure when this was added, I think PHP does not yet use it. I will investigate this and will make sure its used in favor of the deprecated old PQescapeString function. regards, Lukas
Am Montag, 18. September 2006 14:25 schrieb Lukas Kahwe Smith: > Peter Eisentraut wrote: > > Am Montag, 18. September 2006 09:20 schrieb Lukas Kahwe Smith: > >> This just reminds me, are there plans to take into account multibyte > >> server encodings inside the client quote function? > > > > Huh? > > Ah, I just checked the libpq docs and there seems to be a > PQescapeStringConn. Not sure when this was added, I think PHP does not > yet use it. I will investigate this and will make sure its used in favor > of the deprecated old PQescapeString function. That was part of the big security update we had a while back. If PHP missed that, it's their problem. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Michael Paesold <mpaesold@gmx.at> writes:
>> * Set client encoding based on OS environment - Peter E.
> I really hope that this change will only affect psql, not pg_dump, as Peter
> wrote in 2003. I would strongly object to such a change (as much as my
> voice counts). The current behavior of dumping with the database encoding
> is exactly the right thing to do.
Actually, I realize after a quick look at the pg_dump code that its
current behavior is to dump in1. Specified encoding if a -E switch is given.2. PGCLIENTENCODING, if that environment
varexists.3. Else, server encoding.
So there's already an environment dependency, although it's for
something much less likely to be set than LANG. I tend to agree
that we'd better avoid having dumps depend on LANG ... wonder if
we should remove the dependency on PGCLIENTENCODING too.
regards, tom lane
"Jim C. Nasby" <jimn@enterprisedb.com> writes:
> One problem I see with userlock is that you're asking for lock ID
> conflicts unless you control everything on the system that's using
> userlocks.
Well, the lock IDs already include the database OID under the hood,
so you only need to control stuff within your database. Beyond that,
yeah, it's your responsibility...
regards, tom lane
Am Montag, 18. September 2006 15:48 schrieb Tom Lane: > So there's already an environment dependency, although it's for > something much less likely to be set than LANG. I tend to agree > that we'd better avoid having dumps depend on LANG ... wonder if > we should remove the dependency on PGCLIENTENCODING too. I'd say that in principle, any console application should respect the console's encoding settings by default. pg_dump is not only a backup tool but also useful for quasi-interactive use. We could check if stdout goes to a terminal or something like that. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Peter Eisentraut wrote: > Am Montag, 18. September 2006 01:38 schrieb Tom Lane: > > * Set client encoding based on OS environment - Peter E. > > This is not an item for 8.2 in my mind. OK, moved to TODO: * Set client encoding based on the client operating system encoding Currently client_encoding is set in postgresql.conf,which defaults to the server encoding. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
On Mon, Sep 18, 2006 at 10:10:32AM -0400, Tom Lane wrote: > "Jim C. Nasby" <jimn@enterprisedb.com> writes: > > One problem I see with userlock is that you're asking for lock ID > > conflicts unless you control everything on the system that's using > > userlocks. > > Well, the lock IDs already include the database OID under the hood, > so you only need to control stuff within your database. Beyond that, > yeah, it's your responsibility... Which is what I'm worried about for the future... that's why I'd like to recommend to users some kind of best practice that allows for a system in the future that doesn't require managing every single lock in the system. I believe recommending that you not use locks with the first int4 above 16k (and whatever the equivalent would be for int8) would be a good way to do that, as it would allow for segregating locks by schema OID. It's important to recommend this now so that people don't write software that will clobber other locks in the future. -- Jim Nasby jimn@enterprisedb.com EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
"Jim C. Nasby" <jimn@enterprisedb.com> writes:
> I believe recommending that you not use locks with the first
> int4 above 16k (and whatever the equivalent would be for int8) would be
> a good way to do that, as it would allow for segregating locks by schema
> OID.
That seems pretty content-free to me, if not counter-productive. To the
extent that we can foresee the usage for the two-int4s lock style, the
first one is likely to be an OID (eg, a user table's OID) which the user
is not going to be able to control the range of. Also, if you are
locking on the basis of object OIDs, there's no need to worry which
schema they are in.
regards, tom lane
Lukas, > Ah, I just checked the libpq docs and there seems to be a > PQescapeStringConn. Not sure when this was added, I think PHP does not > yet use it. I will investigate this and will make sure its used in favor > of the deprecated old PQescapeString function. PHP driver authors and major PHP projects (PHPBB, SQL-Ledger, GForge, etc.) were notified about this *before* it happened. I beleive that PDO will support it soon, pg_connect already does, and PEAR::DB2 does. -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco
All, Is UserLocks a cool enough feature to be worth mentioning in the 8.2 PR? If so, can someone explain it to me off-list? I still don't get what it does ... -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco
> -----Original Message----- > From: pgsql-hackers-owner@postgresql.org > Yeah, I know, which is why I don't find it absolutely critical that > this make it to beta1. But one of the concerns mentioned in > the thread > is that the changes might break things for older AIX versions. If we > get it into beta1, we have a better chance of finding out > before release > whether there are any issues with AIX versions that aren't represented > in buildfarm. > As a backdoor, we can leave in the code to compile static for the old versions, but I am not sure its worth it. As a side note, I tried to build CVS on an AIX 4.1 box, and it already has issues. The TOC is too large for ld, amongst others. I had tried to get our older box onto the buildfarm, but the version of Perl on that box was too ancient to run the buildfarm code, and I didn't think people would care too much about it missing, since its such an old version anyway. Then I discovered the issues with trying to build by hand, and gave up at that point. I think we should make our cut-off version as 4.3. This is the first to support run time linking, so the changes we are talking about for today will work on that version. It came out in October 97 and was EOL'd Dec 2001. If someone really wants to run on something older than a 9 year old OS, that was EOL'd almost 5 years ago. I think they should do so at their peril, and its not up to us to go out of our way to support them. I don't think we will be making an AIX uesrs sad by adding support for dynamic linking, but breaking old versions of the OS. Thanks,-rocco
Josh Berkus wrote: > All, > > Is UserLocks a cool enough feature to be worth mentioning in the 8.2 PR? > If so, can someone explain it to me off-list? I still don't get what it > does ... Probably not. -- Bruce Momjian bruce@momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian <bruce@momjian.us> writes:
> Josh Berkus wrote:
>> Is UserLocks a cool enough feature to be worth mentioning in the 8.2 PR?
> Probably not.
Especially not since the capability has been there right along.
regards, tom lane
On 9/18/06, Josh Berkus <josh@agliodbs.com> wrote: > All, > > Is UserLocks a cool enough feature to be worth mentioning in the 8.2 PR? > If so, can someone explain it to me off-list? I still don't get what it > does ... > yes, i can explain it in detail, and am willing to kick in some documentation. it is very cool, and relatively undiscovered. email me for details. the short answer is you can do long term locking (not scoped to transaction) using the postgresql high speed internal lock engine. this allows you to bypass various locking hacks such as keeping transactions (too) long term, or maintaining a lock flag in a table, which has cleanup issues. they are great if you need to emulate pessimistic locking and transactions is an awkward fit. its up to you, the user to determine when and what to lock. i really beat on the user (advisory locks) when doing a cobol/isam conversion project. they work fabulous. merlin
On 9/18/06, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Bruce Momjian <bruce@momjian.us> writes: > > Josh Berkus wrote: > >> Is UserLocks a cool enough feature to be worth mentioning in the 8.2 PR? > > > Probably not. > > Especially not since the capability has been there right along. I disagree, almost nobody knows about them, due to lousy name and lack of documentation. I think the pr hook should be merited to their general usefullness, previous implementation notwithstanding. techinally, we added a wait version too, right? 8.1 table partitioning was dressed up inheritance right? :) (not quite the same, i know but...) merlin
"Merlin Moncure" <mmoncure@gmail.com> writes:
> yes, i can explain it in detail, and am willing to kick in some
> documentation.
Ah-hah, you're on the hook for docs then ;-).
I'm going to go ahead with implementing it in-core per my last proposal:
void pg_advisory_lock(int8) waitvoid pg_advisory_lock_shared(int8) waitbool
pg_try_advisory_lock(int8) no waitbool pg_try_advisory_lock_shared(int8) no waitbool
pg_advisory_unlock(int8) returns T if successfulbool pg_advisory_unlock_shared(int8) returns T if
successful
plus all the above taking 2 int4's, plus
void pg_advisory_unlock_all()
but am happy to let someone else do the docs.
As far as the PR material goes, something like "advisory locks
incorporated into core" would be OK, but don't make it sound like
there was nothing there before ...
regards, tom lane
On 9/18/06, Tom Lane <tgl@sss.pgh.pa.us> wrote: > "Merlin Moncure" <mmoncure@gmail.com> writes: > > yes, i can explain it in detail, and am willing to kick in some > > documentation. > > Ah-hah, you're on the hook for docs then ;-). sure no problem. the prototypes you suggested are imo the way to go, with two small considerations: is it worth considering using the oid type instead of int4 since the 'locktag' fields are unsigned? also, the userlocks raised a warning if you tried to release a non-existing lock. should that stay or go? > As far as the PR material goes, something like "advisory locks > incorporated into core" would be OK, but don't make it sound like > there was nothing there before ... ok, thats a good compromise. merlin
Tom, Merlin, > > As far as the PR material goes, something like "advisory locks > > incorporated into core" would be OK, but don't make it sound like > > there was nothing there before ... > > ok, thats a good compromise. Yes, although if I'm doing this for PR, I need to use language which is standard in the industry or I won't be understood. What about "high-concurrency pessimistic locking mechanism"? I don't have a problem covering this in PR even though it's not really "new": a) it's never been in our PR before; b) while it's been available for 4+ versions, who knew about it? c) we have a new refactored code and cleaned up API -- Josh Berkus PostgreSQL @ Sun San Francisco
"Merlin Moncure" <mmoncure@gmail.com> writes:
> sure no problem. the prototypes you suggested are imo the way to go,
> with two small considerations:
> is it worth considering using the oid type instead of int4 since the
> 'locktag' fields are unsigned?
Hmm ... I was thinking it didn't matter, but on closer look, the
int4->oid cast is implicit while the oid->int4 one is only assignment.
So you'd need to write a cast to pass an OID if we declare the functions
as taking int4. But you'll need a cast anyway if you want to pass a
single OID to the int8-taking version (that's an assignment cast too).
The downside of declaring the functions to take OID is that people might
think they could *only* use OIDs, which isn't so, they can use any
int4-sized key they feel like.
Not seeing a strong reason one way or the other ... what do you think?
> also, the userlocks raised a warning if you tried to release a
> non-existing lock. should that stay or go?
That's in the core code I think, so it won't change.
regards, tom lane
Josh Berkus <josh@agliodbs.com> writes:
>>> As far as the PR material goes, something like "advisory locks
>>> incorporated into core" would be OK, but don't make it sound like
>>> there was nothing there before ...
> Yes, although if I'm doing this for PR, I need to use language which is
> standard in the industry or I won't be understood. What about
> "high-concurrency pessimistic locking mechanism"?
"Advisory lock" *is* a standard term. The other sounds, well, mighty PR-ish.
regards, tom lane
On 9/18/06, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Hmm ... I was thinking it didn't matter, but on closer look, the > int4->oid cast is implicit while the oid->int4 one is only assignment. > So you'd need to write a cast to pass an OID if we declare the functions > as taking int4. But you'll need a cast anyway if you want to pass a > single OID to the int8-taking version (that's an assignment cast too). > > The downside of declaring the functions to take OID is that people might > think they could *only* use OIDs, which isn't so, they can use any > int4-sized key they feel like. hm. this is really a byproduct of oid being the catchall unsigned int4 type since it has the most built in casts. i agree 100% though on the oid perception however, i don't like userland oids at all, until such time as an 8 bit one comes out. i would say leave as int4 unless you were willing to sql typedef the oid to some other name. merlin
On Mon, Sep 18, 2006 at 05:06:09PM -0400, Merlin Moncure wrote: > On 9/18/06, Tom Lane <tgl@sss.pgh.pa.us> wrote: > >Hmm ... I was thinking it didn't matter, but on closer look, the > >int4->oid cast is implicit while the oid->int4 one is only assignment. > >So you'd need to write a cast to pass an OID if we declare the functions > >as taking int4. But you'll need a cast anyway if you want to pass a > >single OID to the int8-taking version (that's an assignment cast too). > > > >The downside of declaring the functions to take OID is that people might > >think they could *only* use OIDs, which isn't so, they can use any > >int4-sized key they feel like. > > hm. this is really a byproduct of oid being the catchall unsigned int4 > type since it has the most built in casts. i agree 100% though on the > oid perception however, i don't like userland oids at all, until such > time as an 8 bit one comes out. i would say leave as int4 unless you > were willing to sql typedef the oid to some other name. Would adding OID versions of the functions (so there'd be int8, (int4, int4) and (oid,oid)) be overkill? -- Jim Nasby jimn@enterprisedb.com EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
"Jim C. Nasby" <jimn@enterprisedb.com> writes:
> Would adding OID versions of the functions (so there'd be int8, (int4,
> int4) and (oid,oid)) be overkill?
I think what it'd mostly accomplish is to create "couldn't resolve
ambiguous function" errors :-(
regards, tom lane
Lukas Kahwe Smith wrote: > Peter Eisentraut wrote: >> Am Montag, 18. September 2006 09:20 schrieb Lukas Kahwe Smith: >>> This just reminds me, are there plans to take into account multibyte >>> server encodings inside the client quote function? >> >> Huh? > > Ah, I just checked the libpq docs and there seems to be a > PQescapeStringConn. Not sure when this was added, I think PHP does not > yet use it. I will investigate this and will make sure its used in favor > of the deprecated old PQescapeString function. This will be fixed as of PHP 5.2.0 regards, Lukas