Thread: testing plpython3u on 9.0beta2
I received two errors (described below) in installing 9.0beta2 on Kubuntu 10.04 , RhodiumToad on IRC recommended I post them here.
I did not have a 2.x or 3.x python dev installed, but I was really only interested in python3 via plython3u.
So...
sudo apt-get install python3-all-dev
Configure works fine...
"./configure --with-pgport=5433 --with-python --with-ossp-uuid --with-libxml --with-libxslt --with-perl"
"make" fails while trying to build plpython, Cannot find python.h i believe....(I sadly did not save the text of the error and have since continued onward. But it was not finding a file while building plpython. I believe I would have to uninstall a few things to reproduce.)
So I tried installing the 2.x dev. After running "sudo apt-get install python-all-dev" then "make;make check;sudo make install", it all worke fine.
However, when issuing a "createlang plpython3u template1", I get "createlang: language installation failed: ERROR: could not access file "$libdir/plpython3": No such file or directory". But if I "createlang plpython2u template1" first, then it will allow "createlang plpython3u template1".
Am I doing something incorrect, or...?
(As an aside, functions created with LANGUAGE plpython3u appear to work.)
On Tue, Jun 22, 2010 at 10:41 PM, Chris <rfusca@gmail.com> wrote: > I received two errors (described below) in installing 9.0beta2 on Kubuntu > 10.04 , RhodiumToad on IRC recommended I post them here. > I did not have a 2.x or 3.x python dev installed, but I was really only > interested in python3 via plython3u. > So... > sudo apt-get install python3-all-dev > Configure works fine... > "./configure --with-pgport=5433 --with-python --with-ossp-uuid --with-libxml > --with-libxslt --with-perl" > "make" fails while trying to build plpython, Cannot find python.h i > believe....(I sadly did not save the text of the error and have since > continued onward. But it was not finding a file while building plpython. I > believe I would have to uninstall a few things to reproduce.) > So I tried installing the 2.x dev. After running "sudo apt-get install > python-all-dev" then "make;make check;sudo make install", it all worke fine. > However, when issuing a "createlang plpython3u template1", I get > "createlang: language installation failed: ERROR: could not access file > "$libdir/plpython3": No such file or directory". But if I "createlang > plpython2u template1" first, then it will allow "createlang plpython3u > template1". > Am I doing something incorrect, or...? > (As an aside, functions created with LANGUAGE plpython3u appear to work.) I can reproduce this, here. The problem seems to be that plpython only build either plpython2.so or plython3.so, but both languages expect a call handler called plython_call_handler. So once we load the shared library for one language, the other language just grabs the same call handler. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company
Robert Haas <robertmhaas@gmail.com> writes:
> I can reproduce this, here. The problem seems to be that plpython
> only build either plpython2.so or plython3.so, but both languages
> expect a call handler called plython_call_handler. So once we load
> the shared library for one language, the other language just grabs the
> same call handler.
We had better fix that --- I can definitely foresee installations
wanting to use both plpython2 and plpython3. This'd require a change in
the default contents of pg_pltemplate, though. Does it seem important
enough to bump catversion for?
regards, tom lane
On Wed, Jun 23, 2010 at 10:30 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Robert Haas <robertmhaas@gmail.com> writes: >> I can reproduce this, here. The problem seems to be that plpython >> only build either plpython2.so or plython3.so, but both languages >> expect a call handler called plython_call_handler. So once we load >> the shared library for one language, the other language just grabs the >> same call handler. > > We had better fix that --- I can definitely foresee installations > wanting to use both plpython2 and plpython3. This'd require a change in > the default contents of pg_pltemplate, though. Does it seem important > enough to bump catversion for? Yeah, I think so. As for using both plpython2 and plpython3, it looks to me like right now you can only use one or the other. I think if you somehow manage to install both, you're really just getting the same one twice (I have not tested this, however). -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company
On Wed, Jun 23, 2010 at 10:49 AM, Robert Haas <robertmhaas@gmail.com> wrote: > On Wed, Jun 23, 2010 at 10:30 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> Robert Haas <robertmhaas@gmail.com> writes: >>> I can reproduce this, here. The problem seems to be that plpython >>> only build either plpython2.so or plython3.so, but both languages >>> expect a call handler called plython_call_handler. So once we load >>> the shared library for one language, the other language just grabs the >>> same call handler. >> >> We had better fix that --- I can definitely foresee installations >> wanting to use both plpython2 and plpython3. This'd require a change in >> the default contents of pg_pltemplate, though. Does it seem important >> enough to bump catversion for? > > Yeah, I think so. As for using both plpython2 and plpython3, it looks > to me like right now you can only use one or the other. I think if > you somehow manage to install both, you're really just getting the > same one twice (I have not tested this, however). So, what's the right thing to do here? Should we just fix it so that creating the second language always fails, or should we try to make it possible for both of them to coexist (which is probably a lot more work)? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company
On ons, 2010-06-23 at 07:17 -0400, Robert Haas wrote: > I can reproduce this, here. The problem seems to be that plpython > only build either plpython2.so or plython3.so, but both languages > expect a call handler called plython_call_handler. So once we load > the shared library for one language, the other language just grabs the > same call handler. The problem is apparently that when CREATE LANGUAGE creates a language from a pg_pltemplate entry, it creates the proname from the tmplhandler name, and if it finds a fitting proname entry already, it used that one. So when you create plpython2 first and plpython3 second, the pg_language entries of the latter point to the pg_proc entries of the former. If you fix that up manually (create additional pg_proc entries and fix the pg_language entries to point there), it works better.
Peter Eisentraut <peter_e@gmx.net> writes:
> The problem is apparently that when CREATE LANGUAGE creates a language
> from a pg_pltemplate entry, it creates the proname from the tmplhandler
> name, and if it finds a fitting proname entry already, it used that one.
> So when you create plpython2 first and plpython3 second, the pg_language
> entries of the latter point to the pg_proc entries of the former.
> If you fix that up manually (create additional pg_proc entries and fix
> the pg_language entries to point there), it works better.
The fix ought to be to change the function nmes used by plpython3 ...
regards, tom lane
On fre, 2010-06-25 at 10:17 -0400, Tom Lane wrote: > Peter Eisentraut <peter_e@gmx.net> writes: > > The problem is apparently that when CREATE LANGUAGE creates a language > > from a pg_pltemplate entry, it creates the proname from the tmplhandler > > name, and if it finds a fitting proname entry already, it used that one. > > So when you create plpython2 first and plpython3 second, the pg_language > > entries of the latter point to the pg_proc entries of the former. > > > If you fix that up manually (create additional pg_proc entries and fix > > the pg_language entries to point there), it works better. > > The fix ought to be to change the function nmes used by plpython3 ... Right. What shall we do about the catversion?
Attachment
Peter Eisentraut <peter_e@gmx.net> writes:
> On fre, 2010-06-25 at 10:17 -0400, Tom Lane wrote:
>> The fix ought to be to change the function nmes used by plpython3 ...
> Right. What shall we do about the catversion?
You could argue it either way. The number of beta testers with
plpython3 installations is probably very small, so I'm kinda leaning to
just changing the code without a catversion bump. OTOH, if we want to
encourage testing of pg_upgrade ...
regards, tom lane
> You could argue it either way. The number of beta testers with
> plpython3 installations is probably very small, so I'm kinda leaning to
> just changing the code without a catversion bump. OTOH, if we want to
> encourage testing of pg_upgrade ...
FWIW, the last bump has led to a lot of testing of pgupgrade.
-- -- Josh Berkus PostgreSQL Experts Inc.
http://www.pgexperts.com
On Wednesday 23 June 2010 16:30:54 Tom Lane wrote: > Robert Haas <robertmhaas@gmail.com> writes: > > I can reproduce this, here. The problem seems to be that plpython > > only build either plpython2.so or plython3.so, but both languages > > expect a call handler called plython_call_handler. So once we load > > the shared library for one language, the other language just grabs the > > same call handler. > > We had better fix that --- I can definitely foresee installations > wanting to use both plpython2 and plpython3. This'd require a change in > the default contents of pg_pltemplate, though. Does it seem important > enough to bump catversion for? Has anybody actually researched if it is safe to run python2 and python3 in the same address space? I wouldnt be surprised at all if that where problematic. Andres
On fre, 2010-06-25 at 23:44 +0200, Andres Freund wrote: > Has anybody actually researched if it is safe to run python2 and > python3 in the same address space? You can't run plpython2 and plpython3 in the same session, because the libraries are loaded with dlopen(RTLD_GLOBAL) (with RTLD_LOCAL it would apparently work). But you can use them in different sessions on the same database, for example.
On Fri, Jun 25, 2010 at 2:49 PM, Peter Eisentraut <peter_e@gmx.net> wrote: > On fre, 2010-06-25 at 10:17 -0400, Tom Lane wrote: >> Peter Eisentraut <peter_e@gmx.net> writes: >> > The problem is apparently that when CREATE LANGUAGE creates a language >> > from a pg_pltemplate entry, it creates the proname from the tmplhandler >> > name, and if it finds a fitting proname entry already, it used that one. >> > So when you create plpython2 first and plpython3 second, the pg_language >> > entries of the latter point to the pg_proc entries of the former. >> >> > If you fix that up manually (create additional pg_proc entries and fix >> > the pg_language entries to point there), it works better. >> >> The fix ought to be to change the function nmes used by plpython3 ... > > Right. What shall we do about the catversion? We should go ahead and apply this, either with (my vote) or without (Tom's vote) a catversion bump. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company
Josh Berkus wrote: > > > You could argue it either way. The number of beta testers with > > plpython3 installations is probably very small, so I'm kinda leaning to > > just changing the code without a catversion bump. OTOH, if we want to > > encourage testing of pg_upgrade ... > > FWIW, the last bump has led to a lot of testing of pgupgrade. And fixes, that will appear in 9.0 beta3. :-) Most fixes were related to platform compile portability. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + None of us is going to be here forever. +
Bruce Momjian <bruce@momjian.us> writes:
> Josh Berkus wrote:
>>> You could argue it either way. The number of beta testers with
>>> plpython3 installations is probably very small, so I'm kinda leaning to
>>> just changing the code without a catversion bump. OTOH, if we want to
>>> encourage testing of pg_upgrade ...
>>
>> FWIW, the last bump has led to a lot of testing of pgupgrade.
> And fixes, that will appear in 9.0 beta3. :-) Most fixes were related
> to platform compile portability.
Well, if you think that pg_upgrade has changed materially since beta2,
that would be a good argument for getting some fresh testing for it,
which in turn argues for doing the catversion bump here.
regards, tom lane
Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > Josh Berkus wrote: > >>> You could argue it either way. The number of beta testers with > >>> plpython3 installations is probably very small, so I'm kinda leaning to > >>> just changing the code without a catversion bump. OTOH, if we want to > >>> encourage testing of pg_upgrade ... > >> > >> FWIW, the last bump has led to a lot of testing of pgupgrade. > > > And fixes, that will appear in 9.0 beta3. :-) Most fixes were related > > to platform compile portability. > > Well, if you think that pg_upgrade has changed materially since beta2, > that would be a good argument for getting some fresh testing for it, > which in turn argues for doing the catversion bump here. Attached are the changes since beta2; they are pretty minor. The good news is I think all reporters eventually got it working. I assume using it for beta3 would allow it work even better, and once you have use it once, using it again is simple. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + None of us is going to be here forever. + /check.c /dump.c /function.c /option.c /pg_upgrade.c /pg_upgrade.h /version.c /version_old_8_3.c momjian Have pg_upgrade create its output files in the current directory, rather than in a subdirectory of the $HOME directory, or $TMP in Windows. --- /pg_upgrade.h momjian Update pg_upgrade C comment about cwd. --- /pg_upgrade.h /relfilenode.c momjian Properly define pg_upgrade global variable, per bug report from Chris Ruprecht on Mac (64 bit). --- /option.c momjian Fix storage of getopt() return, should be 'int', for pg_upgrade. Steve Singer --- /tablespace.c momjian Fix pg_upgrade to remove malloc(0) call. --- /dump.c /option.c /pg_upgrade.h /server.c momjian Fix pg_upgrade's use of pg_ctl on Win32 to not send command and sever output to the same file, because it is impossible. Also set user name for pg_dumpall in pg_upgrade. --- /check.c /pg_upgrade.c /server.c momjian Add username designations to all pg_upgrade utility calls that support it. ---
On fre, 2010-06-25 at 18:57 -0400, Peter Eisentraut wrote: > On fre, 2010-06-25 at 23:44 +0200, Andres Freund wrote: > > Has anybody actually researched if it is safe to run python2 and > > python3 in the same address space? > > You can't run plpython2 and plpython3 in the same session, because the > libraries are loaded with dlopen(RTLD_GLOBAL) (with RTLD_LOCAL it would > apparently work). But you can use them in different sessions on the > same database, for example. By the way, I thought there had been some discussion about this in the past, but I couldn't find it ... Why are we using RTLD_GLOBAL?
Peter Eisentraut <peter_e@gmx.net> writes: > Why are we using RTLD_GLOBAL? Some guy named Eisentraut wanted it for plpython: http://archives.postgresql.org/pgsql-hackers/2001-05/msg00563.php http://archives.postgresql.org/pgsql-committers/2001-05/msg00283.php Possibly that no longer applies, though. In general it seems like our usage of loadable modules treats them all as independent objects, so that not using RTLD_GLOBAL would be a more sensible policy. If it won't break any of the existing PLs then I'm fine with removing it. regards, tom lane