Thread: pl/perl and recent perl versions - failing to load internal modules
Hi, I've been trying out PostgreSQL 9.3 with pl/perl built against Ubuntu 14.04 LTS' Perl 5.18 (Sourced from apt.postgresql.org) Maybe I'm doing something wrong, but it appears that plperl has become completely useless, as it can't load any new modules,but modern Perl versions have refactored the language into lots of small modules that are automatically loaded asrequired. Except they can't be loaded as required. For instance, a fairly simple regex will try to load the modules: utf8, utf8_heavy & re Since it is forbidden from loading anything, it will fail. You can still use the unrestricted Perl module, plperlu, but this requires superuser permissions to use, and in the pastits use has been discouraged. I've experimented with using the plperl.on_init configuration option to try and include all required modules, but this requirestrial-and-error and can easily cause new failures to appear if a stored procedure is changed a little. Has anyone else encountered this? Am I doing something completely wrong? Thanks, Toby
On Mon, Jun 30, 2014 at 7:47 PM, Toby Corkindale
<toby.corkindale@strategicdata.com.au> wrote:
> Hi,
> I've been trying out PostgreSQL 9.3 with pl/perl built against Ubuntu 14.04 LTS' Perl 5.18
> (Sourced from apt.postgresql.org)
>
> Maybe I'm doing something wrong, but it appears that plperl has become completely useless, as it can't load any new
modules,but modern Perl versions have refactored the language into lots of small modules that are automatically loaded
asrequired. Except they can't be loaded as required.
>
> For instance, a fairly simple regex will try to load the modules: utf8, utf8_heavy & re
> Since it is forbidden from loading anything, it will fail.
pl/perl should be preloading those modules via "my $a=chr(0x100);
return $a =~ /\\xa9/i". Do you have some example pl/perl code that is
broke? It seems to work for me (I only have 5.20 installed, on this
machine, but I've used 5.18 on 9.2 and 9.3):
$ perl -v
This is perl 5, version 20, subversion 0 (v5.20.0) built for
x86_64-linux-thread-multi
...
$ psql ...
baroque=> select version();
version
─────────────────────────────────────────────────────────────────────────────────────────────────────────
PostgreSQL 9.3.4 on x86_64-unknown-linux-gnu, compiled by gcc (GCC)
4.9.0 20140604 (prerelease), 64-bit
(1 row)
Time: 1.033 ms
baroque=> do $$my $a=chr(0x100); return $a =~ /\\xa9/i$$ language plperl;
DO
Time: 1.161 ms
Hi Alex,
=> do $$ "\N{U+263A}" =~ /[[:punct:]]/ $$ language plperl;
ERROR: Unable to load utf8.pm into plperl at line 1.
BEGIN failed--compilation aborted.
CONTEXT: PL/Perl anonymous code block
----- Original Message -----
> From: "Alex Hunsaker" <badalex@gmail.com>
> To: "Toby Corkindale" <toby.corkindale@strategicdata.com.au>
> Cc: "pgsql-general" <pgsql-general@postgresql.org>
> Sent: Wednesday, 2 July, 2014 9:28:52 AM
> Subject: Re: [GENERAL] pl/perl and recent perl versions - failing to load internal modules
>
> On Mon, Jun 30, 2014 at 7:47 PM, Toby Corkindale
> <toby.corkindale@strategicdata.com.au> wrote:
> > Hi,
> > I've been trying out PostgreSQL 9.3 with pl/perl built against Ubuntu 14.04
> > LTS' Perl 5.18
> > (Sourced from apt.postgresql.org)
> >
> > Maybe I'm doing something wrong, but it appears that plperl has become
> > completely useless, as it can't load any new modules, but modern Perl
> > versions have refactored the language into lots of small modules that are
> > automatically loaded as required. Except they can't be loaded as required.
> >
> > For instance, a fairly simple regex will try to load the modules: utf8,
> > utf8_heavy & re
> > Since it is forbidden from loading anything, it will fail.
>
> pl/perl should be preloading those modules via "my $a=chr(0x100);
> return $a =~ /\\xa9/i". Do you have some example pl/perl code that is
> broke? It seems to work for me (I only have 5.20 installed, on this
> machine, but I've used 5.18 on 9.2 and 9.3):
>
> $ perl -v
>
> This is perl 5, version 20, subversion 0 (v5.20.0) built for
> x86_64-linux-thread-multi
> ...
>
> $ psql ...
> baroque=> select version();
> version
> ─────────────────────────────────────────────────────────────────────────────────────────────────────────
> PostgreSQL 9.3.4 on x86_64-unknown-linux-gnu, compiled by gcc (GCC)
> 4.9.0 20140604 (prerelease), 64-bit
> (1 row)
>
> Time: 1.033 ms
>
> baroque=> do $$my $a=chr(0x100); return $a =~ /\\xa9/i$$ language plperl;
> DO
> Time: 1.161 ms
>
Hi Alex,
your example (chr(0x100) =~ /\\xa9/) works on my instance (pg 9.3.4, plperl 5.18)
However the following code fails:
=> do $$ "\N{U+263A}" =~ /[[:punct:]]/$$ language plperl;
ERROR: Unable to load utf8.pm into plperl at line 1.
BEGIN failed--compilation aborted.
CONTEXT: PL/Perl anonymous code block
Oddly, even if I add utf8 to plperl.oninit, I then get an error about being unable to load the 're' (regex) module. I
don'tunderstand how the example you gave does manage to work!
----- Original Message -----
> From: "Alex Hunsaker" <badalex@gmail.com>
> To: "Toby Corkindale" <toby.corkindale@strategicdata.com.au>
> Cc: "pgsql-general" <pgsql-general@postgresql.org>
> Sent: Wednesday, 2 July, 2014 9:28:52 AM
> Subject: Re: [GENERAL] pl/perl and recent perl versions - failing to load internal modules
>
> On Mon, Jun 30, 2014 at 7:47 PM, Toby Corkindale
> <toby.corkindale@strategicdata.com.au> wrote:
> > Hi,
> > I've been trying out PostgreSQL 9.3 with pl/perl built against Ubuntu 14.04
> > LTS' Perl 5.18
> > (Sourced from apt.postgresql.org)
> >
> > Maybe I'm doing something wrong, but it appears that plperl has become
> > completely useless, as it can't load any new modules, but modern Perl
> > versions have refactored the language into lots of small modules that are
> > automatically loaded as required. Except they can't be loaded as required.
> >
> > For instance, a fairly simple regex will try to load the modules: utf8,
> > utf8_heavy & re
> > Since it is forbidden from loading anything, it will fail.
>
> pl/perl should be preloading those modules via "my $a=chr(0x100);
> return $a =~ /\\xa9/i". Do you have some example pl/perl code that is
> broke? It seems to work for me (I only have 5.20 installed, on this
> machine, but I've used 5.18 on 9.2 and 9.3):
>
> $ perl -v
>
> This is perl 5, version 20, subversion 0 (v5.20.0) built for
> x86_64-linux-thread-multi
> ...
>
> $ psql ...
> baroque=> select version();
> version
> ─────────────────────────────────────────────────────────────────────────────────────────────────────────
> PostgreSQL 9.3.4 on x86_64-unknown-linux-gnu, compiled by gcc (GCC)
> 4.9.0 20140604 (prerelease), 64-bit
> (1 row)
>
> Time: 1.033 ms
>
> baroque=> do $$my $a=chr(0x100); return $a =~ /\\xa9/i$$ language plperl;
> DO
> Time: 1.161 ms
>
On Tue, Jul 1, 2014 at 6:39 PM, Toby Corkindale
<toby.corkindale@strategicdata.com.au> wrote:
> Hi Alex,
> your example (chr(0x100) =~ /\\xa9/) works on my instance (pg 9.3.4, plperl 5.18)
> However the following code fails:
> => do $$ "\N{U+263A}" =~ /[[:punct:]]/$$ language plperl;
> ERROR: Unable to load utf8.pm into plperl at line 1.
> BEGIN failed--compilation aborted.
> CONTEXT: PL/Perl anonymous code block
>
> Oddly, even if I add utf8 to plperl.oninit, I then get an error about being unable to load the 're' (regex) module. I
don'tunderstand how the example you gave does manage to work!
Strange, seems to works for me with 5.20:
=> do $$ "\N{U+263A}" =~ /[[:punct:]]/$$ language plperl;
DO
Time: 12.928 ms
=> show plperl.on_init;
plperl.on_init
────────────────
(1 row)
Time: 0.786 ms
=> show plperl.on_plperl_init;
plperl.on_plperl_init
───────────────────────
(1 row)
One thing you might try is plperl.on_plperl_init = "require
'utf8_heavy.pl'; require unicore/Heavy.pl'; or maybe just
plperl.on_plperl_init = 'use Unicode::UCD;'
If that still fails, You might find the attached useful, it tires to
preload all the unicore files. You should be able to stick it
somewhere and plperl.on_plperl_init = '/path/to/unicore_preload.pm'.
However, hopefully one of the first suggestions will work..
Attachment
On Tue, Jul 1, 2014 at 7:31 PM, Alex Hunsaker <badalex@gmail.com> wrote:
> On Tue, Jul 1, 2014 at 6:39 PM, Toby Corkindale
> <toby.corkindale@strategicdata.com.au> wrote:
>> Hi Alex,
>> However the following code fails:
>> => do $$ "\N{U+263A}" =~ /[[:punct:]]/$$ language plperl;
>> ERROR: Unable to load utf8.pm into plperl at line 1.
>> BEGIN failed--compilation aborted.
>> CONTEXT: PL/Perl anonymous code block
> One thing you might try is plperl.on_plperl_init = "require
> 'utf8_heavy.pl'; require unicore/Heavy.pl'; or maybe just
> plperl.on_plperl_init = 'use Unicode::UCD;'
If you are up to recompiling postgres, you might try this patch. Its
untested, but the basic idea is to always allow require 'utf8.pm' to
work in plperl. Near as I can tell utf8.pm is a strange package, when
we look to see if we already have it loaded it, we can't find it for
example. Which would explain why plperl.on_init = ' use utf8;' does
not work for you. I don't have the time to dig through a bunch of perl
versions to find out whats going on with utf8.pm.
I suspect this will make it fail on loading a different module (my gut
says utf8 is going to want to load unicore stuff once its loaded) but
it would be interesting to see nonetheless.