Thread: pg_upgrade from 12 to 13 failes with plpython2

pg_upgrade from 12 to 13 failes with plpython2

From
Marcin Giedz
Date:
Hi all, trying to performe upgrade from 12 to 13 installed from Centos8 repo gives such error:

cat loadable_libraries.txt
could not load library "$libdir/plpython2": ERROR:  could not access file "$libdir/plpython2": No such file or directory

digging around:
1.
drop extension plpythonu;
ERROR:  extension "plpythonu" does not exist

2.
postgres=# select * from pg_pltemplate ;
  tmplname  | tmpltrusted | tmpldbacreate |      tmplhandler       |        tmplinline        |    tmplvalidator    |    tmpllibrary    | tmplacl
------------+-------------+---------------+------------------------+--------------------------+---------------------+-------------------+---------
 plpgsql    | t           | t             | plpgsql_call_handler   | plpgsql_inline_handler   | plpgsql_validator   | $libdir/plpgsql   |
 pltcl      | t           | t             | pltcl_call_handler     |                          |                     | $libdir/pltcl     |
 pltclu     | f           | f             | pltclu_call_handler    |                          |                     | $libdir/pltcl     |
 plperl     | t           | t             | plperl_call_handler    | plperl_inline_handler    | plperl_validator    | $libdir/plperl    |
 plperlu    | f           | f             | plperlu_call_handler   | plperlu_inline_handler   | plperlu_validator   | $libdir/plperl    |
 plpythonu  | f           | f             | plpython_call_handler  | plpython_inline_handler  | plpython_validator  | $libdir/plpython2 |
 plpython2u | f           | f             | plpython2_call_handler | plpython2_inline_handler | plpython2_validator | $libdir/plpython2 |
 plpython3u | f           | f             | plpython3_call_handler | plpython3_inline_handler | plpython3_validator | $libdir/plpython3 |
(8 rows)

3.
postgres=# select * from pg_language ;
  oid  | lanname  | lanowner | lanispl | lanpltrusted | lanplcallfoid | laninline | lanvalidator | lanacl
-------+----------+----------+---------+--------------+---------------+-----------+--------------+--------
    12 | internal |       10 | f       | f            |             0 |         0 |         2246 |
    13 | c        |       10 | f       | f            |             0 |         0 |         2247 |
    14 | sql      |       10 | f       | t            |             0 |         0 |         2248 |
 14177 | plpgsql  |       10 | t       | t            |         14174 |     14175 |        14176 |
(4 rows)


4.
postgres=# select  DISTINCT l.lanname as function_language
postgres-# from pg_proc p
postgres-# left join pg_language l on p.prolang = l.oid;
 function_language
-------------------
 c
 sql
 internal
(3 rows)

5.

postgres=# SELECT oid::regprocedure
postgres-# FROM pg_catalog.pg_proc
postgres-# WHERE probin = '$libdir/plpython2';
 oid
-----
(0 rows)



Question: where is plpython2 comming from during pg_upgrade? Is this pg_upgrade bug or something else?


Thx
Marcin


Re: pg_upgrade from 12 to 13 failes with plpython2

From
Devrim Gündüz
Date:
Hi,

On Tue, 2020-11-17 at 21:00 +0100, Marcin Giedz wrote:
> Hi all, trying to performe upgrade from 12 to 13 installed from
> Centos8 repo gives such error:
>
> cat loadable_libraries.txt
> could not load library "$libdir/plpython2": ERROR: could not access
> file "$libdir/plpython2": No such file or directory

I dropped PL/Python support along with PostgreSQL 13 RPMs.

Regards,
--
Devrim Gündüz
Open Source Solution Architect, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

Attachment

Re: pg_upgrade from 12 to 13 failes with plpython2

From
Tom Lane
Date:
Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= <devrim@gunduz.org> writes:
> On Tue, 2020-11-17 at 21:00 +0100, Marcin Giedz wrote:
>> Hi all, trying to performe upgrade from 12 to 13 installed from
>> Centos8 repo gives such error: 
>> 
>> cat loadable_libraries.txt 
>> could not load library "$libdir/plpython2": ERROR: could not access
>> file "$libdir/plpython2": No such file or directory 

> I dropped PL/Python support along with PostgreSQL 13 RPMs.

Yeah, I tried to stir up some interest in having a cleaner python 2
to python 3 migration path, but there wasn't any.  Best bet probably
is to convert your plpython[2] functions to plpython3 before you
upgrade.

            regards, tom lane



Re: pg_upgrade from 12 to 13 failes with plpython2

From
Adrian Klaver
Date:
On 11/17/20 12:06 PM, Devrim Gündüz wrote:
> 
> Hi,
> 
> On Tue, 2020-11-17 at 21:00 +0100, Marcin Giedz wrote:
>> Hi all, trying to performe upgrade from 12 to 13 installed from
>> Centos8 repo gives such error:
>>
>> cat loadable_libraries.txt
>> could not load library "$libdir/plpython2": ERROR: could not access
>> file "$libdir/plpython2": No such file or directory
> 
> I dropped PL/Python support along with PostgreSQL 13 RPMs.

This was announced where and when?

> 
> Regards,
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com



Re: pg_upgrade from 12 to 13 failes with plpython2

From
Devrim Gündüz
Date:
Hi,

On Tue, 2020-11-17 at 12:18 -0800, Adrian Klaver wrote:
> This was announced where and when?

https://www.postgresql.org/message-id/333f3aa334ba93019c75fffaec373f2bf4275d28.camel%40gunduz.org

Regards,
--
Devrim Gündüz
Open Source Solution Architect, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

Attachment

Re: pg_upgrade from 12 to 13 failes with plpython2

From
Adrian Klaver
Date:
On 11/17/20 12:49 PM, Devrim Gündüz wrote:
> 
> Hi,
> 
> On Tue, 2020-11-17 at 12:18 -0800, Adrian Klaver wrote:
>> This was announced where and when?
> 
> https://www.postgresql.org/message-id/333f3aa334ba93019c75fffaec373f2bf4275d28.camel%40gunduz.org

So to be clear what was dropped was plpythonu, which means plpython2u. 
plpython3u still exists, correct?

It would be nice to mention this on --announce and here as this still 
exists:

https://www.postgresql.org/docs/13/plpython-python23.html

"

Existing users and users who are currently not interested in Python 3 
use the language name plpythonu and don't have to change anything for 
the foreseeable future. It is recommended to gradually “future-proof” 
the code via migration to Python 2.6/2.7 to simplify the eventual 
migration to Python 3.

In practice, many PL/Python functions will migrate to Python 3 with few 
or no changes.
"

> 
> Regards,
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com



Re: pg_upgrade from 12 to 13 failes with plpython2

From
Devrim Gündüz
Date:
Hi,

On Tue, 2020-11-17 at 13:18 -0800, Adrian Klaver wrote:
> > https://www.postgresql.org/message-id/333f3aa334ba93019c75fffaec373f2bf4275d28.camel%40gunduz.org
>
> So to be clear what was dropped was plpythonu, which means
> plpython2u. plpython3u still exists, correct?

Right.

Regards,
--
Devrim Gündüz
Open Source Solution Architect, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

Attachment

Re: pg_upgrade from 12 to 13 failes with plpython2

From
Tom Lane
Date:
Adrian Klaver <adrian.klaver@aklaver.com> writes:
> It would be nice to mention this on --announce and here as this still 
> exists:
> https://www.postgresql.org/docs/13/plpython-python23.html

You're confusing what the source code can do (which is what the
manual documents) versus what individual packagers choose to support.
The packagers frequently don't have a lot of choice in the matter;
once their platform drops python2, they can't support plpython2.

            regards, tom lane



Re: pg_upgrade from 12 to 13 failes with plpython2

From
Adrian Klaver
Date:
On 11/17/20 1:23 PM, Tom Lane wrote:
> Adrian Klaver <adrian.klaver@aklaver.com> writes:
>> It would be nice to mention this on --announce and here as this still
>> exists:
>> https://www.postgresql.org/docs/13/plpython-python23.html
> 
> You're confusing what the source code can do (which is what the
> manual documents) versus what individual packagers choose to support.
> The packagers frequently don't have a lot of choice in the matter;
> once their platform drops python2, they can't support plpython2.

But CentOS/RH have not dropped Python 2, they have just made Python 3 
the default:

https://www.liquidweb.com/kb/how-to-install-python-on-centos-8/

> 
>             regards, tom lane
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com



Re: pg_upgrade from 12 to 13 failes with plpython2

From
Devrim Gündüz
Date:
Hi,

On Tue, 2020-11-17 at 16:23 -0500, Tom Lane wrote:
> You're confusing what the source code can do (which is what the
> manual documents) versus what individual packagers choose to support.
> The packagers frequently don't have a lot of choice in the matter;
> once their platform drops python2, they can't support plpython2.

Well, CentOS 8 "supports" PY2, however given that Christoph also
dropped PY2 support in Debian packages as of PostgreSQL 12 [1], and
given that *Python 2 is unsupported* anyway, I just wanted to drop
support at this point.

FWIW, older PostgreSQL major versions still have PY 2 support. This is
for PostgreSQL 13 only.

[1] : https://www.postgresql.org/message-id/20200522080114.GA449430%40msg.df7cb.de

Regards,
--
Devrim Gündüz
Open Source Solution Architect, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

Attachment

Re: pg_upgrade from 12 to 13 failes with plpython2

From
Adrian Klaver
Date:
On 11/17/20 2:17 PM, Devrim Gündüz wrote:
> 
> Hi,
> 
> On Tue, 2020-11-17 at 16:23 -0500, Tom Lane wrote:
>> You're confusing what the source code can do (which is what the
>> manual documents) versus what individual packagers choose to support.
>> The packagers frequently don't have a lot of choice in the matter;
>> once their platform drops python2, they can't support plpython2.
> 

First let me say:

1) I don't use Python 2 anymore.

2) I have converted my plpython(2)u functions to plpythonu3u

3) I don't use RH family distros.

> Well, CentOS 8 "supports" PY2, however given that Christoph also
> dropped PY2 support in Debian packages as of PostgreSQL 12 [1], and
> given that *Python 2 is unsupported* anyway, I just wanted to drop
> support at this point.

Read that sentence again and see the inherent disconnect between the 
beginning and end.

As a packager you are in charge of how the packaging is done. Still 
announcing a change that effectively nullifies the documentation would 
to me be something that should be announced somewhere else than a list 
that I'm guessing 99% of the users don't read.

> 
> FWIW, older PostgreSQL major versions still have PY 2 support. This is
> for PostgreSQL 13 only.

> 
> [1] : https://www.postgresql.org/message-id/20200522080114.GA449430%40msg.df7cb.de
> 
> Regards,
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com



Re: pg_upgrade from 12 to 13 failes with plpython2

From
Marcin Giedz
Date:
but my question still remains the same - what causes pg_upgrade failure - are functions the reason? what I did was to delete these 2 rows from pg_pltemplate as I thought this may help:

postgres=# delete from pg_pltemplate where tmplname = 'plpython2u';
DELETE 1
postgres=# delete from pg_pltemplate where tmplname = 'plpythonu';
DELETE 1


but pg_upgrade still complains about plpython2:

cat loadable_libraries.txt
could not load library "$libdir/plpython2": ERROR:  could not access file "$libdir/plpython2": No such file or directory
In database: alaxx
In database: template1



marcin@carbon2:~$ psql -U pgsql -h 192.168.89.64 alaxx
psql (12.4 (Ubuntu 12.4-1))
Type "help" for help.

argosrm=# select * from pg_pltemplate ;
  tmplname  | tmpltrusted | tmpldbacreate |      tmplhandler       |        tmplinline        |    tmplvalidator    |    tmpllibrary    | tmplacl
------------+-------------+---------------+------------------------+--------------------------+---------------------+-------------------+---------
 plpgsql    | t           | t             | plpgsql_call_handler   | plpgsql_inline_handler   | plpgsql_validator   | $libdir/plpgsql   |
 pltcl      | t           | t             | pltcl_call_handler     |                          |                     | $libdir/pltcl     |
 pltclu     | f           | f             | pltclu_call_handler    |                          |                     | $libdir/pltcl     |
 plperl     | t           | t             | plperl_call_handler    | plperl_inline_handler    | plperl_validator    | $libdir/plperl    |
 plperlu    | f           | f             | plperlu_call_handler   | plperlu_inline_handler   | plperlu_validator   | $libdir/plperl    |
 plpython3u | f           | f             | plpython3_call_handler | plpython3_inline_handler | plpython3_validator | $libdir/plpython3 |
(6 rows)

argosrm=# \q
marcin@carbon2:~$ psql -U pgsql -h 192.168.89.64 template1
psql (12.4 (Ubuntu 12.4-1))
Type "help" for help.

template1=# select * from pg_pltemplate ;
  tmplname  | tmpltrusted | tmpldbacreate |      tmplhandler       |        tmplinline        |    tmplvalidator    |    tmpllibrary    | tmplacl
------------+-------------+---------------+------------------------+--------------------------+---------------------+-------------------+---------
 plpgsql    | t           | t             | plpgsql_call_handler   | plpgsql_inline_handler   | plpgsql_validator   | $libdir/plpgsql   |
 pltcl      | t           | t             | pltcl_call_handler     |                          |                     | $libdir/pltcl     |
 pltclu     | f           | f             | pltclu_call_handler    |                          |                     | $libdir/pltcl     |
 plperl     | t           | t             | plperl_call_handler    | plperl_inline_handler    | plperl_validator    | $libdir/plperl    |
 plperlu    | f           | f             | plperlu_call_handler   | plperlu_inline_handler   | plperlu_validator   | $libdir/plperl    |
 plpython3u | f           | f             | plpython3_call_handler | plpython3_inline_handler | plpython3_validator | $libdir/plpython3 |
(6 rows)


what can I do next ?


Thx
Marcin


Od: "Adrian Klaver" <adrian.klaver@aklaver.com>
Do: "Devrim Gündüz" <devrim@gunduz.org>, "Tom Lane" <tgl@sss.pgh.pa.us>
DW: "Marcin Giedz" <marcin.giedz@arise.pl>, "pgsql-general" <pgsql-general@lists.postgresql.org>
Wysłane: wtorek, 17 listopad 2020 23:30:44
Temat: Re: pg_upgrade from 12 to 13 failes with plpython2

On 11/17/20 2:17 PM, Devrim Gündüz wrote:
>
> Hi,
>
> On Tue, 2020-11-17 at 16:23 -0500, Tom Lane wrote:
>> You're confusing what the source code can do (which is what the
>> manual documents) versus what individual packagers choose to support.
>> The packagers frequently don't have a lot of choice in the matter;
>> once their platform drops python2, they can't support plpython2.
>

First let me say:

1) I don't use Python 2 anymore.

2) I have converted my plpython(2)u functions to plpythonu3u

3) I don't use RH family distros.

> Well, CentOS 8 "supports" PY2, however given that Christoph also
> dropped PY2 support in Debian packages as of PostgreSQL 12 [1], and
> given that *Python 2 is unsupported* anyway, I just wanted to drop
> support at this point.

Read that sentence again and see the inherent disconnect between the
beginning and end.

As a packager you are in charge of how the packaging is done. Still
announcing a change that effectively nullifies the documentation would
to me be something that should be announced somewhere else than a list
that I'm guessing 99% of the users don't read.

>
> FWIW, older PostgreSQL major versions still have PY 2 support. This is
> for PostgreSQL 13 only.

>
> [1] : https://www.postgresql.org/message-id/20200522080114.GA449430%40msg.df7cb.de
>
> Regards,
>


--
Adrian Klaver
adrian.klaver@aklaver.com

--
Pozdrawiam
Marcin Giedz
Wiceprezes Zarządu

ARISE Sp. z o.o.
mob. +48 606 673 977
mail: marcin.giedz@arise.pl

Al. Solidarności 117
00-140 Warszawa
tel. +48 (22) 440 56 20
fax  +48 (22) 440 56 22
http://www.arise.pl

Grupa ARISE: ARISE Sp. z o.o. (podmiot dominujący), ARISE Services S.A., Al. Solidarności 117, 00-140 Warszawa, Sąd Rejonowy dla m. st. Warszawy w Warszawie XII Wydział Gospodarczy Krajowego Rejestru Sądowego, KRS 0000316860, REGON 141595449, NIP 5272590610, kapitał zakładowy wpłacony 250,000.00 zł.

Re: pg_upgrade from 12 to 13 failes with plpython2

From
Magnus Hagander
Date:
On Wed, Nov 18, 2020 at 8:11 AM Marcin Giedz <marcin.giedz@arise.pl> wrote:
>
> but my question still remains the same - what causes pg_upgrade failure - are functions the reason? what I did was to
deletethese 2 rows from pg_pltemplate as I thought this may help:
 
>
> postgres=# delete from pg_pltemplate where tmplname = 'plpython2u';
> DELETE 1
> postgres=# delete from pg_pltemplate where tmplname = 'plpythonu';
> DELETE 1
>
>
> but pg_upgrade still complains about plpython2:
>
> cat loadable_libraries.txt
> could not load library "$libdir/plpython2": ERROR:  could not access file "$libdir/plpython2": No such file or
directory
> In database: alaxx
> In database: template1


It's not the template that's interesting, it's the language itself you
need to drop. Log into each database and try to do that, and you will
get something like this if you still have functions using it:
postgres=# DROP LANGUAGE plpython2u;
ERROR:  cannot drop language plpython2u because other objects depend on it
DETAIL:  function testfunc() depends on language plpython2u
HINT:  Use DROP ... CASCADE to drop the dependent objects too.

If you have no functions using it, it will just go away, and once you
have dropped it in both databases you should be good to go.

And of course, if there are functions depending on it, you should
rebuild those on plpython3u before you drop plpython2u (or drop the
functions if they're not in use).

-- 
 Magnus Hagander
 Me: https://www.hagander.net/
 Work: https://www.redpill-linpro.com/



Re: pg_upgrade from 12 to 13 failes with plpython2

From
Marcin Giedz
Date:
right, I had one function relaying on plpython2u so I changed it... but the again pg_upgrade claims error with python:

cat loadable_libraries.txt
could not load library "$libdir/plpython2": ERROR:  could not access file "$libdir/plpython2": No such file or directory
In database: argosrm
In database: template1




marcin@carbon2:~$ psql -U pgsql -h 192.168.89.64 postgres
psql (12.4 (Ubuntu 12.4-1), server 12.5)
Type "help" for help.

postgres=# select * from pg_language ;
  oid  | lanname  | lanowner | lanispl | lanpltrusted | lanplcallfoid | laninline | lanvalidator | lanacl
-------+----------+----------+---------+--------------+---------------+-----------+--------------+--------
    12 | internal |       10 | f       | f            |             0 |         0 |         2246 |
    13 | c        |       10 | f       | f            |             0 |         0 |         2247 |
    14 | sql      |       10 | f       | t            |             0 |         0 |         2248 |
 14177 | plpgsql  |       10 | t       | t            |         14174 |     14175 |        14176 |
(4 rows)

postgres=# \c argosrm
psql (12.4 (Ubuntu 12.4-1), server 12.5)
You are now connected to database "argosrm" as user "pgsql".
argosrm=# select * from pg_language ;
   oid   |  lanname   | lanowner | lanispl | lanpltrusted | lanplcallfoid | laninline | lanvalidator | lanacl
---------+------------+----------+---------+--------------+---------------+-----------+--------------+--------
      12 | internal   |       10 | f       | f            |             0 |         0 |         2246 |
      13 | c          |       10 | f       | f            |             0 |         0 |         2247 |
      14 | sql        |       10 | f       | t            |             0 |         0 |         2248 |
   14177 | plpgsql    |       10 | t       | t            |         14174 |     14175 |        14176 |
 1824389 | plpython3u |       10 | t       | f            |       1824386 |   1824387 |      1824388 |
(5 rows)

argosrm=# \c template1
psql (12.4 (Ubuntu 12.4-1), server 12.5)
You are now connected to database "template1" as user "pgsql".
template1=# select * from pg_language ;
  oid  | lanname  | lanowner | lanispl | lanpltrusted | lanplcallfoid | laninline | lanvalidator | lanacl
-------+----------+----------+---------+--------------+---------------+-----------+--------------+--------
    12 | internal |       10 | f       | f            |             0 |         0 |         2246 |
    13 | c        |       10 | f       | f            |             0 |         0 |         2247 |
    14 | sql      |       10 | f       | t            |             0 |         0 |         2248 |
 14177 | plpgsql  |       10 | t       | t            |         14174 |     14175 |        14176 |
(4 rows)

template1=# drop language plpython2u;
ERROR:  language "plpython2u" does not exist
template1=# drop language plpython;
ERROR:  language "plpython" does not exist
template1=# drop language plpythonu;
ERROR:  language "plpythonu" does not exist

template1=# \c postgres
psql (12.4 (Ubuntu 12.4-1), server 12.5)
You are now connected to database "postgres" as user "pgsql".
postgres=# drop language plpythonu;
ERROR:  language "plpythonu" does not exist
postgres=# drop language plpython;
ERROR:  language "plpython" does not exist
postgres=# drop language plpython2u;
ERROR:  language "plpython2u" does not exist


postgres=# \c argosrm
psql (12.4 (Ubuntu 12.4-1), server 12.5)
You are now connected to database "argosrm" as user "pgsql".
argosrm=# drop language plpython2u;
ERROR:  language "plpython2u" does not exist
argosrm=# drop language plpython;
ERROR:  language "plpython" does not exist
argosrm=# drop language plpythonu;
ERROR:  language "plpythonu" does not exist
argosrm=#


argosrm=# select * from pg_pltemplate    ;
  tmplname  | tmpltrusted | tmpldbacreate |      tmplhandler       |        tmplinline        |    tmplvalidator    |    tmpllibrary    | tmplacl
------------+-------------+---------------+------------------------+--------------------------+---------------------+-------------------+---------
 plpgsql    | t           | t             | plpgsql_call_handler   | plpgsql_inline_handler   | plpgsql_validator   | $libdir/plpgsql   |
 pltcl      | t           | t             | pltcl_call_handler     |                          |                     | $libdir/pltcl     |
 pltclu     | f           | f             | pltclu_call_handler    |                          |                     | $libdir/pltcl     |
 plperl     | t           | t             | plperl_call_handler    | plperl_inline_handler    | plperl_validator    | $libdir/plperl    |
 plperlu    | f           | f             | plperlu_call_handler   | plperlu_inline_handler   | plperlu_validator   | $libdir/plperl    |
 plpython3u | f           | f             | plpython3_call_handler | plpython3_inline_handler | plpython3_validator | $libdir/plpython3 |
(6 rows)

argosrm=# \c postgres
psql (12.4 (Ubuntu 12.4-1), server 12.5)
You are now connected to database "postgres" as user "pgsql".
postgres=# select * from pg_pltemplate    ;
  tmplname  | tmpltrusted | tmpldbacreate |      tmplhandler       |        tmplinline        |    tmplvalidator    |    tmpllibrary    | tmplacl
------------+-------------+---------------+------------------------+--------------------------+---------------------+-------------------+---------
 plpgsql    | t           | t             | plpgsql_call_handler   | plpgsql_inline_handler   | plpgsql_validator   | $libdir/plpgsql   |
 pltcl      | t           | t             | pltcl_call_handler     |                          |                     | $libdir/pltcl     |
 pltclu     | f           | f             | pltclu_call_handler    |                          |                     | $libdir/pltcl     |
 plperl     | t           | t             | plperl_call_handler    | plperl_inline_handler    | plperl_validator    | $libdir/plperl    |
 plperlu    | f           | f             | plperlu_call_handler   | plperlu_inline_handler   | plperlu_validator   | $libdir/plperl    |
 plpython3u | f           | f             | plpython3_call_handler | plpython3_inline_handler | plpython3_validator | $libdir/plpython3 |
(6 rows)

postgres=# \c template1
psql (12.4 (Ubuntu 12.4-1), server 12.5)
You are now connected to database "template1" as user "pgsql".
template1=# select * from pg_pltemplate    ;
  tmplname  | tmpltrusted | tmpldbacreate |      tmplhandler       |        tmplinline        |    tmplvalidator    |    tmpllibrary    | tmplacl
------------+-------------+---------------+------------------------+--------------------------+---------------------+-------------------+---------
 plpgsql    | t           | t             | plpgsql_call_handler   | plpgsql_inline_handler   | plpgsql_validator   | $libdir/plpgsql   |
 pltcl      | t           | t             | pltcl_call_handler     |                          |                     | $libdir/pltcl     |
 pltclu     | f           | f             | pltclu_call_handler    |                          |                     | $libdir/pltcl     |
 plperl     | t           | t             | plperl_call_handler    | plperl_inline_handler    | plperl_validator    | $libdir/plperl    |
 plperlu    | f           | f             | plperlu_call_handler   | plperlu_inline_handler   | plperlu_validator   | $libdir/plperl    |
 plpython3u | f           | f             | plpython3_call_handler | plpython3_inline_handler | plpython3_validator | $libdir/plpython3 |
(6 rows)

template1=#






Od: "Magnus Hagander" <magnus@hagander.net>
Do: "Marcin Giedz" <marcin.giedz@arise.pl>
DW: "Adrian Klaver" <adrian.klaver@aklaver.com>, "Tom Lane" <tgl@sss.pgh.pa.us>, "Devrim Gündüz" <devrim@gunduz.org>, "pgsql-general" <pgsql-general@lists.postgresql.org>
Wysłane: środa, 18 listopad 2020 10:36:10
Temat: Re: pg_upgrade from 12 to 13 failes with plpython2

On Wed, Nov 18, 2020 at 8:11 AM Marcin Giedz <marcin.giedz@arise.pl> wrote:
>
> but my question still remains the same - what causes pg_upgrade failure - are functions the reason? what I did was to delete these 2 rows from pg_pltemplate as I thought this may help:
>
> postgres=# delete from pg_pltemplate where tmplname = 'plpython2u';
> DELETE 1
> postgres=# delete from pg_pltemplate where tmplname = 'plpythonu';
> DELETE 1
>
>
> but pg_upgrade still complains about plpython2:
>
> cat loadable_libraries.txt
> could not load library "$libdir/plpython2": ERROR:  could not access file "$libdir/plpython2": No such file or directory
> In database: alaxx
> In database: template1


It's not the template that's interesting, it's the language itself you
need to drop. Log into each database and try to do that, and you will
get something like this if you still have functions using it:
postgres=# DROP LANGUAGE plpython2u;
ERROR:  cannot drop language plpython2u because other objects depend on it
DETAIL:  function testfunc() depends on language plpython2u
HINT:  Use DROP ... CASCADE to drop the dependent objects too.

If you have no functions using it, it will just go away, and once you
have dropped it in both databases you should be good to go.

And of course, if there are functions depending on it, you should
rebuild those on plpython3u before you drop plpython2u (or drop the
functions if they're not in use).

--
 Magnus Hagander
 Me: https://www.hagander.net/
 Work: https://www.redpill-linpro.com/

--
Pozdrawiam
Marcin Giedz
Wiceprezes Zarządu

ARISE Sp. z o.o.
mob. +48 606 673 977
mail: marcin.giedz@arise.pl

Al. Solidarności 117
00-140 Warszawa
tel. +48 (22) 440 56 20
fax  +48 (22) 440 56 22
http://www.arise.pl

Grupa ARISE: ARISE Sp. z o.o. (podmiot dominujący), ARISE Services S.A., Al. Solidarności 117, 00-140 Warszawa, Sąd Rejonowy dla m. st. Warszawy w Warszawie XII Wydział Gospodarczy Krajowego Rejestru Sądowego, KRS 0000316860, REGON 141595449, NIP 5272590610, kapitał zakładowy wpłacony 250,000.00 zł.

Re: pg_upgrade from 12 to 13 failes with plpython2

From
Laurenz Albe
Date:
On Wed, 2020-11-18 at 11:05 +0100, Marcin Giedz wrote:
> right, I had one function relaying on plpython2u so I changed it... but the again pg_upgrade claims error with
python:
> 
> cat loadable_libraries.txt 
> could not load library "$libdir/plpython2": ERROR:  could not access file "$libdir/plpython2": No such file or
directory
> In database: argosrm
> In database: template1

The problematic function is perhaps in another database.
Look everywhere.

Yours,
Laurenz Albe
-- 
Cybertec | https://www.cybertec-postgresql.com




Re: pg_upgrade from 12 to 13 failes with plpython2

From
Marcin Giedz
Date:
Hi, not the case I believe :

postgres=# \l
                              List of databases
   Name    | Owner | Encoding |   Collate   |    Ctype    | Access privileges
-----------+-------+----------+-------------+-------------+-------------------
 argosrm   | pgsql | UTF8     | pl_PL.UTF-8 | pl_PL.UTF-8 |
 postgres  | pgsql | UTF8     | pl_PL.UTF-8 | pl_PL.UTF-8 |
 template0 | pgsql | UTF8     | pl_PL.UTF-8 | pl_PL.UTF-8 | =c/pgsql         +
           |       |          |             |             | pgsql=CTc/pgsql
 template1 | pgsql | UTF8     | pl_PL.UTF-8 | pl_PL.UTF-8 | =c/pgsql         +
           |       |          |             |             | pgsql=CTc/pgsql
(4 rows)

all DBs checked and no plpython(2u) is found except for plpython3u

...

cat loadable_libraries.txt
could not load library "$libdir/plpython2": ERROR:  could not access file "$libdir/plpython2": No such file or directory
In database: argosrm
In database: template1

Thx
M.



Od: "Laurenz Albe" <laurenz.albe@cybertec.at>
Do: "Marcin Giedz" <marcin.giedz@arise.pl>, "Magnus Hagander" <magnus@hagander.net>
DW: "Adrian Klaver" <adrian.klaver@aklaver.com>, "Tom Lane" <tgl@sss.pgh.pa.us>, "Devrim Gündüz" <devrim@gunduz.org>, "pgsql-general" <pgsql-general@lists.postgresql.org>
Wysłane: środa, 18 listopad 2020 12:58:45
Temat: Re: pg_upgrade from 12 to 13 failes with plpython2

On Wed, 2020-11-18 at 11:05 +0100, Marcin Giedz wrote:
> right, I had one function relaying on plpython2u so I changed it... but the again pg_upgrade claims error with python:
>
> cat loadable_libraries.txt
> could not load library "$libdir/plpython2": ERROR:  could not access file "$libdir/plpython2": No such file or directory
> In database: argosrm
> In database: template1

The problematic function is perhaps in another database.
Look everywhere.

Yours,
Laurenz Albe
--
Cybertec | https://www.cybertec-postgresql.com


Re: pg_upgrade from 12 to 13 failes with plpython2

From
Tom Lane
Date:
Marcin Giedz <marcin.giedz@arise.pl> writes:
> all DBs checked and no plpython(2u) is found except for plpython3u 

I think you also need to make sure you've dropped the plpythonu
and plpython2u extensions in every database.

            regards, tom lane



Re: pg_upgrade from 12 to 13 failes with plpython2

From
Marcin Giedz
Date:
so look at this:


marcin@carbon2:~$ psql -U pgsql -h 192.168.89.64 postgres
psql (12.4 (Ubuntu 12.4-1), server 12.5)
Type "help" for help.

postgres=# drop extension plpython;
ERROR:  extension "plpython" does not exist
postgres=# drop extension plpythonu;
ERROR:  extension "plpythonu" does not exist
postgres=# drop extension plpython2u;
ERROR:  extension "plpython2u" does not exist
postgres=# \q
marcin@carbon2:~$ psql -U pgsql -h 192.168.89.64 template1
psql (12.4 (Ubuntu 12.4-1), server 12.5)
Type "help" for help.

template1=# drop extension plpython2u;
ERROR:  extension "plpython2u" does not exist
template1=# drop extension plpythonu;
ERROR:  extension "plpythonu" does not exist
template1=# drop extension plpython;
ERROR:  extension "plpython" does not exist
template1=# \q
marcin@carbon2:~$ psql -U pgsql -h 192.168.89.64 argosrm
psql (12.4 (Ubuntu 12.4-1), server 12.5)
Type "help" for help.

argosrm=# drop extension plpython;
ERROR:  extension "plpython" does not exist
argosrm=# drop extension plpythonu;
ERROR:  extension "plpythonu" does not exist
argosrm=# drop extension plpython2u;
ERROR:  extension "plpython2u" does not exist



Od: "Tom Lane" <tgl@sss.pgh.pa.us>
Do: "Marcin Giedz" <marcin.giedz@arise.pl>
DW: "Laurenz Albe" <laurenz.albe@cybertec.at>, "Magnus Hagander" <magnus@hagander.net>, "Adrian Klaver" <adrian.klaver@aklaver.com>, "Devrim Gündüz" <devrim@gunduz.org>, "pgsql-general" <pgsql-general@lists.postgresql.org>
Wysłane: środa, 18 listopad 2020 19:08:25
Temat: Re: pg_upgrade from 12 to 13 failes with plpython2

Marcin Giedz <marcin.giedz@arise.pl> writes:
> all DBs checked and no plpython(2u) is found except for plpython3u

I think you also need to make sure you've dropped the plpythonu
and plpython2u extensions in every database.

                        regards, tom lane


Re: pg_upgrade from 12 to 13 failes with plpython2

From
Tom Lane
Date:
[ please don't top-post, it makes conversations unreadable ]

Marcin Giedz <marcin.giedz@arise.pl> writes:
> so look at this: 
> postgres=# drop extension plpython; 
> ERROR: extension "plpython" does not exist 
> postgres=# drop extension plpythonu; 
> ERROR: extension "plpythonu" does not exist 
> postgres=# drop extension plpython2u; 
> ERROR: extension "plpython2u" does not exist 

Well, the pg_upgrade failure clearly shows that you've got some
functions referencing plpython2.  Maybe they're "loose" instead
of being bound into an extension --- that's quite possible if
this database has been brought forward from some pre-9.1 state.
Try looking in each database with

select * from pg_proc where probin like '%python2%';

            regards, tom lane



Re: pg_upgrade from 12 to 13 failes with plpython2

From
Marcin Giedz
Date:

[ please don't top-post, it makes conversations unreadable ]

Marcin Giedz <marcin.giedz@arise.pl> writes:
> so look at this:
> postgres=# drop extension plpython;
> ERROR: extension "plpython" does not exist
> postgres=# drop extension plpythonu;
> ERROR: extension "plpythonu" does not exist
> postgres=# drop extension plpython2u;
> ERROR: extension "plpython2u" does not exist

Well, the pg_upgrade failure clearly shows that you've got some
functions referencing plpython2.  Maybe they're "loose" instead
of being bound into an extension --- that's quite possible if
this database has been brought forward from some pre-9.1 state.
Try looking in each database with

select * from pg_proc where probin like '%python2%';

                        regards, tom lane

--------------------------------------------------------------------------------------------------------------------------------

sorry for top-posting not a day-to-day habits in our env ;)

anyway got this from your query:


  oid  |         proname         | pronamespace | proowner | prolang | procost | prorows | provariadic | prosupport | prokind | prosecdef | proleakproof | proisstrict | proretset | provolatile | proparallel | pronargs | pronargdefaults | prorettype | proargtypes | proallargtypes | proargmodes | proargnames | proargdefaults | protrftypes |         prosrc          |      probin       | proconfig | proacl
-------+-------------------------+--------------+----------+---------+---------+---------+-------------+------------+---------+-----------+--------------+-------------+-----------+-------------+-------------+----------+-----------------+------------+-------------+----------------+-------------+-------------+----------------+-------------+-------------------------+-------------------+-----------+--------
 16402 | plpython_call_handler   |           11 |       10 |      13 |       1 |       0 |           0 | -          | f       | f         | f            | f           | f         | v           | u           |        0 |               0 |       2280 |             |                |             |             |                |             | plpython_call_handler   | $libdir/plpython2 |           |
 16403 | plpython_inline_handler |           11 |       10 |      13 |       1 |       0 |           0 | -          | f       | f         | f            | t           | f         | v           | u           |        1 |               0 |       2278 | 2281        |                |             |             |                |             | plpython_inline_handler | $libdir/plpython2 |           |
 16404 | plpython_validator      |           11 |       10 |      13 |       1 |       0 |           0 | -          | f       | f         | f            | t           | f         | v           | u           |        1 |               0 |       2278 | 26          |                |             |             |                |             | plpython_validator      | $libdir/plpython2 |           |
(3 rows)

(END)

sounds nothing for me I'm afraid but I hope gives a hint for you ;)


Many thx
M.



Re: pg_upgrade from 12 to 13 failes with plpython2

From
Tom Lane
Date:
Marcin Giedz <marcin.giedz@arise.pl> writes:
> anyway got this from your query:

> 16402 | plpython_call_handler | 11 | 10 | 13 | 1 | 0 | 0 | - | f | f | f | f | f | v | u | 0 | 0 | 2280 | | | | | | |
plpython_call_handler| $libdir/plpython2 | |  
> 16403 | plpython_inline_handler | 11 | 10 | 13 | 1 | 0 | 0 | - | f | f | f | t | f | v | u | 1 | 0 | 2278 | 2281 | |
|| | | plpython_inline_handler | $libdir/plpython2 | |  
> 16404 | plpython_validator | 11 | 10 | 13 | 1 | 0 | 0 | - | f | f | f | t | f | v | u | 1 | 0 | 2278 | 26 | | | | | |
plpython_validator| $libdir/plpython2 | |  

Uh-huh, so there you have it.  These must be leftovers from some
pre-extension incarnation of plpython that was never cleaned up
properly.  Try

DROP FUNCTION pg_catalog.plpython_call_handler();
DROP FUNCTION pg_catalog.plpython_inline_handler(internal);
DROP FUNCTION pg_catalog.plpython_validator(oid);

It'll be interesting to see if there are any dependencies.

            regards, tom lane



Re: pg_upgrade from 12 to 13 failes with plpython2

From
Marcin Giedz
Date:

> anyway got this from your query:

> 16402 | plpython_call_handler | 11 | 10 | 13 | 1 | 0 | 0 | - | f | f | f | f | f | v | u | 0 | 0 | 2280 | | | | | | | plpython_call_handler | $libdir/plpython2 | |
> 16403 | plpython_inline_handler | 11 | 10 | 13 | 1 | 0 | 0 | - | f | f | f | t | f | v | u | 1 | 0 | 2278 | 2281 | | | | | | plpython_inline_handler | $libdir/plpython2 | |
> 16404 | plpython_validator | 11 | 10 | 13 | 1 | 0 | 0 | - | f | f | f | t | f | v | u | 1 | 0 | 2278 | 26 | | | | | | plpython_validator | $libdir/plpython2 | |

Uh-huh, so there you have it.  These must be leftovers from some
pre-extension incarnation of plpython that was never cleaned up
properly.  Try

DROP FUNCTION pg_catalog.plpython_call_handler();
DROP FUNCTION pg_catalog.plpython_inline_handler(internal);
DROP FUNCTION pg_catalog.plpython_validator(oid);

It'll be interesting to see if there are any dependencies.

                        regards, tom lane

-------------------------------------

BINGO! after drops all went smooth and easy


Many thx Tom!

M.

Re: pg_upgrade from 12 to 13 failes with plpython2

From
Bruce Momjian
Date:
On Wed, Nov 18, 2020 at 08:59:58PM +0100, Marcin Giedz wrote:
> 
> > anyway got this from your query:
> 
> > 16402 | plpython_call_handler | 11 | 10 | 13 | 1 | 0 | 0 | - | f | f | f | f
> | f | v | u | 0 | 0 | 2280 | | | | | | | plpython_call_handler | $libdir/
> plpython2 | |
> > 16403 | plpython_inline_handler | 11 | 10 | 13 | 1 | 0 | 0 | - | f | f | f |
> t | f | v | u | 1 | 0 | 2278 | 2281 | | | | | | plpython_inline_handler |
> $libdir/plpython2 | |
> > 16404 | plpython_validator | 11 | 10 | 13 | 1 | 0 | 0 | - | f | f | f | t | f
> | v | u | 1 | 0 | 2278 | 26 | | | | | | plpython_validator | $libdir/plpython2
> | |
> 
> Uh-huh, so there you have it.  These must be leftovers from some
> pre-extension incarnation of plpython that was never cleaned up
> properly.  Try
> 
> DROP FUNCTION pg_catalog.plpython_call_handler();
> DROP FUNCTION pg_catalog.plpython_inline_handler(internal);
> DROP FUNCTION pg_catalog.plpython_validator(oid);
> 
> It'll be interesting to see if there are any dependencies.
> 
>                         regards, tom lane
> 
> -------------------------------------
> 
> BINGO! after drops all went smooth and easy

I think one big problem is that when pg_upgrade fails in this way, users
are required to do some complex system catalog queries to diagnose the
cause.  Is there a way to simplify this for them?

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee




Re: pg_upgrade from 12 to 13 failes with plpython2

From
Tom Lane
Date:
Bruce Momjian <bruce@momjian.us> writes:
> I think one big problem is that when pg_upgrade fails in this way, users
> are required to do some complex system catalog queries to diagnose the
> cause.  Is there a way to simplify this for them?

Maybe pg_upgrade should print the actual function names, not just the
probin values.

            regards, tom lane



Re: pg_upgrade from 12 to 13 failes with plpython2

From
Bruce Momjian
Date:
On Wed, Nov 18, 2020 at 03:25:56PM -0500, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > I think one big problem is that when pg_upgrade fails in this way, users
> > are required to do some complex system catalog queries to diagnose the
> > cause.  Is there a way to simplify this for them?
> 
> Maybe pg_upgrade should print the actual function names, not just the
> probin values.

It is done that way so we don't overwhelm them with lots of function
names, and they can focus on the library.  I was thinking of showing
them a query that would allow them to investigate.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee




Re: pg_upgrade from 12 to 13 failes with plpython2

From
Tom Lane
Date:
Bruce Momjian <bruce@momjian.us> writes:
> On Wed, Nov 18, 2020 at 03:25:56PM -0500, Tom Lane wrote:
>> Maybe pg_upgrade should print the actual function names, not just the
>> probin values.

> It is done that way so we don't overwhelm them with lots of function
> names, and they can focus on the library.  I was thinking of showing
> them a query that would allow them to investigate.

These days "focus on the extension" would be more helpful.  Maybe
we could check to see if all the functions referencing a given .so
belong to the same extension, and if so complain about that extension?

I think there's a reasonable argument that functions that don't
belong to any known extension should be printed individually,
because more than likely the user's gonna have to clean them up
manually, as we saw here.

            regards, tom lane



Re: pg_upgrade from 12 to 13 failes with plpython2

From
Devrim Gündüz
Date:
Hi,

On Wed, 2020-11-18 at 14:54 -0500, Tom Lane wrote:
> Uh-huh, so there you have it.  These must be leftovers from some
> pre-extension incarnation of plpython that was never cleaned up
> properly. 

I think this was broken when Marcin first dropped the language. We
should just have dropped the extension, I guess.

Regards,

--
Devrim Gündüz
Open Source Solution Architect, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

Attachment

Re: pg_upgrade from 12 to 13 failes with plpython2

From
Bruce Momjian
Date:
On Wed, Nov 18, 2020 at 10:06:17PM +0000, Devrim Gunduz wrote:
> Hi,
> 
> On Wed, 2020-11-18 at 14:54 -0500, Tom Lane wrote:
> > Uh-huh, so there you have it.  These must be leftovers from some
> > pre-extension incarnation of plpython that was never cleaned up
> > properly. 
> 
> I think this was broken when Marcin first dropped the language. We
> should just have dropped the extension, I guess.

pg_upgrade does have some code to handle plpython call handlers in
function.c:get_loadable_libraries();

         * Systems that install plpython before 8.1 have
         * plpython_call_handler() defined in the "public" schema, causing
         * pg_dump to dump it.  However that function still references
         * "plpython" (no "2"), so it throws an error on restore.  This code
         * checks for the problem function, reports affected databases to the
         * user and explains how to remove them. 8.1 git commit:
         * e0dedd0559f005d60c69c9772163e69c204bac69
         * http://archives.postgresql.org/pgsql-hackers/2012-03/msg01101.php
         * http://archives.postgresql.org/pgsql-bugs/2012-05/msg00206.php

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee




Re: pg_upgrade from 12 to 13 failes with plpython2

From
Rob Sargent
Date:

> On Nov 18, 2020, at 9:29 PM, Bruce Momjian <bruce@momjian.us> wrote:
>
> On Wed, Nov 18, 2020 at 10:06:17PM +0000, Devrim Gunduz wrote:
>> Hi,
>>
>> On Wed, 2020-11-18 at 14:54 -0500, Tom Lane wrote:
>>> Uh-huh, so there you have it.  These must be leftovers from some
>>> pre-extension incarnation of plpython that was never cleaned up
>>> properly.
>>
>> I think this was broken when Marcin first dropped the language. We
>> should just have dropped the extension, I guess.
>
> pg_upgrade does have some code to handle plpython call handlers in
> function.c:get_loadable_libraries();
>
>         * Systems that install plpython before 8.1 have
>         * plpython_call_handler() defined in the "public" schema, causing
>         * pg_dump to dump it.  However that function still references
>         * "plpython" (no "2"), so it throws an error on restore.  This code
>         * checks for the problem function, reports affected databases to the
>         * user and explains how to remove them. 8.1 git commit:
>         * e0dedd0559f005d60c69c9772163e69c204bac69
>         * http://archives.postgresql.org/pgsql-hackers/2012-03/msg01101.php
>         * http://archives.postgresql.org/pgsql-bugs/2012-05/msg00206.php
>
> --
>  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
>  EnterpriseDB                             https://enterprisedb.com
>
>  The usefulness of a cup is in its emptiness, Bruce Lee
>
>
Unless it stop at prompts for “Yes, ok, I understand.” this could easily fly out the window.

>




Re: pg_upgrade from 12 to 13 failes with plpython2

From
Bruce Momjian
Date:
On Wed, Nov 18, 2020 at 09:35:20PM -0700, Rob Sargent wrote:
> > pg_upgrade does have some code to handle plpython call handlers in
> > function.c:get_loadable_libraries();
> > 
> >         * Systems that install plpython before 8.1 have
> >         * plpython_call_handler() defined in the "public" schema, causing
> >         * pg_dump to dump it.  However that function still references
> >         * "plpython" (no "2"), so it throws an error on restore.  This code
> >         * checks for the problem function, reports affected databases to the
> >         * user and explains how to remove them. 8.1 git commit:
> >         * e0dedd0559f005d60c69c9772163e69c204bac69
> >         * http://archives.postgresql.org/pgsql-hackers/2012-03/msg01101.php
> >         * http://archives.postgresql.org/pgsql-bugs/2012-05/msg00206.php
> > 
> > -- 
> >  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
> >  EnterpriseDB                             https://enterprisedb.com
> > 
> >  The usefulness of a cup is in its emptiness, Bruce Lee
> > 
> > 
> Unless it stop at prompts for “Yes, ok, I understand.” this could easily fly out the window.

It issues this message and fails:

            if (PQntuples(res) > 0)
            {
                if (!found_public_plpython_handler)
                {
                    pg_log(PG_WARNING,
                           "\nThe old cluster has a \"plpython_call_handler\" function defined\n"
                           "in the \"public\" schema which is a duplicate of the one defined\n"
                           "in the \"pg_catalog\" schema.  You can confirm this by executing\n"
                           "in psql:\n"
                           "\n"
                           "    \\df *.plpython_call_handler\n"
                           "\n"
                           "The \"public\" schema version of this function was created by a\n"
                           "pre-8.1 install of plpython, and must be removed for pg_upgrade\n"
                           "to complete because it references a now-obsolete \"plpython\"\n"
                           "shared object file.  You can remove the \"public\" schema version\n"
                           "of this function by running the following command:\n"
                           "\n"
                           "    DROP FUNCTION public.plpython_call_handler()\n"
                           "\n"
                           "in each affected database:\n"
                           "\n");
                }
                pg_log(PG_WARNING, "    %s\n", active_db->db_name);
                found_public_plpython_handler = true;
            }
            PQclear(res);
        }

        PQfinish(conn);
    }

    if (found_public_plpython_handler)
        pg_fatal("Remove the problem functions from the old cluster to continue.\n");


-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee




Re: pg_upgrade from 12 to 13 failes with plpython2

From
Rob Sargent
Date:


> On Nov 18, 2020, at 9:39 PM, Bruce Momjian <bruce@momjian.us> wrote:
>
> On Wed, Nov 18, 2020 at 09:35:20PM -0700, Rob Sargent wrote:
>>> pg_upgrade does have some code to handle plpython call handlers in
>>> function.c:get_loadable_libraries();
>>>
>>>        * Systems that install plpython before 8.1 have
>>>        * plpython_call_handler() defined in the "public" schema, causing
>>>        * pg_dump to dump it.  However that function still references
>>>        * "plpython" (no "2"), so it throws an error on restore.  This code
>>>        * checks for the problem function, reports affected databases to the
>>>        * user and explains how to remove them. 8.1 git commit:
>>>        * e0dedd0559f005d60c69c9772163e69c204bac69
>>>        * http://archives.postgresql.org/pgsql-hackers/2012-03/msg01101.php
>>>        * http://archives.postgresql.org/pgsql-bugs/2012-05/msg00206.php
>>>
>>> --
>>> Bruce Momjian  <bruce@momjian.us>        https://momjian.us
>>> EnterpriseDB                             https://enterprisedb.com
>>>
>>> The usefulness of a cup is in its emptiness, Bruce Lee
>>>
>>>
>> Unless it stop at prompts for “Yes, ok, I understand.” this could easily fly out the window.
>
> It issues this message and fails:
>
>            if (PQntuples(res) > 0)
>            {
>                if (!found_public_plpython_handler)
>                {
>                    pg_log(PG_WARNING,
>                           "\nThe old cluster has a \"plpython_call_handler\" function defined\n"
>                           "in the \"public\" schema which is a duplicate of the one defined\n"
>                           "in the \"pg_catalog\" schema.  You can confirm this by executing\n"
>                           "in psql:\n"
>                           "\n"
>                           "    \\df *.plpython_call_handler\n"
>                           "\n"
>                           "The \"public\" schema version of this function was created by a\n"
>                           "pre-8.1 install of plpython, and must be removed for pg_upgrade\n"
>                           "to complete because it references a now-obsolete \"plpython\"\n"
>                           "shared object file.  You can remove the \"public\" schema version\n"
>                           "of this function by running the following command:\n"
>                           "\n"
>                           "    DROP FUNCTION public.plpython_call_handler()\n"
>                           "\n"
>                           "in each affected database:\n"
>                           "\n");
>                }
>                pg_log(PG_WARNING, "    %s\n", active_db->db_name);
>                found_public_plpython_handler = true;
>            }
>            PQclear(res);
>        }
>
>        PQfinish(conn);
>    }
>
>    if (found_public_plpython_handler)
>        pg_fatal("Remove the problem functions from the old cluster to continue.\n");
>
>
Does this jive with OP’s experience?  Or should it?


Re: pg_upgrade from 12 to 13 failes with plpython2

From
Bruce Momjian
Date:
On Wed, Nov 18, 2020 at 10:57:00PM -0700, Rob Sargent wrote:
> > It issues this message and fails:
> > 
> >            if (PQntuples(res) > 0)
> >            {
> >                if (!found_public_plpython_handler)
> >                {
> >                    pg_log(PG_WARNING,
> >                           "\nThe old cluster has a \"plpython_call_handler\" function defined\n"
> >                           "in the \"public\" schema which is a duplicate of the one defined\n"
> >                           "in the \"pg_catalog\" schema.  You can confirm this by executing\n"
> >                           "in psql:\n"
> >                           "\n"
> >                           "    \\df *.plpython_call_handler\n"
> >                           "\n"
> >                           "The \"public\" schema version of this function was created by a\n"
> >                           "pre-8.1 install of plpython, and must be removed for pg_upgrade\n"
> >                           "to complete because it references a now-obsolete \"plpython\"\n"
> >                           "shared object file.  You can remove the \"public\" schema version\n"
> >                           "of this function by running the following command:\n"
> >                           "\n"
> >                           "    DROP FUNCTION public.plpython_call_handler()\n"
> >                           "\n"
> >                           "in each affected database:\n"
> >                           "\n");
> >                }
> >                pg_log(PG_WARNING, "    %s\n", active_db->db_name);
> >                found_public_plpython_handler = true;
> >            }
> >            PQclear(res);
> >        }
> > 
> >        PQfinish(conn);
> >    }
> > 
> >    if (found_public_plpython_handler)
> >        pg_fatal("Remove the problem functions from the old cluster to continue.\n");
> > 
> > 
> Does this jive with OP’s experience?  Or should it?

It didn't trigger this message for him, and I am also wondering if it
should have.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee




Re: pg_upgrade from 12 to 13 failes with plpython2

From
Tom Lane
Date:
Bruce Momjian <bruce@momjian.us> writes:
> It didn't trigger this message for him, and I am also wondering if it
> should have.

The extra functions in this case were in pg_catalog, not public,
so there is exactly no part of that error message that is applicable.

In any case, that seems an overly specific solution.  The generic
problem is how to usefully identify some functions that have dangling
probin pointers.  I don't want a solution that only works for the
plpython functions.

            regards, tom lane



Re: pg_upgrade from 12 to 13 failes with plpython2

From
Devrim Gündüz
Date:
Hi,

On Tue, 2020-11-17 at 14:30 -0800, Adrian Klaver wrote:
> As a packager you are in charge of how the packaging is done. Still
> announcing a change that effectively nullifies the documentation
> would to me be something that should be announced somewhere else
> than a list  that I'm guessing 99% of the users don't read.

Published a blog post today:

https://people.planetpostgresql.org/devrim/index.php?/archives/106-What-is-new-in-PostgreSQL-13-RPMs.html

Same text will go to yum.postgresql.org tomorrow.

(Sorry for the delay)

Regards,
--
Devrim Gündüz
Open Source Solution Architect, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

Attachment