Re: Resolving the python 2 -> python 3 mess - Mailing list pgsql-hackers

I wrote:
> Here's an updated pair of patches that attempt to fix the MSVC
> scripts (pretty blindly) and provide a very simple regression test.

A little *too* blindly, evidently.  Try again ...

            regards, tom lane

diff --git a/doc/src/sgml/plpython.sgml b/doc/src/sgml/plpython.sgml
index 1921915..ac989a3 100644
--- a/doc/src/sgml/plpython.sgml
+++ b/doc/src/sgml/plpython.sgml
@@ -164,13 +164,6 @@
   </para>

   <para>
-   See also the
-   document <ulink url="https://docs.python.org/3/whatsnew/3.0.html">What's
-   New In Python 3.0</ulink> for more information about porting to
-   Python 3.
-  </para>
-
-  <para>
    It is not allowed to use PL/Python based on Python 2 and PL/Python
    based on Python 3 in the same session, because the symbols in the
    dynamic modules would clash, which could result in crashes of the
@@ -179,6 +172,90 @@
    a mismatch is detected.  It is possible, however, to use both
    PL/Python variants in the same database, from separate sessions.
   </para>
+
+  <sect2 id="plpython-python3-conversion">
+   <title>Converting from Python 2 to Python 3</title>
+
+   <para>
+    See the
+    document <ulink url="https://docs.python.org/3/whatsnew/3.0.html">What's
+    New In Python 3.0</ulink> for the Python community's information and
+    recommendations about porting to Python 3.
+   </para>
+
+   <para>
+    <productname>PostgreSQL</productname> provides some support for helping
+    you to convert existing Python 2 routines to Python 3.  In an
+    installation built with Python 3, there is an
+    extension <filename>convert_python3</filename> that changes functions
+    and procedures from the <literal>plpythonu</literal>
+    and <literal>plpython2u</literal> languages to
+    the <literal>plpython3u</literal> language.  While doing so, it applies
+    the <filename>2to3</filename> tool described in the above document to
+    the body of each such routine.
+   </para>
+
+   <para>
+    Using <filename>convert_python3</filename> can be as simple as:
+<programlisting>
+CREATE EXTENSION convert_python3;
+CALL convert_python3_all();
+</programlisting>
+    This must be done as database superuser.  If you wish, you can drop the
+    extension once you're done converting everything.
+   </para>
+
+   <para>
+    Since <filename>convert_python3</filename> is Python 3 code, be careful
+    not to install or run it in a session that has previously executed any
+    Python 2 code.  As explained above, that won't work.
+   </para>
+
+   <para>
+    <function>convert_python3_all</function> has two optional arguments: the
+    name of the conversion tool to use (by default <literal>2to3</literal>,
+    but you might for instance need to provide a full path name) and any
+    special command-line options to provide to it.  You might for example
+    want to adjust the set of <quote>fixer</quote> rules
+    that <literal>2to3</literal> applies:
+<programlisting>
+CALL convert_python3_all(options => '-f idioms -x apply');
+</programlisting>
+    See <literal>2to3</literal>'s
+    <ulink url="https://docs.python.org/3/library/2to3.html">documentation</ulink>
+    for more information.
+   </para>
+
+   <para>
+    The <filename>convert_python3</filename> extension also provides a
+    procedure that converts just one Python 2 function at a time:
+<programlisting>
+CALL convert_python3_one('myfunc(int)');
+</programlisting>
+    The argument is the target function's OID, which can be written as
+    a <type>regprocedure</type> constant (see
+    <xref linkend="datatype-oid"/>).  The main reason to use this would be
+    if you need to use different options for different functions.  It has
+    the same optional arguments as <function>convert_python3_all</function>:
+<programlisting>
+CALL convert_python3_one('otherfunc(text)', tool => '/usr/bin/2to3',
+                         options => '-f idioms');
+</programlisting>
+   </para>
+
+   <para>
+    If you have needs that go beyond this, consult the source code for
+    the <filename>convert_python3</filename> extension (it's just a
+    couple of <literal>plpython3u</literal> procedures) and adapt those
+    procedures as necessary.
+   </para>
+
+   <para>
+    Keep in mind that if you've constructed any <command>DO</command> blocks
+    that use Python 2 code, those will have to be fixed up manually,
+    wherever the source code for them exists.
+   </para>
+  </sect2>
  </sect1>

  <sect1 id="plpython-funcs">
diff --git a/src/pl/plpython/Makefile b/src/pl/plpython/Makefile
index 9e95285..03f858b 100644
--- a/src/pl/plpython/Makefile
+++ b/src/pl/plpython/Makefile
@@ -38,6 +38,9 @@ DATA = $(NAME)u.control $(NAME)u--1.0.sql
 ifeq ($(python_majorversion),2)
 DATA += plpythonu.control plpythonu--1.0.sql
 endif
+ifeq ($(python_majorversion),3)
+DATA += convert_python3.control convert_python3--1.0.sql
+endif

 # header files to install - it's not clear which of these might be needed
 # so install them all.
diff --git a/src/pl/plpython/convert_python3--1.0.sql b/src/pl/plpython/convert_python3--1.0.sql
new file mode 100644
index 0000000..3444ac0
--- /dev/null
+++ b/src/pl/plpython/convert_python3--1.0.sql
@@ -0,0 +1,149 @@
+/* convert_python3--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION convert_python3" to load this file. \quit
+
+-- This module provides two procedures, one to convert all python2
+-- functions and one to do just one.  They're nearly identical, and
+-- in principle convert_python3_all() could be written as a loop
+-- around convert_python3_one().  It's not done that way since
+-- creating a temp directory for each function in a bulk conversion
+-- could get expensive.
+
+-- For some benighted reason, lib2to3 has exactly no documented API,
+-- so we must use the command-line API "2to3" instead.  User may pass
+-- in the name of that program (in case it's not in the server's PATH)
+-- as well as any desired options for it (perhaps some -f switches).
+
+create procedure convert_python3_all(tool text default '2to3',
+                                     options text default '')
+language plpython3u as $$
+import re, subprocess, tempfile
+
+# pattern to extract just the function header from pg_get_functiondef result
+aspat = re.compile("^(.*?\nAS )", re.DOTALL)
+# pattern for replacing LANGUAGE portion
+langpat = re.compile("\n LANGUAGE plpython2?u\n")
+
+# collect info about functions to update
+rv = plpy.execute("""
+select p.oid::pg_catalog.regprocedure as funcid,
+       pg_catalog.pg_get_functiondef(p.oid) as fd,
+       prosrc as body
+from pg_catalog.pg_proc p join pg_catalog.pg_language l on p.prolang = l.oid
+where lanname in ('plpythonu', 'plpython2u')
+order by proname
+""")
+
+# Make a temp directory to hold the file for 2to3 to work on.
+with tempfile.TemporaryDirectory() as tmpdirname:
+
+    # process each function
+    for r in rv:
+        # emit notices so user can tell which function failed, if one does
+        plpy.notice("converting function " + r["funcid"])
+
+        # extract everything but the body from pg_get_functiondef result
+        m = aspat.match(r["fd"])
+        if not m:
+            raise ValueError('unexpected match failure')
+        fheader = m.group(1)
+
+        # replace the language clause
+        fheader = langpat.sub("\n LANGUAGE plpython3u\n", fheader, 1)
+
+        # put body in a temp file so we can apply 2to3
+        f = open(tmpdirname + "/temp.py", mode = 'w')
+        f.write(r["body"])
+        f.close()
+
+        # apply 2to3 to body
+        subprocess.check_call(tool + " " + options + " --no-diffs -w " + tmpdirname + "/temp.py", shell=True)
+        f = open(tmpdirname + "/temp.py", mode = 'r')
+        fbody = f.read()
+        f.close()
+
+        # ensure check_function_bodies is enabled
+        plpy.execute("set local check_function_bodies = true")
+
+        # construct and execute SQL command to replace the function
+        newstmt = fheader + plpy.quote_literal(fbody)
+        # uncomment this for debugging purposes:
+        # plpy.info(newstmt)
+        plpy.execute(newstmt)
+
+        # commit after each successful replacement, in case a later one fails
+        plpy.commit()
+$$;
+
+-- The above procedure has to be superuser-only since it trivially allows
+-- executing random programs.  But you'd have to be superuser anyway
+-- to replace the definitions of plpython functions.
+
+revoke all on procedure convert_python3_all(text, text) from public;
+
+
+-- Here's the one-function version.
+
+create procedure convert_python3_one(funcid regprocedure,
+                                     tool text default '2to3',
+                                     options text default '')
+language plpython3u as $$
+import re, subprocess, tempfile
+
+# pattern to extract just the function header from pg_get_functiondef result
+aspat = re.compile("^(.*?\nAS )", re.DOTALL)
+# pattern for replacing LANGUAGE portion
+langpat = re.compile("\n LANGUAGE plpython2?u\n")
+
+# collect info about function to update, making sure it's the right language
+plan = plpy.prepare("""
+select p.oid::pg_catalog.regprocedure as funcid,
+       pg_catalog.pg_get_functiondef(p.oid) as fd,
+       prosrc as body
+from pg_catalog.pg_proc p join pg_catalog.pg_language l on p.prolang = l.oid
+where p.oid = $1 and lanname in ('plpythonu', 'plpython2u')
+""", ["pg_catalog.regprocedure"])
+
+rv = plpy.execute(plan, [funcid])
+
+# Make a temp directory to hold the file for 2to3 to work on.
+with tempfile.TemporaryDirectory() as tmpdirname:
+
+    # process each function (we only expect one, but it's easy to loop)
+    for r in rv:
+        # extract everything but the body from pg_get_functiondef result
+        m = aspat.match(r["fd"])
+        if not m:
+            raise ValueError('unexpected match failure')
+        fheader = m.group(1)
+
+        # replace the language clause
+        fheader = langpat.sub("\n LANGUAGE plpython3u\n", fheader, 1)
+
+        # put body in a temp file so we can apply 2to3
+        f = open(tmpdirname + "/temp.py", mode = 'w')
+        f.write(r["body"])
+        f.close()
+
+        # apply 2to3 to body
+        subprocess.check_call(tool + " " + options + " --no-diffs -w " + tmpdirname + "/temp.py", shell=True)
+        f = open(tmpdirname + "/temp.py", mode = 'r')
+        fbody = f.read()
+        f.close()
+
+        # ensure check_function_bodies is enabled
+        plpy.execute("set local check_function_bodies = true")
+
+        # construct and execute SQL command to replace the function
+        newstmt = fheader + plpy.quote_literal(fbody)
+        # uncomment this for debugging purposes:
+        # plpy.info(newstmt)
+        plpy.execute(newstmt)
+$$;
+
+-- The above procedure has to be superuser-only since it trivially allows
+-- executing random programs.  But you'd have to be superuser anyway
+-- to replace the definitions of plpython functions.
+
+revoke all on procedure convert_python3_one(regprocedure, text, text) from public;
diff --git a/src/pl/plpython/convert_python3.control b/src/pl/plpython/convert_python3.control
new file mode 100644
index 0000000..8debb3b
--- /dev/null
+++ b/src/pl/plpython/convert_python3.control
@@ -0,0 +1,5 @@
+# convert_python3 extension
+comment = 'convert plpython[2]u functions to plpython3u'
+default_version = '1.0'
+relocatable = true
+requires = 'plpython3u'
diff --git a/configure b/configure
index d2c74e5..3feebf6 100755
--- a/configure
+++ b/configure
@@ -714,6 +714,7 @@ with_ldap
 with_krb_srvnam
 krb_srvtab
 with_gssapi
+with_python2_stub
 with_python
 with_perl
 with_tcl
@@ -847,6 +848,7 @@ with_tcl
 with_tclconfig
 with_perl
 with_python
+with_python2_stub
 with_gssapi
 with_krb_srvnam
 with_pam
@@ -1546,6 +1548,7 @@ Optional Packages:
   --with-tclconfig=DIR    tclConfig.sh is in DIR
   --with-perl             build Perl modules (PL/Perl)
   --with-python           build Python modules (PL/Python)
+  --with-python2-stub     build Python 2 compatibility stub
   --with-gssapi           build with GSSAPI support
   --with-krb-srvnam=NAME  default service principal name in Kerberos (GSSAPI)
                           [postgres]
@@ -7644,6 +7647,32 @@ fi
 $as_echo "$with_python" >&6; }


+
+
+
+# Check whether --with-python2-stub was given.
+if test "${with_python2_stub+set}" = set; then :
+  withval=$with_python2_stub;
+  case $withval in
+    yes)
+      :
+      ;;
+    no)
+      :
+      ;;
+    *)
+      as_fn_error $? "no argument expected for --with-python2-stub option" "$LINENO" 5
+      ;;
+  esac
+
+else
+  with_python2_stub=no
+
+fi
+
+
+
+
 #
 # GSSAPI
 #
@@ -9745,6 +9774,12 @@ $as_echo "${python_libspec} ${python_additional_libs}" >&6; }



+  # Disable building Python 2 stub if primary version isn't Python 3
+  if test "$python_majorversion" -lt 3; then
+    with_python2_stub=no
+  fi
+else
+  with_python2_stub=no
 fi

 if test "$cross_compiling" = yes && test -z "$with_system_tzdata"; then
diff --git a/configure.in b/configure.in
index 0b0a871..ffa49c9 100644
--- a/configure.in
+++ b/configure.in
@@ -766,6 +766,9 @@ PGAC_ARG_BOOL(with, python, no, [build Python modules (PL/Python)])
 AC_MSG_RESULT([$with_python])
 AC_SUBST(with_python)

+PGAC_ARG_BOOL(with, python2-stub, no, [build Python 2 compatibility stub])
+AC_SUBST(with_python2_stub)
+
 #
 # GSSAPI
 #
@@ -1042,6 +1045,12 @@ fi
 if test "$with_python" = yes; then
   PGAC_PATH_PYTHON
   PGAC_CHECK_PYTHON_EMBED_SETUP
+  # Disable building Python 2 stub if primary version isn't Python 3
+  if test "$python_majorversion" -lt 3; then
+    with_python2_stub=no
+  fi
+else
+  with_python2_stub=no
 fi

 if test "$cross_compiling" = yes && test -z "$with_system_tzdata"; then
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index cc242dc..ff81ea8 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -877,6 +877,23 @@ build-postgresql:
       </varlistentry>

       <varlistentry>
+       <term><option>--with-python2-stub</option></term>
+       <listitem>
+        <para>
+         Build a stub version of the Python
+         2 <application>PL/Python</application> language, to aid in
+         transitioning old <application>PL/Python</application> code.
+         This option is recommended if you are
+         building <application>PL/Python</application> for Python 3
+         and do not intend to also build a version for Python 2.
+         This option is ignored if you do not also
+         specify <option>--with-python</option>, or if the selected Python
+         implementation is Python 2.
+        </para>
+       </listitem>
+      </varlistentry>
+
+      <varlistentry>
        <term><option>--with-tcl</option></term>
        <listitem>
         <para>
diff --git a/doc/src/sgml/plpython.sgml b/doc/src/sgml/plpython.sgml
index ac989a3..9014750 100644
--- a/doc/src/sgml/plpython.sgml
+++ b/doc/src/sgml/plpython.sgml
@@ -77,13 +77,7 @@
      <para>
       The language named <literal>plpythonu</literal> implements
       PL/Python based on the default Python language variant, which is
-      currently Python 2.  (This default is independent of what any
-      local Python installations might consider to be
-      their <quote>default</quote>, for example,
-      what <filename>/usr/bin/python</filename> might be.)  The
-      default will probably be changed to Python 3 in a distant future
-      release of PostgreSQL, depending on the progress of the
-      migration to Python 3 in the Python community.
+      determined by the person who builds and installs PostgreSQL.
      </para>
     </listitem>
    </itemizedlist>
@@ -103,9 +97,22 @@
     The built variant depends on which Python version was found during
     the installation or which version was explicitly set using
     the <envar>PYTHON</envar> environment variable;
-    see <xref linkend="install-procedure"/>.  To make both variants of
-    PL/Python available in one installation, the source tree has to be
-    configured and built twice.
+    see <xref linkend="install-procedure"/>.  To make working versions of
+    both variants of PL/Python available in one installation, the source
+    tree has to be configured and built twice.
+   </para>
+
+   <para>
+    However, if the builder specifies both <option>--with-python</option>
+    and <option>--with-python2-stub</option> while building with Python 3,
+    then in addition to the working Python-3-based PL/Python,
+    a <quote>stub</quote> version of <literal>plpython2u</literal> is
+    produced.  This stub is non-functional and will simply throw errors if
+    any <literal>plpython2u</literal> function is executed, but its
+    presence allows such functions to be loaded into the database in
+    preparation for conversion to <literal>plpython3u</literal>.  This
+    configuration also causes <literal>plpythonu</literal> to become
+    Python 3 rather than Python 2.
    </para>
   </tip>

@@ -115,49 +122,38 @@
    <itemizedlist>
     <listitem>
      <para>
-      Existing users and users who are currently not interested in
-      Python 3 use the language name <literal>plpythonu</literal> and
-      don't have to change anything for the foreseeable future.  It is
-      recommended to gradually <quote>future-proof</quote> the code
-      via migration to Python 2.6/2.7 to simplify the eventual
-      migration to Python 3.
+      Use the language name <literal>plpythonu</literal> if you have
+      simple Python code that works for either Python 2 or Python 3;
+      or if you are prepared to migrate your code whenever your packager
+      decides to change the default Python version.
      </para>

      <para>
       In practice, many PL/Python functions will migrate to Python 3
-      with few or no changes.
+      with few or no changes.  However, there are some cases that
+      will require more work.
      </para>
     </listitem>

     <listitem>
      <para>
-      Users who know that they have heavily Python 2 dependent code
-      and don't plan to ever change it can make use of
+      If you know that you have heavily Python 2 dependent code
+      and don't plan to ever change it, you can make use of
       the <literal>plpython2u</literal> language name.  This will
-      continue to work into the very distant future, until Python 2
-      support might be completely dropped by PostgreSQL.
-     </para>
-    </listitem>
-
-    <listitem>
-     <para>
-      Users who want to dive into Python 3 can use
-      the <literal>plpython3u</literal> language name, which will keep
-      working forever by today's standards.  In the distant future,
-      when Python 3 might become the default, they might like to
-      remove the <quote>3</quote> for aesthetic reasons.
+      continue to work for as long as your operating system platform
+      continues to provide Python 2.  PostgreSQL itself may eventually
+      drop support for this option, but that is unlikely to happen
+      as long as any platform support remains in the wild.
      </para>
     </listitem>

     <listitem>
      <para>
-      Daredevils, who want to build a Python-3-only operating system
-      environment, can change the contents of
-      <literal>plpythonu</literal>'s extension control and script files
-      to make <literal>plpythonu</literal> be equivalent
-      to <literal>plpython3u</literal>, keeping in mind that this
-      would make their installation incompatible with most of the rest
-      of the world.
+      If you know your code requires Python 3, use
+      the <literal>plpython3u</literal> language name.  You can also
+      use this language name to tag functions that have been successfully
+      converted from Python 2, so as to keep track of which functions have
+      been converted and which have not.
      </para>
     </listitem>
    </itemizedlist>
@@ -171,6 +167,8 @@
    Python major versions in a session, which will abort the session if
    a mismatch is detected.  It is possible, however, to use both
    PL/Python variants in the same database, from separate sessions.
+   (Also, the <quote>stub</quote> version of <literal>plpython2u</literal>
+   does not present any conflicts.)
   </para>

   <sect2 id="plpython-python3-conversion">
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index e4db3e8..1f880b6 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -180,6 +180,7 @@ bitcodedir = $(pkglibdir)/bitcode
 with_icu    = @with_icu@
 with_perl    = @with_perl@
 with_python    = @with_python@
+with_python2_stub = @with_python2_stub@
 with_tcl    = @with_tcl@
 with_openssl    = @with_openssl@
 with_readline    = @with_readline@
diff --git a/src/pl/Makefile b/src/pl/Makefile
index c4a0d1c..fe9cd0f 100644
--- a/src/pl/Makefile
+++ b/src/pl/Makefile
@@ -26,6 +26,12 @@ else
 ALWAYS_SUBDIRS += plpython
 endif

+ifeq ($(with_python2_stub), yes)
+SUBDIRS += stub_plpython2
+else
+ALWAYS_SUBDIRS += stub_plpython2
+endif
+
 ifeq ($(with_tcl), yes)
 SUBDIRS += tcl
 else
diff --git a/src/pl/plpython/Makefile b/src/pl/plpython/Makefile
index 03f858b..ac262fa 100644
--- a/src/pl/plpython/Makefile
+++ b/src/pl/plpython/Makefile
@@ -37,6 +37,9 @@ OBJS = \
 DATA = $(NAME)u.control $(NAME)u--1.0.sql
 ifeq ($(python_majorversion),2)
 DATA += plpythonu.control plpythonu--1.0.sql
+else ifeq ($(with_python2_stub), yes)
+# install extension files for the stub module (see ../stub_plpython2)
+DATA += plpythonu.control plpythonu--1.0.sql plpython2u.control plpython2u--1.0.sql
 endif
 ifeq ($(python_majorversion),3)
 DATA += convert_python3.control convert_python3--1.0.sql
@@ -109,6 +112,7 @@ REGRESS = \
     plpython_composite \
     plpython_subtransaction \
     plpython_transaction \
+    plpython_stub \
     plpython_drop

 REGRESS_PLPYTHON3_MANGLE := $(REGRESS)
diff --git a/src/pl/plpython/expected/plpython_stub.out b/src/pl/plpython/expected/plpython_stub.out
new file mode 100644
index 0000000..2a565b4
--- /dev/null
+++ b/src/pl/plpython/expected/plpython_stub.out
@@ -0,0 +1,71 @@
+--
+-- Test stub plpython2 and convert_python3 extension
+--
+-- skip test unless we have python3 installed and the others are available
+SELECT (SELECT count(*) FROM pg_extension WHERE extname = 'plpython3u') = 0 OR
+       (SELECT count(*) FROM pg_available_extensions WHERE name = 'plpython2u') = 0 OR
+       (SELECT count(*) FROM pg_available_extensions WHERE name = 'plpythonu') = 0 OR
+       (SELECT count(*) FROM pg_available_extensions WHERE name = 'convert_python3') = 0
+       AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+-- Funny formatting of some commands below is to prevent
+-- regress-python3-mangle.mk from being "helpful".
+-- install the additional extensions
+CREATE EXTENSION
+plpython2u;
+CREATE EXTENSION
+plpythonu;
+CREATE EXTENSION convert_python3;
+-- create some functions that need conversion
+create function convert1() returns int
+as 'return 123l'
+language
+plpython2u
+immutable;
+-- disable syntax checking since if plpythonu is python 3, this'll fail
+set check_function_bodies = false;
+create function convert2() returns int
+as 'return 123l'
+language
+plpythonu
+immutable;
+-- can't use \sf here because mangling would affect the LANGUAGE clauses
+select proname, lanname, prosrc
+from pg_proc p join pg_language l on prolang = l.oid
+where proname = 'convert1';
+ proname  |  lanname   |   prosrc
+----------+------------+-------------
+ convert1 | plpython2u | return 123l
+(1 row)
+
+select proname, lanname, prosrc
+from pg_proc p join pg_language l on prolang = l.oid
+where proname = 'convert2';
+ proname  |  lanname  |   prosrc
+----------+-----------+-------------
+ convert2 | plpythonu | return 123l
+(1 row)
+
+call convert_python3_all();
+NOTICE:  converting function convert1()
+NOTICE:  converting function convert2()
+\sf convert1()
+CREATE OR REPLACE FUNCTION public.convert1()
+ RETURNS integer
+ LANGUAGE plpython3u
+ IMMUTABLE
+AS $function$return 123$function$
+\sf convert2()
+CREATE OR REPLACE FUNCTION public.convert2()
+ RETURNS integer
+ LANGUAGE plpython3u
+ IMMUTABLE
+AS $function$return 123$function$
+-- clean up
+DROP EXTENSION
+plpython2u;
+DROP EXTENSION
+plpythonu;
+DROP EXTENSION convert_python3;
diff --git a/src/pl/plpython/expected/plpython_stub_1.out b/src/pl/plpython/expected/plpython_stub_1.out
new file mode 100644
index 0000000..bb625b9
--- /dev/null
+++ b/src/pl/plpython/expected/plpython_stub_1.out
@@ -0,0 +1,11 @@
+--
+-- Test stub plpython2 and convert_python3 extension
+--
+-- skip test unless we have python3 installed and the others are available
+SELECT (SELECT count(*) FROM pg_extension WHERE extname = 'plpython3u') = 0 OR
+       (SELECT count(*) FROM pg_available_extensions WHERE name = 'plpython2u') = 0 OR
+       (SELECT count(*) FROM pg_available_extensions WHERE name = 'plpythonu') = 0 OR
+       (SELECT count(*) FROM pg_available_extensions WHERE name = 'convert_python3') = 0
+       AS skip_test \gset
+\if :skip_test
+\quit
diff --git a/src/pl/plpython/sql/plpython_stub.sql b/src/pl/plpython/sql/plpython_stub.sql
new file mode 100644
index 0000000..8c587fd
--- /dev/null
+++ b/src/pl/plpython/sql/plpython_stub.sql
@@ -0,0 +1,60 @@
+--
+-- Test stub plpython2 and convert_python3 extension
+--
+
+-- skip test unless we have python3 installed and the others are available
+SELECT (SELECT count(*) FROM pg_extension WHERE extname = 'plpython3u') = 0 OR
+       (SELECT count(*) FROM pg_available_extensions WHERE name = 'plpython2u') = 0 OR
+       (SELECT count(*) FROM pg_available_extensions WHERE name = 'plpythonu') = 0 OR
+       (SELECT count(*) FROM pg_available_extensions WHERE name = 'convert_python3') = 0
+       AS skip_test \gset
+\if :skip_test
+\quit
+\endif
+
+-- Funny formatting of some commands below is to prevent
+-- regress-python3-mangle.mk from being "helpful".
+
+-- install the additional extensions
+CREATE EXTENSION
+plpython2u;
+CREATE EXTENSION
+plpythonu;
+CREATE EXTENSION convert_python3;
+
+-- create some functions that need conversion
+create function convert1() returns int
+as 'return 123l'
+language
+plpython2u
+immutable;
+
+-- disable syntax checking since if plpythonu is python 3, this'll fail
+set check_function_bodies = false;
+
+create function convert2() returns int
+as 'return 123l'
+language
+plpythonu
+immutable;
+
+-- can't use \sf here because mangling would affect the LANGUAGE clauses
+select proname, lanname, prosrc
+from pg_proc p join pg_language l on prolang = l.oid
+where proname = 'convert1';
+
+select proname, lanname, prosrc
+from pg_proc p join pg_language l on prolang = l.oid
+where proname = 'convert2';
+
+call convert_python3_all();
+
+\sf convert1()
+\sf convert2()
+
+-- clean up
+DROP EXTENSION
+plpython2u;
+DROP EXTENSION
+plpythonu;
+DROP EXTENSION convert_python3;
diff --git a/src/pl/stub_plpython2/Makefile b/src/pl/stub_plpython2/Makefile
new file mode 100644
index 0000000..67e7559
--- /dev/null
+++ b/src/pl/stub_plpython2/Makefile
@@ -0,0 +1,38 @@
+# src/pl/stub_plpython2/Makefile
+
+# Note that this Makefile only builds and installs a quasi-dummy
+# version of plpython2.so.  The control and script files for the
+# plpythonu and plpython2u extensions are installed by ../plpython;
+# they are the same whether we're using real or stub plpython2.
+# (We'd probably not have this separate subdirectory at all, except
+# that Makefile.shlib can only build one shlib per directory.)
+
+subdir = src/pl/stub_plpython2
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+PGFILEDESC = "PL/Python - procedural language stub for Python 2"
+
+NAME = plpython2
+
+OBJS = \
+    $(WIN32RES) \
+    stub_plpython2.o
+
+include $(top_srcdir)/src/Makefile.shlib
+
+all: all-lib
+
+# Ensure parallel safety if a build is started in this directory
+$(OBJS): | submake-generated-headers
+
+install: all install-lib
+
+installdirs: installdirs-lib
+
+uninstall: uninstall-lib
+
+clean distclean: clean-lib
+    rm -f $(OBJS)
+
+maintainer-clean: distclean
diff --git a/src/pl/stub_plpython2/stub_plpython2.c b/src/pl/stub_plpython2/stub_plpython2.c
new file mode 100644
index 0000000..4ade88b
--- /dev/null
+++ b/src/pl/stub_plpython2/stub_plpython2.c
@@ -0,0 +1,105 @@
+/*
+ * PL/Python stub for Python 2, in an environment that has only Python 3
+ *
+ * Our strategy is to pass through "plpythonu" functions to Python 3,
+ * but throw a not-implemented error for "plpython2u".
+ *
+ * Pass-through is implemented by using dfmgr.c to look up the appropriate
+ * function in plpython3.so, rather than trying to resolve the reference
+ * directly.  This greatly simplifies building this as an independent
+ * shared library, and it ensures that we can't somehow pull in a different
+ * version of plpython3 (and thence libpython) than would get loaded for
+ * a plpython3u function.
+ *
+ * src/pl/stub_plpython2/stub_plpython2.c
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+
+#define PLPYTHON_LIBNAME "$libdir/plpython3"
+
+/*
+ * exported functions
+ */
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(plpython_validator);
+PG_FUNCTION_INFO_V1(plpython_call_handler);
+PG_FUNCTION_INFO_V1(plpython_inline_handler);
+
+PG_FUNCTION_INFO_V1(plpython2_validator);
+PG_FUNCTION_INFO_V1(plpython2_call_handler);
+PG_FUNCTION_INFO_V1(plpython2_inline_handler);
+
+
+Datum
+plpython_validator(PG_FUNCTION_ARGS)
+{
+    static PGFunction plpython3_validator = NULL;
+
+    if (plpython3_validator == NULL)
+        plpython3_validator =
+            load_external_function(PLPYTHON_LIBNAME,
+                                   "plpython3_validator",
+                                   true, NULL);
+
+    return (*plpython3_validator) (fcinfo);
+}
+
+Datum
+plpython_call_handler(PG_FUNCTION_ARGS)
+{
+    static PGFunction plpython3_call_handler = NULL;
+
+    if (plpython3_call_handler == NULL)
+        plpython3_call_handler =
+            load_external_function(PLPYTHON_LIBNAME,
+                                   "plpython3_call_handler",
+                                   true, NULL);
+
+    return (*plpython3_call_handler) (fcinfo);
+}
+
+Datum
+plpython_inline_handler(PG_FUNCTION_ARGS)
+{
+    static PGFunction plpython3_inline_handler = NULL;
+
+    if (plpython3_inline_handler == NULL)
+        plpython3_inline_handler =
+            load_external_function(PLPYTHON_LIBNAME,
+                                   "plpython3_inline_handler",
+                                   true, NULL);
+
+    return (*plpython3_inline_handler) (fcinfo);
+}
+
+Datum
+plpython2_validator(PG_FUNCTION_ARGS)
+{
+    /* It seems more convenient to do nothing here than throw an error. */
+    PG_RETURN_VOID();
+}
+
+Datum
+plpython2_call_handler(PG_FUNCTION_ARGS)
+{
+    ereport(ERROR,
+            (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+             errmsg("Python 2 is no longer supported"),
+             errhint("Convert the function to use plpython3u.")));
+    PG_RETURN_NULL();            /* keep compiler quiet */
+}
+
+Datum
+plpython2_inline_handler(PG_FUNCTION_ARGS)
+{
+    ereport(ERROR,
+            (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+             errmsg("Python 2 is no longer supported"),
+             errhint("Convert the DO block to use plpython3u.")));
+    PG_RETURN_NULL();            /* keep compiler quiet */
+}
diff --git a/src/tools/msvc/Install.pm b/src/tools/msvc/Install.pm
index 1a92ed2..cf8b1e2 100644
--- a/src/tools/msvc/Install.pm
+++ b/src/tools/msvc/Install.pm
@@ -154,6 +154,7 @@ sub Install
         my @pldirs             = ('src/pl/plpgsql/src');
         push @pldirs, "src/pl/plperl"   if $config->{perl};
         push @pldirs, "src/pl/plpython" if $config->{python};
+        push @pldirs, "src/pl/stub_plpython2" if $config->{python2_stub};
         push @pldirs, "src/pl/tcl"      if $config->{tcl};
         File::Find::find(
             {
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 727a8fd..64d7431 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -531,6 +531,13 @@ sub mkvcbuild
             'ltree',                        'contrib');
         $ltree_plpython->AddDefine(
             'PLPYTHON_LIBNAME="plpython' . $pymajorver . '"');
+
+        # Ignore --with-python2-stub if building real plpython2
+        $solution->{options}->{python2_stub} = 0 if $pymajorver < 3;
+    }
+    else
+    {
+        $solution->{options}->{python2_stub} = 0;
     }

     if ($solution->{options}->{perl})
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 6b4a6ee..848e118 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -1145,6 +1145,7 @@ sub GetFakeConfigure
     $cfg .= ' --with-tcl'           if ($self->{options}->{tcl});
     $cfg .= ' --with-perl'          if ($self->{options}->{perl});
     $cfg .= ' --with-python'        if ($self->{options}->{python});
+    $cfg .= ' --with-python2-stub'  if ($self->{options}->{python2_stub});

     return $cfg;
 }
diff --git a/src/tools/msvc/clean.bat b/src/tools/msvc/clean.bat
index 672bb2d..4ad5f07 100755
--- a/src/tools/msvc/clean.bat
+++ b/src/tools/msvc/clean.bat
@@ -26,6 +26,7 @@ if exist src\interfaces\ecpg\test\win32ver.rc del /q src\interfaces\ecpg\test\wi
 if exist src\pl\plperl\win32ver.rc del /q src\pl\plperl\win32ver.rc
 if exist src\pl\plpgsql\src\win32ver.rc del /q src\pl\plpgsql\src\win32ver.rc
 if exist src\pl\plpython\win32ver.rc del /q src\pl\plpython\win32ver.rc
+if exist src\pl\stub_plpython2\win32ver.rc del /q src\pl\stub_plpython2\win32ver.rc
 if exist src\pl\tcl\win32ver.rc del /q src\pl\tcl\win32ver.rc
 if exist src\test\isolation\win32ver.rc del /q src\test\isolation\win32ver.rc
 if exist src\test\regress\win32ver.rc del /q src\test\regress\win32ver.rc
diff --git a/src/tools/msvc/config_default.pl b/src/tools/msvc/config_default.pl
index 2ef2cfc..d7f9b60 100644
--- a/src/tools/msvc/config_default.pl
+++ b/src/tools/msvc/config_default.pl
@@ -16,6 +16,7 @@ our $config = {
     tcl       => undef,    # --with-tcl=<path>
     perl      => undef,    # --with-perl=<path>
     python    => undef,    # --with-python=<path>
+    python2_stub => 1,     # --with-python2-stub (ignored unless Python is v3)
     openssl   => undef,    # --with-openssl=<path>
     uuid      => undef,    # --with-uuid=<path>
     xml       => undef,    # --with-libxml=<path>