Thread: submake-errcodes

Hi,

I used to run

"
cd src/backend
make submake-errcodes
"

in the RPM spec file, but looks like it was removed recently. Is that replaced
with something else, or removed completely?

Regards,
--
Devrim Gündüz
EnterpriseDB: https://www.enterprisedb.com
PostgreSQL Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

Re: Devrim Gündüz 2018-04-10 <1523353963.8169.26.camel@gunduz.org>
> I used to run
> 
> "
> cd src/backend
> make submake-errcodes
> "
> 
> in the RPM spec file, but looks like it was removed recently. Is that replaced
> with something else, or removed completely?

It is gone:
https://git.postgresql.org/pg/commitdiff/3b8f6e75f3c8c6d192621f21624cc8cee04ec3cb

... but that doesn't seem to work transparently either, in an out-of-tree build:

   debian/rules override_dh_auto_build-arch
make[1]: Verzeichnis „/home/cbe/projects/postgresql/pg/master“ wird betreten
/usr/bin/make -C build-py3/src/pl/plpython
make[2]: Verzeichnis „/home/cbe/projects/postgresql/pg/master/build-py3/src/pl/plpython“ wird betreten
/usr/bin/msgfmt -c -o po/cs.mo /home/cbe/projects/postgresql/pg/master/build-py3/../src/pl/plpython/po/cs.po
/usr/bin/msgfmt -c -o po/de.mo /home/cbe/projects/postgresql/pg/master/build-py3/../src/pl/plpython/po/de.po
/usr/bin/msgfmt -c -o po/es.mo /home/cbe/projects/postgresql/pg/master/build-py3/../src/pl/plpython/po/es.po
/usr/bin/msgfmt -c -o po/fr.mo /home/cbe/projects/postgresql/pg/master/build-py3/../src/pl/plpython/po/fr.po
/usr/bin/msgfmt -c -o po/it.mo /home/cbe/projects/postgresql/pg/master/build-py3/../src/pl/plpython/po/it.po
/usr/bin/msgfmt -c -o po/ja.mo /home/cbe/projects/postgresql/pg/master/build-py3/../src/pl/plpython/po/ja.po
/usr/bin/msgfmt -c -o po/ko.mo /home/cbe/projects/postgresql/pg/master/build-py3/../src/pl/plpython/po/ko.po
/usr/bin/msgfmt -c -o po/pl.mo /home/cbe/projects/postgresql/pg/master/build-py3/../src/pl/plpython/po/pl.po
/usr/bin/msgfmt -c -o po/pt_BR.mo /home/cbe/projects/postgresql/pg/master/build-py3/../src/pl/plpython/po/pt_BR.po
/usr/bin/msgfmt -c -o po/ru.mo /home/cbe/projects/postgresql/pg/master/build-py3/../src/pl/plpython/po/ru.po
/usr/bin/msgfmt -c -o po/sv.mo /home/cbe/projects/postgresql/pg/master/build-py3/../src/pl/plpython/po/sv.po
/usr/bin/msgfmt -c -o po/zh_CN.mo /home/cbe/projects/postgresql/pg/master/build-py3/../src/pl/plpython/po/zh_CN.po
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security-fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -g -O2
-fdebug-prefix-map=/home/cbe/projects/postgresql/pg/master=.-fstack-protector-strong -Wformat -Werror=format-security
-fno-omit-frame-pointer-fPIC -I. -I/home/cbe/projects/postgresql/pg/master/build-py3/../src/pl/plpython
-I/usr/include/python3.6m-I../../../src/include -I/home/cbe/projects/postgresql/pg/master/build-py3/../src/include
-Wdate-time-D_FORTIFY_SOURCE=2 -D_GNU_SOURCE   -c -o plpy_cursorobject.o
/home/cbe/projects/postgresql/pg/master/build-py3/../src/pl/plpython/plpy_cursorobject.c
In file included from /home/cbe/projects/postgresql/pg/master/build-py3/../src/include/postgres.h:47:0,
                 from /home/cbe/projects/postgresql/pg/master/build-py3/../src/pl/plpython/plpy_cursorobject.c:7:
/home/cbe/projects/postgresql/pg/master/build-py3/../src/include/utils/elog.h:71:10: fatal error: utils/errcodes.h:
Dateioder Verzeichnis nicht gefunden
 
 #include "utils/errcodes.h"
          ^~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [<eingebaut>: plpy_cursorobject.o] Fehler 1
make[2]: Verzeichnis „/home/cbe/projects/postgresql/pg/master/build-py3/src/pl/plpython“ wird verlassen

Christoph

Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= <devrim@gunduz.org> writes:
> I used to run
> "
> cd src/backend
> make submake-errcodes
> "
> in the RPM spec file, but looks like it was removed recently. Is that replaced
> with something else, or removed completely?

You could replace it with submake-generated-headers, since that's more
general, but in principle you shouldn't need anything because that
target is invoked automatically as of yesterday.  What's the larger
context here --- why do you need any of this?

            regards, tom lane

Hi,

On Tue, 2018-04-10 at 10:01 -0400, Tom Lane wrote:
> You could replace it with submake-generated-headers, since that's more
> general, but in principle you shouldn't need anything because that
> target is invoked automatically as of yesterday.  What's the larger
> context here --- why do you need any of this?

Good question -- IIRC we used it to build PL/Python. Just confirmed that
removing from v10 spec file does not break anything.

However, as Christoph wrote, builds against git master fail:

==========================================
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security-fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -O2 -g -pipe -Wall -Werror=format-security
-Wp,-D_FORTIFY_SOURCE=2-fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1-m64 -mtune=generic -fasynchronous-unwind-tables -fPIC -I. -I.
-I/usr/include/python3.6m-I../../../src/include  -D_GNU_SOURCE -I/usr/include/libxml2  -I/usr/include  -c -o
plpy_resultobject.oplpy_resultobject.c 
In file included from ../../../src/include/postgres.h:47:0,
                 from plpy_cursorobject.c:7:
../../../src/include/utils/elog.h:71:10: fatal error: utils/errcodes.h: No such file or directory
 #include "utils/errcodes.h"
          ^~~~~~~~~~~~~~~~~~
In file included from ../../../src/include/postgres.h:47:0,
                 from plpy_procedure.c:7:
../../../src/include/utils/elog.h:71:10: fatal error: utils/errcodes.h: No such file or directory
 #include "utils/errcodes.h"
          ^~~~~~~~~~~~~~~~~~
compilation terminated.
compilation terminated.
make[1]: *** [<builtin>: plpy_procedure.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [<builtin>: plpy_cursorobject.o] Error 1
==========================================

Regards,

--
Devrim Gündüz
EnterpriseDB: https://www.enterprisedb.com
PostgreSQL Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= <devrim@gunduz.org> writes:
> On Tue, 2018-04-10 at 10:01 -0400, Tom Lane wrote:
>> You could replace it with submake-generated-headers, since that's more
>> general, but in principle you shouldn't need anything because that
>> target is invoked automatically as of yesterday.  What's the larger
>> context here --- why do you need any of this?

> However, as Christoph wrote, builds against git master fail:

Hm ... you're cd'ing into src/pl/plpython and issuing "make all"?
That works for me.

... or, wait ... with -j it doesn't.  That's strange, will look.

            regards, tom lane

I wrote:
> Hm ... you're cd'ing into src/pl/plpython and issuing "make all"?
> That works for me.
> ... or, wait ... with -j it doesn't.  That's strange, will look.

So after a bit of digging, it seems that the locution

all: submake-generated-headers

doesn't result in ensuring that submake-generated-headers is complete
before we go to build the other targets required by "all"; it only
says that submake-generated-headers must be complete before we execute
the (empty) list of commands attached to the "all" target.

I'd tested high-j runs pretty carefully at top level, but it turns
out that that works because in both the toplevel GNUmakefile and
src/Makefile, all the interesting work happens in recursive sub-makes,
and we force the ordering of those properly with the dependencies on
the recursive make rules:

$(1)-$(2)-recurse: $(if $(filter all install, $(3)), submake-generated-headers) $(if $(filter check, $(3)),
temp-install)
    $$(MAKE) -C $(2) $(3)

If you go to, eg, src/pl/plpython and issue "make -j", there's nothing
to prevent the builds of object files from happening before the header
build finishes.  There *was* something there before:

$(OBJS): | submake-generated-headers

but I took it out thinking it was no longer needed.

The short-term solution seems to be to put that back, but that's sort
of annoying because it means this isn't a bulletproof solution.  It
will only work for builds started in one of the directories that we
take the trouble to put this defense into, and I can't see doing that
everywhere.  Still, such things didn't work reliably before either
except in these few directories, so maybe it won't matter.

            regards, tom lane

Tom Lane wrote:

> $(OBJS): | submake-generated-headers
> 
> but I took it out thinking it was no longer needed.
> 
> The short-term solution seems to be to put that back, but that's sort
> of annoying because it means this isn't a bulletproof solution.  It
> will only work for builds started in one of the directories that we
> take the trouble to put this defense into, and I can't see doing that
> everywhere.  Still, such things didn't work reliably before either
> except in these few directories, so maybe it won't matter.

Isn't it possible to put it in some common.mk file rather than each
individual Makefile?

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> Tom Lane wrote:
>> $(OBJS): | submake-generated-headers
>> but I took it out thinking it was no longer needed.
>> 
>> The short-term solution seems to be to put that back, but that's sort
>> of annoying because it means this isn't a bulletproof solution.

> Isn't it possible to put it in some common.mk file rather than each
> individual Makefile?

I thought about putting the above into Makefile.global, but it'd only
work in cases where $(OBJS) gets set before including Makefile.global,
which turns out to be nearly noplace.  So we'd still end up touching
an awful lot of makefiles to make it work, for relatively little
practical benefit.

            regards, tom lane

Re: Tom Lane 2018-04-10 <24426.1523387062@sss.pgh.pa.us>
> The short-term solution seems to be to put that back, but that's sort
> of annoying because it means this isn't a bulletproof solution.  It
> will only work for builds started in one of the directories that we
> take the trouble to put this defense into, and I can't see doing that
> everywhere.  Still, such things didn't work reliably before either
> except in these few directories, so maybe it won't matter.

Is that supposed to work now, without explicitly invoking sub-generated-headers?
It still doesn't work on current HEAD:

/home/cbe/projects/postgresql/pg/master/build-py3/../src/include/utils/elog.h:71:10: fatal error: utils/errcodes.h:
Dateioder Verzeichnis nicht gefunden
 
 #include "utils/errcodes.h"
          ^~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [<eingebaut>: plpy_cursorobject.o] Fehler 1
make[2]: Verzeichnis „/home/cbe/projects/postgresql/pg/master/build-py3/src/pl/plpython“ wird verlassen

The other directories we'd need it are contrib/{hstore,jsonb,ltree}_plpython.

Christoph

Christoph Berg <myon@debian.org> writes:
> It still doesn't work on current HEAD:

*What* still doesn't work on current HEAD?  I don't know what commands
you are running to get this.

            regards, tom lane

Hi,

On Wed, 2018-04-11 at 09:38 -0400, Tom Lane wrote:
> *What* still doesn't work on current HEAD?  I don't know what commands
> you are running to get this.

I think my build and Christoph's builds fail because of the same reason again
(the same as yesterday):

==================================
+ cd src/pl/plpython
+ make all
<snip>

gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security-fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -O2 -g -pipe -Wall -Werror=format-security
-Wp,-D_FORTIFY_SOURCE=2-fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1-m64 -mtune=generic -fasynchronous-unwind-tables -fPIC -I. -I.
-I/usr/include/python3.6m-I../../../src/include  -D_GNU_SOURCE -I/usr/include/libxml2  -I/usr/include  -c -o
plpy_cursorobject.oplpy_cursorobject.c 
In file included from ../../../src/include/postgres.h:47:0,
                 from plpy_cursorobject.c:7:
../../../src/include/utils/elog.h:71:10: fatal error: utils/errcodes.h: No such file or directory
 #include "utils/errcodes.h"
          ^~~~~~~~~~~~~~~~~~
compilation terminated.
make[1]: *** [<builtin>: plpy_cursorobject.o] Error 1
===========

Regards,
--
Devrim Gündüz
EnterpriseDB: https://www.enterprisedb.com
PostgreSQL Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= <devrim@gunduz.org> writes:
> On Wed, 2018-04-11 at 09:38 -0400, Tom Lane wrote:
>> *What* still doesn't work on current HEAD?  I don't know what commands
>> you are running to get this.

> I think my build and Christoph's builds fail because of the same reason again
> (the same as yesterday):

> + cd src/pl/plpython
> + make all

That works for me:

$ git clean -dfx
$ ./configure ... --with-python and other stuff ...
$ cd src/pl/plpython
$ make all
make -C ../../../src/backend generated-headers
make[1]: Entering directory `/home/postgres/pgsql/src/backend'
make -C catalog distprep generated-header-symlinks
make[2]: Entering directory `/home/postgres/pgsql/src/backend/catalog'
...
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security-fno-strict-aliasing -fwrapv -g -O2 -fPIC -I. -I. -I/usr/include/python2.6 -I../../../src/include
-D_GNU_SOURCE  -c -o plpy_util.o plpy_util.c 
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security-fno-strict-aliasing -fwrapv -g -O2 -fPIC -shared -o plpython2.so plpy_cursorobject.o plpy_elog.o
plpy_exec.oplpy_main.o plpy_planobject.o plpy_plpymodule.o plpy_procedure.o plpy_resultobject.o plpy_spi.o
plpy_subxactobject.oplpy_typeio.o plpy_util.o  -L../../../src/port -L../../../src/common    -Wl,--as-needed
-Wl,-rpath,'/usr/lib64',--enable-new-dtags -L/usr/lib64 -lpython2.6 -lpthread -ldl  -lutil -lm   

You sure you're on 31f1f0bb4fd642643994d35c35ecb5b929711a99 or later?
Which gmake version is this?

            regards, tom lane

On Wed, Apr 11, 2018 at 10:16:24AM -0400, Tom Lane wrote:
> That works for me:
>
> $ git clean -dfx
> $ ./configure ... --with-python and other stuff ...
> $ cd src/pl/plpython
> $ make all
> make -C ../../../src/backend generated-headers
> make[1]: Entering directory `/home/postgres/pgsql/src/backend'
> make -C catalog distprep generated-header-symlinks
> make[2]: Entering directory `/home/postgres/pgsql/src/backend/catalog'
> ...
> gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels
-Wmissing-format-attribute-Wformat-security -fno-strict-aliasing -fwrapv -g -O2 -fPIC -I. -I. -I/usr/include/python2.6
-I../../../src/include -D_GNU_SOURCE   -c -o plpy_util.o plpy_util.c 
> gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels
-Wmissing-format-attribute-Wformat-security -fno-strict-aliasing -fwrapv -g -O2 -fPIC -shared -o plpython2.so
plpy_cursorobject.oplpy_elog.o plpy_exec.o plpy_main.o plpy_planobject.o plpy_plpymodule.o plpy_procedure.o
plpy_resultobject.oplpy_spi.o plpy_subxactobject.o plpy_typeio.o plpy_util.o  -L../../../src/port -L../../../src/common
  -Wl,--as-needed -Wl,-rpath,'/usr/lib64',--enable-new-dtags  -L/usr/lib64 -lpython2.6 -lpthread -ldl  -lutil -lm   
>
> You sure you're on 31f1f0bb4fd642643994d35c35ecb5b929711a99 or later?
> Which gmake version is this?

For what it's worth, all those combinations work for me when on
d1e90792:
./configure blah
cd src/pl/plpython/
make -j 4 [ all | install ]

But this does not work:
./configure blah
cd src/pl/plpython/
make -j 4 check

make[2]: *** [Makefile:89: keywords_srv.o] Error 1
make[2]: *** Deleting file 'keywords_srv.o'
make[2]: Leaving directory '/home/ioltas/git/postgres/src/common'
make[1]: *** [../../../src/Makefile.global:568: submake-libpgport] Error 2

Thanks,
--
Michael

Re: Michael Paquier 2018-04-12 <20180411235843.GG32449@paquier.xyz>
> > You sure you're on 31f1f0bb4fd642643994d35c35ecb5b929711a99 or later?
> > Which gmake version is this?

GNU Make 4.2.1

> For what it's worth, all those combinations work for me when on
> d1e90792:
> ./configure blah
> cd src/pl/plpython/
> make -j 4 [ all | install ]
> 
> But this does not work:
> ./configure blah
> cd src/pl/plpython/
> make -j 4 check
> 
> make[2]: *** [Makefile:89: keywords_srv.o] Error 1
> make[2]: *** Deleting file 'keywords_srv.o'
> make[2]: Leaving directory '/home/ioltas/git/postgres/src/common'
> make[1]: *** [../../../src/Makefile.global:568: submake-libpgport] Error 2

Most of these work for me as well. The actual incantation via
debian/rules fails, but I couldn't really narrow down what the
difference is. Manually invoking the debian/rules targets:

$ debian/rules stamp/configure-build-py3
mkdir -p stamp build-py3
cd build-py3 && ../configure \
           --with-python \
           PYTHON=/usr/bin/python3 \
           --mandir=/usr/share/postgresql/11/man --docdir=/usr/share/doc/postgresql-doc-11
--sysconfdir=/etc/postgresql-common--datarootdir=/usr/share/ --datadir=/usr/share/postgresql/11
--bindir=/usr/lib/postgresql/11/bin--libdir=/usr/lib/x86_64-linux-gnu/ --libexecdir=/usr/lib/postgresql/
--includedir=/usr/include/postgresql/--with-extra-version=" (Debian 11~~devel-1)" --enable-nls
--enable-integer-datetimes--enable-thread-safety --enable-tap-tests --enable-debug  --disable-rpath --with-llvm
--with-uuid=e2fs--with-gnu-ld --with-pgport=5432 --with-system-tzdata=/usr/share/zoneinfo --with-systemd  CFLAGS='-g
-O2-fdebug-prefix-map=/srv/projects/postgresql/pg/master=. -fstack-protector-strong -Wformat -Werror=format-security
-fno-omit-frame-pointer'LDFLAGS='-Wl,-z,relro -Wl,-z,now'
 
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
...
checking for PYTHON... /usr/bin/python3
configure: using python 3.6.5 (default, Apr  1 2018, 05:46:30)
checking for Python distutils module... yes
checking Python configuration directory... /usr/lib/python3.6/config-3.6m-x86_64-linux-gnu
checking Python include directories... -I/usr/include/python3.6m
checking how to link an embedded Python application... -L/usr/lib/python3.6/config-3.6m-x86_64-linux-gnu -lpython3.6m
-lpthread-ldl  -lutil -lm
 
...
preparing build tree... done
configure: creating ./config.status
config.status: creating GNUmakefile
config.status: creating src/Makefile.global
config.status: creating src/include/pg_config.h
config.status: creating src/include/pg_config_ext.h
config.status: creating src/interfaces/ecpg/include/ecpg_config.h
config.status: linking ../src/backend/port/tas/dummy.s to src/backend/port/tas.s
config.status: linking ../src/backend/port/dynloader/linux.c to src/backend/port/dynloader.c
config.status: linking ../src/backend/port/posix_sema.c to src/backend/port/pg_sema.c
config.status: linking ../src/backend/port/sysv_shmem.c to src/backend/port/pg_shmem.c
config.status: linking ../src/backend/port/dynloader/linux.h to src/include/dynloader.h
config.status: linking ../src/include/port/linux.h to src/include/pg_config_os.h
config.status: linking ../src/makefiles/Makefile.linux to src/Makefile.port
touch "stamp/configure-build-py3"

$ debian/rules stamp/build-py3
/usr/bin/make -C build-py3/src/pl/plpython
make[1]: Verzeichnis „/srv/projects/postgresql/pg/master/build-py3/src/pl/plpython“ wird betreten
/usr/bin/msgfmt -c -o po/cs.mo /srv/projects/postgresql/pg/master/build-py3/../src/pl/plpython/po/cs.po
... more msgfmt
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security-fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -g -O2
-fdebug-prefix-map=/srv/projects/postgresql/pg/master=.-fstack-protector-strong -Wformat -Werror=format-security
-fno-omit-frame-pointer-fPIC -I. -I/srv/projects/postgresql/pg/master/build-py3/../src/pl/plpython
-I/usr/include/python3.6m-I../../../src/include -I/srv/projects/postgresql/pg/master/build-py3/../src/include
-Wdate-time-D_FORTIFY_SOURCE=2 -D_GNU_SOURCE   -c -o plpy_cursorobject.o
/srv/projects/postgresql/pg/master/build-py3/../src/pl/plpython/plpy_cursorobject.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security-fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -g -O2
-fdebug-prefix-map=/srv/projects/postgresql/pg/master=.-fstack-protector-strong -Wformat -Werror=format-security
-fno-omit-frame-pointer-fPIC -I. -I/srv/projects/postgresql/pg/master/build-py3/../src/pl/plpython
-I/usr/include/python3.6m-I../../../src/include -I/srv/projects/postgresql/pg/master/build-py3/../src/include
-Wdate-time-D_FORTIFY_SOURCE=2 -D_GNU_SOURCE   -c -o plpy_elog.o
/srv/projects/postgresql/pg/master/build-py3/../src/pl/plpython/plpy_elog.c
In file included from /srv/projects/postgresql/pg/master/build-py3/../src/include/postgres.h:47:0,
                 from /srv/projects/postgresql/pg/master/build-py3/../src/pl/plpython/plpy_cursorobject.c:7:
/srv/projects/postgresql/pg/master/build-py3/../src/include/utils/elog.h:71:10: fatal error: utils/errcodes.h: Datei
oderVerzeichnis nicht gefunden
 
 #include "utils/errcodes.h"
          ^~~~~~~~~~~~~~~~~~
compilation terminated.
In file included from /srv/projects/postgresql/pg/master/build-py3/../src/include/postgres.h:47:0,
                 from /srv/projects/postgresql/pg/master/build-py3/../src/pl/plpython/plpy_elog.c:7:
/srv/projects/postgresql/pg/master/build-py3/../src/include/utils/elog.h:71:10: fatal error: utils/errcodes.h: Datei
oderVerzeichnis nicht gefunden
 
 #include "utils/errcodes.h"
          ^~~~~~~~~~~~~~~~~~
compilation terminated.
make[1]: *** [<eingebaut>: plpy_cursorobject.o] Fehler 1
make[1]: *** Es wird auf noch nicht beendete Prozesse gewartet....
make[1]: *** [<eingebaut>: plpy_elog.o] Fehler 1
make[1]: Verzeichnis „/srv/projects/postgresql/pg/master/build-py3/src/pl/plpython“ wird verlassen
make: *** [debian/rules:156: stamp/build-py3] Fehler 2


Invoking the commands manually works. debian/rules is
https://salsa.debian.org/postgresql/postgresql/blob/11/debian/rules

(Known bug: ltree_plpython is not yet built for python 3)

Christoph

Christoph Berg <myon@debian.org> writes:
> Most of these work for me as well. The actual incantation via
> debian/rules fails, but I couldn't really narrow down what the
> difference is. Manually invoking the debian/rules targets:

For a moment I thought that this might be the critical difference:
> preparing build tree... done
ie that you're doing a VPATH build.  But no, that case works as
expected too, for me.

The only conclusion I can come to is that you're using sources that
predate my recent makefile fixes, particularly 3b8f6e75f.  Please
double-check that.

            regards, tom lane

Michael Paquier <michael@paquier.xyz> writes:
> But this does not work:
> ./configure blah
> cd src/pl/plpython/
> make -j 4 check

Hm.  That shows yet another parallel-safety hazard, which can be resolved
like this:

diff --git a/src/pl/plpython/Makefile b/src/pl/plpython/Makefile
index 653fe64..c17015b 100644
--- a/src/pl/plpython/Makefile
+++ b/src/pl/plpython/Makefile
@@ -131,7 +131,7 @@ installcheck: submake-pg-regress
 
 
 .PHONY: submake-pg-regress
-submake-pg-regress:
+submake-pg-regress: | submake-generated-headers
     $(MAKE) -C $(top_builddir)/src/test/regress pg_regress$(X)
 
 clean distclean: clean-lib

to ensure that we finish the generated-headers work before launching
that child Make run.

I'm beginning to get dissatisfied with this approach of expecting the
topmost Make run to do the generated-headers work; it's bleeding into
more places than I'd hoped.  I don't see any really good alternative
though.  If we allow the child runs to try to do it, we're going to
have issues with parallel runs clobbering each others' output.  It's
somewhat surprising that that didn't occur more often before; the
only real difference since the bootstrap data restructuring is that
we have more generated headers than we used to.

I guess the good news is that the supported cases will be a whole lot
more bulletproof against high -j counts than they were before.  I never
used to dare going beyond -j8, because builds tended to fall over.

            regards, tom lane

Hi Tom,

On Wed, 2018-04-11 at 10:16 -0400, Tom Lane wrote:
> You sure you're on 31f1f0bb4fd642643994d35c35ecb5b929711a99 or later?

To make sure, I am using latest git snapshot:

https://download.postgresql.org/pub/snapshot/dev/postgresql-snapshot.tar.bz2

and still same error. Easy to reproduce on F-27 box.

> Which gmake version is this?

4.2.1

Regards,
--
Devrim Gündüz
EnterpriseDB: https://www.enterprisedb.com
PostgreSQL Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

>>>>> "Tom" == Tom Lane <tgl@sss.pgh.pa.us> writes:

 Tom> I'm beginning to get dissatisfied with this approach of expecting
 Tom> the topmost Make run to do the generated-headers work

Is it worth exploring the idea of changing to a non-recursive style of
makefile?

-- 
Andrew (irc:RhodiumToad)

Andrew Gierth <andrew@tao11.riddles.org.uk> writes:
> "Tom" == Tom Lane <tgl@sss.pgh.pa.us> writes:
>  Tom> I'm beginning to get dissatisfied with this approach of expecting
>  Tom> the topmost Make run to do the generated-headers work

> Is it worth exploring the idea of changing to a non-recursive style of
> makefile?

Not post-feature-freeze, for sure.  Whether it's worth the work as a
long-term project, I dunno.

            regards, tom lane

Andrew Gierth wrote:
> >>>>> "Tom" == Tom Lane <tgl@sss.pgh.pa.us> writes:
> 
>  Tom> I'm beginning to get dissatisfied with this approach of expecting
>  Tom> the topmost Make run to do the generated-headers work
> 
> Is it worth exploring the idea of changing to a non-recursive style of
> makefile?

I looked onto this a while ago.  It seems easily doable for the backend
code proper, modulo potential command-line length issues; but anything
involving shared libraries requires Makefile.shlib to be involved, which
seems painful to resolve.  Once I realized the shlib problem, I stopped
trying, but if you find some way around that it may be worthwhile.

I'm altogether ignorant on how might we use it for the headers problem,
mind.  I only tried to tackle the main executable.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Thu, Apr 12, 2018 at 11:00 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Andrew Gierth <andrew@tao11.riddles.org.uk> writes:
>> Is it worth exploring the idea of changing to a non-recursive style of
>> makefile?
>
> Not post-feature-freeze, for sure.  Whether it's worth the work as a
> long-term project, I dunno.

I've been taking a look at this in my limited free time, so I might as
well publicly register my interest here. Moving to non-recursive Make
will probably be quite a bit of work/pain, but I also think it's
probably worth it in the end.

(We make heavy use of automated builds, and having a complete
dependency graph should speed those up considerably, in addition to
the maintenance benefits already implied here.)

--Jacob

Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= <devrim@gunduz.org> writes:
> ... still same error. Easy to reproduce on F-27 box. 

I don't have F27 at hand, but I tried F26 and F28, and I can't reproduce
on either one.  I tried various combinations of python2 versus python3,
in-tree build versus VPATH from bare checkout versus VPATH from
distprep'd tree, and they all work for me.

Just to clarify, I'm experimenting with

$ git clean -dfx
$ ./configure ... --with-python [ PYTHON=/usr/bin/python3 ]
$ cd src/pl/plpython/
$ make -j25

and variants of that, and what I get as a result is a make trace
starting with

make -C ../../../src/backend generated-headers

If you're not seeing that, something's very wrong, and I do not
know what.

[ time passes ]

... or then again, maybe I do.  Is it possible that your build
recipe involves invoking our makefiles from an outer "make" run?
If so, maybe you need to explicitly set MAKELEVEL=0 when invoking
our build, to keep it from thinking it is a sub-make.  Not sure
about whether it'd be wise to reset MAKEFLAGS as well.

            regards, tom lane

>>>>> "Alvaro" == Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

 Alvaro> I'm altogether ignorant on how might we use it for the headers
 Alvaro> problem, mind. I only tried to tackle the main executable.

Solving the headers problem would seem to require making things
non-recursive at the topmost level rather than trying to do only a
partial conversion.

-- 
Andrew (irc:RhodiumToad)

On 2018-04-12 11:22:45 -0700, Jacob Champion wrote:
> On Thu, Apr 12, 2018 at 11:00 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Andrew Gierth <andrew@tao11.riddles.org.uk> writes:
> >> Is it worth exploring the idea of changing to a non-recursive style of
> >> makefile?
> >
> > Not post-feature-freeze, for sure.  Whether it's worth the work as a
> > long-term project, I dunno.
> 
> I've been taking a look at this in my limited free time, so I might as
> well publicly register my interest here. Moving to non-recursive Make
> will probably be quite a bit of work/pain, but I also think it's
> probably worth it in the end.

Yea, it'd sure be nice.  Whether it's worth the pain or whether the time
is better spent moving to cmake or such, I'm not sure.

Greetings,

Andres Freund

Re: Tom Lane 2018-04-12 <10354.1523558160@sss.pgh.pa.us>
> ... or then again, maybe I do.  Is it possible that your build
> recipe involves invoking our makefiles from an outer "make" run?
> If so, maybe you need to explicitly set MAKELEVEL=0 when invoking
> our build, to keep it from thinking it is a sub-make.  Not sure
> about whether it'd be wise to reset MAKEFLAGS as well.

I don't know about Devrim's case, but debian/rules is indeed a
makefile. Will look closer again once I'm back from pgconf.de, thanks
for the hint.

Christoph

Hi Tom,

On Thu, 2018-04-12 at 14:36 -0400, Tom Lane wrote:
> I don't have F27 at hand, but I tried F26 and F28, and I can't reproduce
> on either one.

I uploaded the SRPM for you:

https://gunduz.org/temp/postgresql11-11.0-20180415_1PGDG.f27.src.rpm

I built this SRPM using daily snapshot at:
https://download.postgresql.org/pub/snapshot/dev/

When you rebuild it, you'll see the error.

Regards,

--
Devrim Gündüz
EnterpriseDB: https://www.enterprisedb.com
PostgreSQL Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

Re: To Tom Lane 2018-04-12 <20180412202717.GA32601@msg.df7cb.de>
> > ... or then again, maybe I do.  Is it possible that your build
> > recipe involves invoking our makefiles from an outer "make" run?
> > If so, maybe you need to explicitly set MAKELEVEL=0 when invoking
> > our build, to keep it from thinking it is a sub-make.  Not sure
> > about whether it'd be wise to reset MAKEFLAGS as well.
> 
> I don't know about Devrim's case, but debian/rules is indeed a
> makefile. Will look closer again once I'm back from pgconf.de, thanks
> for the hint.

Fwiw, setting MAKELEVEL=0 worked. Thanks!

https://salsa.debian.org/postgresql/postgresql/blob/11/debian/rules#L147-148

Christoph

Hi,

On Mon, 2018-04-16 at 09:24 +0200, Christoph Berg wrote:
> Fwiw, setting MAKELEVEL=0 worked. Thanks!

Great, it solved my problem as well! Thanks Tom.

Regards,
--
Devrim Gündüz
EnterpriseDB: https://www.enterprisedb.com
PostgreSQL Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR