Thread: pgsql: Build all Flex files standalone
Build all Flex files standalone The proposed Meson build system will need a way to ignore certain generated files in order to coexist with the autoconf build system, and C files generated by Flex which are #include'd into .y files make this more difficult. In similar vein to 72b1e3a21, arrange for all Flex C files to compile to their own .o targets. Reviewed by Andres Freund Discussion: https://www.postgresql.org/message-id/20220810171935.7k5zgnjwqzalzmtm%40awork3.anarazel.de Discussion: https://www.postgresql.org/message-id/CAFBsxsF8Gc2StS3haXofshHCzqNMRXiSxvQEYGwnFsTmsdwNeg@mail.gmail.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/dac048f71ebbcf2f980d280711f8ff8001331c5d Modified Files -------------- contrib/cube/.gitignore | 1 + contrib/cube/Makefile | 16 +++++---- contrib/cube/cube.c | 6 ++-- contrib/cube/cubedata.h | 6 ++-- contrib/cube/cubeparse.y | 6 +--- contrib/cube/cubescan.l | 44 ++++++++++++++--------- contrib/seg/.gitignore | 1 + contrib/seg/Makefile | 15 +++++--- contrib/seg/segparse.y | 3 -- contrib/seg/segscan.l | 28 +++++++++------ src/backend/Makefile | 7 ++-- src/backend/bootstrap/.gitignore | 1 + src/backend/bootstrap/Makefile | 11 ++++-- src/backend/bootstrap/bootparse.y | 2 -- src/backend/bootstrap/bootscanner.l | 60 +++++++++++++++++-------------- src/backend/replication/.gitignore | 2 ++ src/backend/replication/Makefile | 22 +++++++++--- src/backend/replication/repl_gram.y | 2 -- src/backend/replication/repl_scanner.l | 31 ++++++++++------ src/backend/replication/syncrep_gram.y | 2 -- src/backend/replication/syncrep_scanner.l | 21 +++++++---- src/backend/utils/adt/.gitignore | 1 + src/backend/utils/adt/Makefile | 11 ++++-- src/backend/utils/adt/jsonpath_gram.y | 27 +------------- src/backend/utils/adt/jsonpath_internal.h | 32 +++++++++++++++++ src/backend/utils/adt/jsonpath_scan.l | 29 +++++++++------ src/backend/utils/misc/Makefile | 5 +-- src/backend/utils/misc/guc-file.l | 8 +++-- src/backend/utils/misc/guc.c | 2 -- src/bin/pgbench/.gitignore | 1 + src/bin/pgbench/Makefile | 13 +++++-- src/bin/pgbench/exprparse.y | 15 -------- src/bin/pgbench/exprscan.l | 12 ++++++- src/test/isolation/.gitignore | 1 + src/test/isolation/Makefile | 15 +++++--- src/test/isolation/specparse.y | 2 -- src/test/isolation/specscanner.l | 28 ++++++++++----- src/tools/pginclude/headerscheck | 8 +++++ 38 files changed, 307 insertions(+), 190 deletions(-)
Crake fails with headerscheck: Sep 04 01:56:01 In file included from /tmp/headerscheck.UNgmKy/test.c:2: Sep 04 01:56:01 /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: fatal error: jsonpath_gram.h: No such file or directory Sep 04 01:56:01 26 | #include "jsonpath_gram.h" Sep 04 01:56:01 | ^~~~~~~~~~~~~~~~~ Sep 04 01:56:01 compilation terminated. Sep 04 01:56:14 make: *** [GNUmakefile:138: headerscheck] Error 1 cpluspluscheck: Sep 04 01:57:19 In file included from /tmp/cpluspluscheck.HhRV5k/test.cpp:3: Sep 04 01:57:19 /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: fatal error: jsonpath_gram.h: No such file or directory Sep 04 01:57:19 26 | #include "jsonpath_gram.h" Sep 04 01:57:19 | ^~~~~~~~~~~~~~~~~ I don't get the same in an already-built tree. I also don't see an exception for gramparse.h, which I imagine would have the same issue? -- John Naylor EDB: http://www.enterprisedb.com
The second failure is on drongo (MSVC): "c:\\prog\\bf\\root\\HEAD\\pgsql.build\\pgsql.sln" (default target) (1) -> "c:\\prog\\bf\\root\\HEAD\\pgsql.build\\isolationtester.vcxproj" (default target) (13) -> (Link target) -> specparse.obj : error LNK2019: unresolved external symbol spec_yylex referenced in function spec_yyparse [c:\\prog\\bf\\root\\HEAD\\pgsql.build\\isolationtester.vcxproj] specparse.obj : error LNK2019: unresolved external symbol spec_yyerror referenced in function spec_yyparse [c:\\prog\\bf\\root\\HEAD\\pgsql.build\\isolationtester.vcxproj] .\\Release\\isolationtester\\isolationtester.exe : fatal error LNK1120: 2 unresolved externals [c:\\prog\\bf\\root\\HEAD\\pgsql.build\\isolationtester.vcxproj] For this, Mkvcbuild.pm specifies specparse.c explicitly, so maybe something like the following is needed: --- a/src/tools/msvc/Mkvcbuild.pm +++ b/src/tools/msvc/Mkvcbuild.pm @@ -362,6 +362,7 @@ sub mkvcbuild $isolation_tester->AddFile('src/test/isolation/specparse.y'); $isolation_tester->AddFile('src/test/isolation/specscanner.l'); $isolation_tester->AddFile('src/test/isolation/specparse.c'); + $isolation_tester->AddFile('src/test/isolation/specscanner.c'); $isolation_tester->AddIncludeDir('src/test/isolation'); -- John Naylor EDB: http://www.enterprisedb.com
On Sun, Sep 4, 2022 at 4:36 PM John Naylor <john.naylor@enterprisedb.com> wrote: > > For this, Mkvcbuild.pm specifies specparse.c explicitly, so maybe > something like the following is needed: > > --- a/src/tools/msvc/Mkvcbuild.pm > +++ b/src/tools/msvc/Mkvcbuild.pm > @@ -362,6 +362,7 @@ sub mkvcbuild > $isolation_tester->AddFile('src/test/isolation/specparse.y'); > $isolation_tester->AddFile('src/test/isolation/specscanner.l'); > $isolation_tester->AddFile('src/test/isolation/specparse.c'); > + $isolation_tester->AddFile('src/test/isolation/specscanner.c'); > $isolation_tester->AddIncludeDir('src/test/isolation'); This gets CI green (tip o' the hat to Thomas Munro for off-list advice on getting that working), so pushed. -- John Naylor EDB: http://www.enterprisedb.com
On 2022-09-04 Su 02:19, John Naylor wrote: > Crake fails with > > headerscheck: > Sep 04 01:56:01 In file included from /tmp/headerscheck.UNgmKy/test.c:2: > Sep 04 01:56:01 > /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: > fatal error: jsonpath_gram.h: No such file or directory > Sep 04 01:56:01 26 | #include "jsonpath_gram.h" > Sep 04 01:56:01 | ^~~~~~~~~~~~~~~~~ > Sep 04 01:56:01 compilation terminated. > Sep 04 01:56:14 make: *** [GNUmakefile:138: headerscheck] Error 1 > cpluspluscheck: > Sep 04 01:57:19 In file included from /tmp/cpluspluscheck.HhRV5k/test.cpp:3: > Sep 04 01:57:19 > /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: > fatal error: jsonpath_gram.h: No such file or directory > Sep 04 01:57:19 26 | #include "jsonpath_gram.h" > Sep 04 01:57:19 | ^~~~~~~~~~~~~~~~~ > > I don't get the same in an already-built tree. I also don't see an > exception for gramparse.h, which I imagine would have the same issue? Probably happens because crake does vpath builds. I think you're going to have to set up EXTRAINCLUDES for it - see near the bottom of the headerscheck script. Possibly something like src/backend/utils/adt/*) EXTRAINCLUDES="-I $builddir/src/backend/utils/adt" ;; cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
Hi, On 2022-09-04 18:04:13 +0700, John Naylor wrote: > On Sun, Sep 4, 2022 at 4:36 PM John Naylor <john.naylor@enterprisedb.com> wrote: > > > > For this, Mkvcbuild.pm specifies specparse.c explicitly, so maybe > > something like the following is needed: > > > > --- a/src/tools/msvc/Mkvcbuild.pm > > +++ b/src/tools/msvc/Mkvcbuild.pm > > @@ -362,6 +362,7 @@ sub mkvcbuild > > $isolation_tester->AddFile('src/test/isolation/specparse.y'); > > $isolation_tester->AddFile('src/test/isolation/specscanner.l'); > > $isolation_tester->AddFile('src/test/isolation/specparse.c'); > > + $isolation_tester->AddFile('src/test/isolation/specscanner.c'); > > $isolation_tester->AddIncludeDir('src/test/isolation'); > > This gets CI green (tip o' the hat to Thomas Munro for off-list advice > on getting that working), so pushed. It doesn't seem to have gotten all of CI green, cpluspluscheck fails: https://cirrus-ci.com/task/6561938334285824 I think this just needs a copy the logic you added to headerscheck also to cpluspluscheck (and yes, we should merge these files one of these days). Greetings, Andres Freund
Hi, On 2022-09-04 13:19:56 +0700, John Naylor wrote: > Crake fails with > > headerscheck: > Sep 04 01:56:01 In file included from /tmp/headerscheck.UNgmKy/test.c:2: > Sep 04 01:56:01 > /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: > fatal error: jsonpath_gram.h: No such file or directory > Sep 04 01:56:01 26 | #include "jsonpath_gram.h" > Sep 04 01:56:01 | ^~~~~~~~~~~~~~~~~ > Sep 04 01:56:01 compilation terminated. > Sep 04 01:56:14 make: *** [GNUmakefile:138: headerscheck] Error 1 > cpluspluscheck: > Sep 04 01:57:19 In file included from /tmp/cpluspluscheck.HhRV5k/test.cpp:3: > Sep 04 01:57:19 > /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: > fatal error: jsonpath_gram.h: No such file or directory > Sep 04 01:57:19 26 | #include "jsonpath_gram.h" > Sep 04 01:57:19 | ^~~~~~~~~~~~~~~~~ > > I don't get the same in an already-built tree. I also don't see an > exception for gramparse.h, which I imagine would have the same issue? It doesn't happen for gramparse.h because gram.h is copied to src/include/parser (see src/backend/Makefile), and included as parser/gram.h. I think this could be addressed by always adding something like -iquote $builddir/$(dirname "$f") -iquote $srcdir/$(dirname "$f") \ to cpluspluscheck/headerscheck. Greetings, Andres Freund
On Sun, Sep 4, 2022 at 8:34 PM Andrew Dunstan <andrew@dunslane.net> wrote: > Probably happens because crake does vpath builds. I think you're going > to have to set up EXTRAINCLUDES for it - see near the bottom of the > headerscheck script. Possibly something like > > > src/backend/utils/adt/*) > > EXTRAINCLUDES="-I $builddir/src/backend/utils/adt" ;; Okay, I'll look at this and the cpp header check within 24 hours. -- John Naylor EDB: http://www.enterprisedb.com
On Sun, Sep 4, 2022 at 8:34 PM Andrew Dunstan <andrew@dunslane.net> wrote: > > > On 2022-09-04 Su 02:19, John Naylor wrote: > > Crake fails with > > > > headerscheck: > > Sep 04 01:56:01 In file included from /tmp/headerscheck.UNgmKy/test.c:2: > > Sep 04 01:56:01 > > /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: > > fatal error: jsonpath_gram.h: No such file or directory > > Sep 04 01:56:01 26 | #include "jsonpath_gram.h" > > Sep 04 01:56:01 | ^~~~~~~~~~~~~~~~~ > > Sep 04 01:56:01 compilation terminated. > > Sep 04 01:56:14 make: *** [GNUmakefile:138: headerscheck] Error 1 > > cpluspluscheck: > > Sep 04 01:57:19 In file included from /tmp/cpluspluscheck.HhRV5k/test.cpp:3: > > Sep 04 01:57:19 > > /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: > > fatal error: jsonpath_gram.h: No such file or directory > > Sep 04 01:57:19 26 | #include "jsonpath_gram.h" > > Sep 04 01:57:19 | ^~~~~~~~~~~~~~~~~ > > > > I don't get the same in an already-built tree. I also don't see an > > exception for gramparse.h, which I imagine would have the same issue? > > > > Probably happens because crake does vpath builds. I think you're going > to have to set up EXTRAINCLUDES for it - see near the bottom of the > headerscheck script. Possibly something like > > > src/backend/utils/adt/*) > > EXTRAINCLUDES="-I $builddir/src/backend/utils/adt" ;; I just tried a vpath build and the failing check doesn't reproduce for me there. Any ideas? -- John Naylor EDB: http://www.enterprisedb.com
Hi, On September 5, 2022 8:33:37 PM PDT, John Naylor <john.naylor@enterprisedb.com> wrote: >On Sun, Sep 4, 2022 at 8:34 PM Andrew Dunstan <andrew@dunslane.net> wrote: >> >> >> On 2022-09-04 Su 02:19, John Naylor wrote: >> > Crake fails with >> > >> > headerscheck: >> > Sep 04 01:56:01 In file included from /tmp/headerscheck.UNgmKy/test.c:2: >> > Sep 04 01:56:01 >> > /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: >> > fatal error: jsonpath_gram.h: No such file or directory >> > Sep 04 01:56:01 26 | #include "jsonpath_gram.h" >> > Sep 04 01:56:01 | ^~~~~~~~~~~~~~~~~ >> > Sep 04 01:56:01 compilation terminated. >> > Sep 04 01:56:14 make: *** [GNUmakefile:138: headerscheck] Error 1 >> > cpluspluscheck: >> > Sep 04 01:57:19 In file included from /tmp/cpluspluscheck.HhRV5k/test.cpp:3: >> > Sep 04 01:57:19 >> > /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: >> > fatal error: jsonpath_gram.h: No such file or directory >> > Sep 04 01:57:19 26 | #include "jsonpath_gram.h" >> > Sep 04 01:57:19 | ^~~~~~~~~~~~~~~~~ >> > >> > I don't get the same in an already-built tree. I also don't see an >> > exception for gramparse.h, which I imagine would have the same issue? >> >> >> >> Probably happens because crake does vpath builds. I think you're going >> to have to set up EXTRAINCLUDES for it - see near the bottom of the >> headerscheck script. Possibly something like >> >> >> src/backend/utils/adt/*) >> >> EXTRAINCLUDES="-I $builddir/src/backend/utils/adt" ;; > >I just tried a vpath build and the failing check doesn't reproduce for >me there. Any ideas? Presumably it only happens if you previously triggered the parsers to be generated. Andres -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
On Tue, Sep 6, 2022 at 10:54 AM Andres Freund <andres@anarazel.de> wrote: > >I just tried a vpath build and the failing check doesn't reproduce for > >me there. Any ideas? > > Presumably it only happens if you previously triggered the parsers to be generated. If you attempt to run the check before building, the first thing you get is fatal error: utils/errcodes.h: No such file or directory so not sure what you mean. -- John Naylor EDB: http://www.enterprisedb.com
John Naylor <john.naylor@enterprisedb.com> writes: > On Tue, Sep 6, 2022 at 10:54 AM Andres Freund <andres@anarazel.de> wrote: >> Presumably it only happens if you previously triggered the parsers to be generated. > If you attempt to run the check before building, the first thing you get is > fatal error: utils/errcodes.h: No such file or directory > so not sure what you mean. crake is still unhappy: Sep 06 00:09:55 In file included from /tmp/headerscheck.AwLxya/test.c:2: Sep 06 00:09:55 /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: fatal error:jsonpath_gram.h: No such file or directory Sep 06 00:09:55 26 | #include "jsonpath_gram.h" Sep 06 00:09:55 | ^~~~~~~~~~~~~~~~~ I wonder if that file is not getting installed in a directory that this test expects to search. regards, tom lane
On Tue, Sep 6, 2022 at 11:49 AM Tom Lane <tgl@sss.pgh.pa.us> wrote: > > crake is still unhappy: > > Sep 06 00:09:55 In file included from /tmp/headerscheck.AwLxya/test.c:2: > Sep 06 00:09:55 /home/andrew/bf/root/HEAD/pgsql.build/../pgsql/src/backend/utils/adt/jsonpath_internal.h:26:10: fatal error:jsonpath_gram.h: No such file or directory > Sep 06 00:09:55 26 | #include "jsonpath_gram.h" > Sep 06 00:09:55 | ^~~~~~~~~~~~~~~~~ > > I wonder if that file is not getting installed in a directory that > this test expects to search. There are two proposed fixes upthread, which I will copy below, but if I can't reproduce locally, I don't have much confidence in which one is the way to go to get crake green again: Andrew: > Probably happens because crake does vpath builds. I think you're going > to have to set up EXTRAINCLUDES for it - see near the bottom of the > headerscheck script. Possibly something like > > > src/backend/utils/adt/*) > > EXTRAINCLUDES="-I $builddir/src/backend/utils/adt" ;; Andres: > It doesn't happen for gramparse.h because gram.h is copied to > src/include/parser (see src/backend/Makefile), and included as parser/gram.h. > > I think this could be addressed by always adding something like > -iquote $builddir/$(dirname "$f") -iquote $srcdir/$(dirname "$f") \ > to cpluspluscheck/headerscheck. -- John Naylor EDB: http://www.enterprisedb.com
I see why I couldn't reproduce: I neglected to add jsonpath_gram.h to the backend's maintainer-clean so it failed to be deleted. Working on the fix now. -- John Naylor EDB: http://www.enterprisedb.com