Thread: pgsql: Refactor flex and bison make rules
Refactor flex and bison make rules Numerous flex and bison make rules have appeared in the source tree over time, and they are all virtually identical, so we can replace them by pattern rules with some variables for customization. Users of pgxs will also be able to benefit from this. Branch ------ master Details ------- http://git.postgresql.org/pg/commitdiff/8521d131941be5a177270bc428fa8e684cd645b5 Modified Files -------------- contrib/cube/Makefile | 14 -------------- contrib/seg/Makefile | 14 -------------- src/Makefile.global.in | 16 ++++++++++++++++ src/backend/bootstrap/Makefile | 16 ---------------- src/backend/parser/Makefile | 20 ++++---------------- src/backend/replication/Makefile | 14 -------------- src/backend/utils/misc/Makefile | 7 ------- src/bin/psql/Makefile | 11 ++--------- src/interfaces/ecpg/preproc/Makefile | 15 +-------------- src/pl/plpgsql/src/Makefile | 9 +-------- src/test/isolation/Makefile | 14 -------------- 11 files changed, 24 insertions(+), 126 deletions(-)
Peter Eisentraut wrote: > Refactor flex and bison make rules > > Numerous flex and bison make rules have appeared in the source tree > over time, and they are all virtually identical, so we can replace > them by pattern rules with some variables for customization. Okapi has been failing sporadically on ecpg, and I wonder if it's related to this change. See, for example, http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=okapi&dt=2012-11-28%2018%3A15%3A01 icc -O3 -xSSSE3 -parallel -ip -mp1 -fno-strict-aliasing -g -pthread -D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS-DECPG_COMPILE -I../include -I../../../../src/interfaces/ecpg/include -I. -I. -DMAJOR_VERSION=4-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include/et -c -o parser.o parser.c icc -O3 -xSSSE3 -parallel -ip -mp1 -fno-strict-aliasing -g -pthread -D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS-DECPG_COMPILE -I../include -I../../../../src/interfaces/ecpg/include -I. -I. -DMAJOR_VERSION=4-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include/et -c -o keywords.o keywords.c parser.c(25): catastrophic error: could not open source file "preproc.h" #include "preproc.h" ^ compilation aborted for parser.c (code 4) make[4]: *** [parser.o] Error 4 make[4]: *** Waiting for unfinished jobs.... keywords.c(20): catastrophic error: could not open source file "preproc.h" #include "preproc.h" ^ compilation aborted for keywords.c (code 4) -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On 11/28/2012 02:14 PM, Alvaro Herrera wrote: > Peter Eisentraut wrote: >> Refactor flex and bison make rules >> >> Numerous flex and bison make rules have appeared in the source tree >> over time, and they are all virtually identical, so we can replace >> them by pattern rules with some variables for customization. > Okapi has been failing sporadically on ecpg, and I wonder if it's > related to this change. See, for example, > http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=okapi&dt=2012-11-28%2018%3A15%3A01 > > icc -O3 -xSSSE3 -parallel -ip -mp1 -fno-strict-aliasing -g -pthread -D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS-DECPG_COMPILE -I../include -I../../../../src/interfaces/ecpg/include -I. -I. -DMAJOR_VERSION=4-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include/et -c -o parser.o parser.c > icc -O3 -xSSSE3 -parallel -ip -mp1 -fno-strict-aliasing -g -pthread -D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS-DECPG_COMPILE -I../include -I../../../../src/interfaces/ecpg/include -I. -I. -DMAJOR_VERSION=4-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include/et -c -o keywords.o keywords.c > parser.c(25): catastrophic error: could not open source file "preproc.h" > #include "preproc.h" > ^ > > compilation aborted for parser.c (code 4) > make[4]: *** [parser.o] Error 4 > make[4]: *** Waiting for unfinished jobs.... > keywords.c(20): catastrophic error: could not open source file "preproc.h" > #include "preproc.h" > ^ > > compilation aborted for keywords.c (code 4) Well, it looks like the make is broken and missing a clear dependency requirement. I think we need to ask Jeremy to turn off parallel build for okapi. cheers andrew
Andrew Dunstan <andrew@dunslane.net> writes: > On 11/28/2012 02:14 PM, Alvaro Herrera wrote: >> Okapi has been failing sporadically on ecpg, and I wonder if it's >> related to this change. > Well, it looks like the make is broken and missing a clear dependency > requirement. I think we need to ask Jeremy to turn off parallel build > for okapi. Yeah, we already know that unpatched make 3.82 has got serious parallelism bugs: http://archives.postgresql.org/pgsql-hackers/2012-09/msg00397.php I wonder whether adding another .NOTPARALLEL directive would be a better idea than insisting people get hold of patched versions. regards, tom lane
On 11/28/12 6:01 PM, Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: >> On 11/28/2012 02:14 PM, Alvaro Herrera wrote: >>> Okapi has been failing sporadically on ecpg, and I wonder if it's >>> related to this change. > >> Well, it looks like the make is broken and missing a clear dependency >> requirement. I think we need to ask Jeremy to turn off parallel build >> for okapi. > > Yeah, we already know that unpatched make 3.82 has got serious > parallelism bugs: > http://archives.postgresql.org/pgsql-hackers/2012-09/msg00397.php > > I wonder whether adding another .NOTPARALLEL directive would be a better > idea than insisting people get hold of patched versions. We could put ifeq ($(MAKE_VERSION),3.82) .NOTPARALLEL: endif into Makefile.global.
On 11/28/2012 06:01 PM, Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: >> On 11/28/2012 02:14 PM, Alvaro Herrera wrote: >>> Okapi has been failing sporadically on ecpg, and I wonder if it's >>> related to this change. >> Well, it looks like the make is broken and missing a clear dependency >> requirement. I think we need to ask Jeremy to turn off parallel build >> for okapi. > Yeah, we already know that unpatched make 3.82 has got serious > parallelism bugs: > http://archives.postgresql.org/pgsql-hackers/2012-09/msg00397.php > > I wonder whether adding another .NOTPARALLEL directive would be a better > idea than insisting people get hold of patched versions. > > You mean in the preproc Makefile? Maybe. cheers andrew
Peter Eisentraut <peter_e@gmx.net> writes:
> On 11/28/12 6:01 PM, Tom Lane wrote:
>> I wonder whether adding another .NOTPARALLEL directive would be a better
>> idea than insisting people get hold of patched versions.
> We could put
> ifeq ($(MAKE_VERSION),3.82)
> .NOTPARALLEL:
> endif
> into Makefile.global.
I don't wish to go *that* far. Parallel make works fine for most of the
tree in 3.82, and shutting it off would penalize developers a lot.
It appears to me that the case that okapi is hitting is specific to the
ecpg preprocessor build rules, and indeed specific to the case where
preproc.c needs to be rebuilt. A .NOTPARALLEL in ecpg/preproc/Makefile
would probably be enough to fix it. (I'm a bit tempted to make the one
already added to ecpg/Makefile conditional on the make version, as you
suggest above, too.)
regards, tom lane
On 11/28/2012 06:19 PM, Tom Lane wrote: > Peter Eisentraut <peter_e@gmx.net> writes: >> On 11/28/12 6:01 PM, Tom Lane wrote: >>> I wonder whether adding another .NOTPARALLEL directive would be a better >>> idea than insisting people get hold of patched versions. >> We could put >> ifeq ($(MAKE_VERSION),3.82) >> .NOTPARALLEL: >> endif >> into Makefile.global. > I don't wish to go *that* far. Parallel make works fine for most of the > tree in 3.82, and shutting it off would penalize developers a lot. > > It appears to me that the case that okapi is hitting is specific to the > ecpg preprocessor build rules, and indeed specific to the case where > preproc.c needs to be rebuilt. A .NOTPARALLEL in ecpg/preproc/Makefile > would probably be enough to fix it. (I'm a bit tempted to make the one > already added to ecpg/Makefile conditional on the make version, as you > suggest above, too.) > > There is something odd about okapi, because my linux/gcc buildfarm animal is using make 3.82 happily, with make_jobs = 4. cheers andrew
Andrew Dunstan <andrew@dunslane.net> writes:
> On 11/28/2012 06:19 PM, Tom Lane wrote:
>> It appears to me that the case that okapi is hitting is specific to the
>> ecpg preprocessor build rules, and indeed specific to the case where
>> preproc.c needs to be rebuilt. A .NOTPARALLEL in ecpg/preproc/Makefile
>> would probably be enough to fix it. (I'm a bit tempted to make the one
>> already added to ecpg/Makefile conditional on the make version, as you
>> suggest above, too.)
> There is something odd about okapi, because my linux/gcc buildfarm
> animal is using make 3.82 happily, with make_jobs = 4.
Yeah, and nobody else has seen this either. It might just be that okapi
has exactly the right number of processors with exactly the right speeds
to make the failure a lot more probable. Or maybe there's something
weird about Gentoo's version of make (wouldn't be the first time).
Anyway, deparallelizing just the ecpg/preproc build would cost very
little in build time, since it's totally dominated by the preproc.c and
preproc.o build steps anyway. I'm inclined to just do it and see if
the problem goes away.
regards, tom lane