Thread: build farm machine using mixed results

Excerpts from Robert Creager's message of sáb sep 01 12:12:51 -0400 2012:
>
> I change the build-farm.conf file to have the following make line:
>
>     make => 'make -j 8', # or gmake if required. can include path if necessary.
>
> 2 pass, 4 fail.  Is this a build configuration you want to pursue?

Sure, why not?  Parallel building is going to become more common, so
it's a good idea to investigate the failures.  I would have it build
only HEAD though, because we're not likely to backpatch these fixes.

> I can either create a new machine, or change one of my existing machines.  Makes no difference to me.

If we want to have it run only HEAD I would say you should create a new
animal.

Perhaps you should wait until after 9.2 has been released, though, to
avoid distracting people :-)

--
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 9/1/12 12:12 PM, Robert Creager wrote:
> 
> I change the build-farm.conf file to have the following make line:
> 
>     make => 'make -j 8', # or gmake if required. can include path if
> necessary.
> 
> 2 pass, 4 fail.  Is this a build configuration you want to pursue?

Sure that would be useful, but it's pretty clear that the check stages
don't work in parallel.  It think it's because the ports conflict, but
there could be any number of other problems.

That said, it would be useful, in my mind, to support parallel checks.
But unless someone is going to put in the work first, you should
restrict your parallel runs to the build and install phases.

Excerpts from Peter Eisentraut's message of mar sep 04 18:49:46 -0300 2012:
> On 9/1/12 12:12 PM, Robert Creager wrote:
> >
> > I change the build-farm.conf file to have the following make line:
> >
> >     make => 'make -j 8', # or gmake if required. can include path if
> > necessary.
> >
> > 2 pass, 4 fail.  Is this a build configuration you want to pursue?
>
> Sure that would be useful, but it's pretty clear that the check stages
> don't work in parallel.  It think it's because the ports conflict, but
> there could be any number of other problems.

Is that really the problem?  As far as I know, buildfarm doesn't use
anything like installcheck-world or similar targets; each check target
is run separately, serially, by the BF script.

--
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 09/04/2012 05:49 PM, Peter Eisentraut wrote:
> On 9/1/12 12:12 PM, Robert Creager wrote:
>> I change the build-farm.conf file to have the following make line:
>>
>>      make => 'make -j 8', # or gmake if required. can include path if
>> necessary.
>>
>> 2 pass, 4 fail.  Is this a build configuration you want to pursue?
> Sure that would be useful, but it's pretty clear that the check stages
> don't work in parallel.  It think it's because the ports conflict, but
> there could be any number of other problems.
>
> That said, it would be useful, in my mind, to support parallel checks.
> But unless someone is going to put in the work first, you should
> restrict your parallel runs to the build and install phases.
>
>


The buildfarm code doesn't contain a facility to use a different make 
incantation for each step. It's pretty much an all or nothing deal. Of 
course, you can hack run_build.pl to make it do that, but I highly 
discourage that. For one thing, it makes upgrading that much more 
difficult. All the  tweaking is supposed to be done vie the config file. 
I guess I could add a setting that allowed for per step make flags.

Frankly, I have had enough failures of parallel make that I think doing 
this would generate a significant number of non-repeatable failures (I 
had one just the other day that took three invocations of make to get 
right). So I'm not sure doing this would advance us much, although I'm 
open to persuasion.

cheers

andrew

<p dir="ltr"><br /> On Sep 4, 2012 6:06 PM, "Andrew Dunstan" <<a
href="mailto:andrew@dunslane.net">andrew@dunslane.net</a>>wrote:<br /> ><br /> ><br /> > Frankly, I have
hadenough failures of parallel make that I think doing this would generate a significant number of non-repeatable
failures(I had one just the other day that took three invocations of make to get right). So I'm not sure doing this
wouldadvance us much, although I'm open to persuasion.<p dir="ltr">Seeing as most PostgreSQL bugs appear with
concurrency,I think we should leave our default config with 1 for max connections.<p dir="ltr"> ;-)<p
dir="ltr">Parallelmake failures are bugs in the dependencies as described in our make files.<p dir="ltr">For the build
phase,I don't recall parallel problems and as a habit I usually use parallel makes.  I would like that to be supported
andI think I've seen fixes applied when we had issues before.  Cutting build times to 1/2 to 1/4 is good.<p
dir="ltr">Checksand tests are harder because often they can't run in parallel. But then we shouldn't have them listed
asindependent prerequisites for targets.   Ideally.  But fixing it might not be worth the cost since an "acceptable"
workaround (rely upon make to serialize the test sequences in the particular order) is pretty painless (so far).  <p
dir="ltr">Ofcourse, having the ability to run the tests 8 at a time (or more) and reduce the time by 80% would be nice
.;-)<divclass="gmail_quote">On Sep 4, 2012 6:06 PM, "Andrew Dunstan" <<a
href="mailto:andrew@dunslane.net">andrew@dunslane.net</a>>wrote:<br type="attribution" /><blockquote
class="gmail_quote"style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br /> On 09/04/2012 05:49 PM,
PeterEisentraut wrote:<br /><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">On 9/1/12 12:12 PM, Robert Creager wrote:<br /><blockquote class="gmail_quote" style="margin:0
00 .8ex;border-left:1px #ccc solid;padding-left:1ex"> I change the build-farm.conf file to have the following make
line:<br/><br />      make => 'make -j 8', # or gmake if required. can include path if<br /> necessary.<br /><br />
2pass, 4 fail.  Is this a build configuration you want to pursue?<br /></blockquote> Sure that would be useful, but
it'spretty clear that the check stages<br /> don't work in parallel.  It think it's because the ports conflict, but<br
/>there could be any number of other problems.<br /><br /> That said, it would be useful, in my mind, to support
parallelchecks.<br /> But unless someone is going to put in the work first, you should<br /> restrict your parallel
runsto the build and install phases.<br /><br /><br /></blockquote><br /><br /> The buildfarm code doesn't contain a
facilityto use a different make incantation for each step. It's pretty much an all or nothing deal. Of course, you can
hack<a href="http://run_build.pl" target="_blank">run_build.pl</a> to make it do that, but I highly discourage that.
Forone thing, it makes upgrading that much more difficult. All the  tweaking is supposed to be done vie the config
file.I guess I could add a setting that allowed for per step make flags.<br /><br /> Frankly, I have had enough
failuresof parallel make that I think doing this would generate a significant number of non-repeatable failures (I had
onejust the other day that took three invocations of make to get right). So I'm not sure doing this would advance us
much,although I'm open to persuasion.<br /><br /> cheers<br /><br /> andrew<br /><br /><br /> -- <br /> Sent via
pgsql-hackersmailing list (<a href="mailto:pgsql-hackers@postgresql.org"
target="_blank">pgsql-hackers@postgresql.org</a>)<br/> To make changes to your subscription:<br /><a
href="http://www.postgresql.org/mailpref/pgsql-hackers"
target="_blank">http://www.postgresql.org/<u></u>mailpref/pgsql-hackers</a><br/><br /></blockquote></div> 

Andrew Dunstan <andrew@dunslane.net> writes:
> Frankly, I have had enough failures of parallel make that I think doing 
> this would generate a significant number of non-repeatable failures (I 
> had one just the other day that took three invocations of make to get 
> right). So I'm not sure doing this would advance us much, although I'm 
> open to persuasion.

Really?  I routinely use -j4 for building, and it's been a long time
since I've seen failures.  I can believe that for instance "make check"
in contrib would have a problem running in parallel, but the build
process per se seems reliable enough from here.
        regards, tom lane

On 09/04/2012 08:37 PM, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>> Frankly, I have had enough failures of parallel make that I think doing
>> this would generate a significant number of non-repeatable failures (I
>> had one just the other day that took three invocations of make to get
>> right). So I'm not sure doing this would advance us much, although I'm
>> open to persuasion.
> Really?  I routinely use -j4 for building, and it's been a long time
> since I've seen failures.  I can believe that for instance "make check"
> in contrib would have a problem running in parallel, but the build
> process per se seems reliable enough from here.
>
>             


Both cases were vpath builds, which is what I usually use, if that's a 
useful data point.

Maybe I run on lower level hardware than you do. I saw this again this 
afternoon after I posted the above. In both cases this was the machine 
that runs the buildfarm's crake. I'll try to get a handle on it.

cheers

andrew

On 09/04/2012 08:51 PM, Andrew Dunstan wrote:
>
> On 09/04/2012 08:37 PM, Tom Lane wrote:
>> Andrew Dunstan <andrew@dunslane.net> writes:
>>> Frankly, I have had enough failures of parallel make that I think doing
>>> this would generate a significant number of non-repeatable failures (I
>>> had one just the other day that took three invocations of make to get
>>> right). So I'm not sure doing this would advance us much, although I'm
>>> open to persuasion.
>> Really?  I routinely use -j4 for building, and it's been a long time
>> since I've seen failures.  I can believe that for instance "make check"
>> in contrib would have a problem running in parallel, but the build
>> process per se seems reliable enough from here.
>>
>>
>
>
> Both cases were vpath builds, which is what I usually use, if that's a 
> useful data point.
>
> Maybe I run on lower level hardware than you do. I saw this again this 
> afternoon after I posted the above. In both cases this was the machine 
> that runs the buildfarm's crake. I'll try to get a handle on it.
>
>


Well, it looks like it's always failing on ecpg, with preproc.h not 
being made in the right order. Here is the last bit of a make log 
starting from when it starts on ecpg. This is pretty repeatable.

cheers

andrew


-----

make -C ecpg all
make[3]: Entering directory 
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg'
make -C include all
make[4]: Entering directory 
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/include'
make[4]: Nothing to be done for `all'.
make[4]: Leaving directory 
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/include'
make -C pgtypeslib all
make -C preproc all
make[4]: Entering directory 
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/pgtypeslib'
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/include/utils 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=3  -c -o 
numeric.o 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/numeric.c -MMD 
-MP -MF .deps/numeric.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/include/utils 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=3  -c -o 
common.o 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/common.c -MMD 
-MP -MF .deps/common.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/include/utils 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=3  -c -o 
datetime.o 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/datetime.c -MMD 
-MP -MF .deps/datetime.Po
make[4]: Entering directory 
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/preproc'
make -C ../../../../src/port all
make[5]: Entering directory `/home/pgl/npgl/vpath.testpar/src/port'
make -C ../backend submake-errcodes
make[6]: Entering directory `/home/pgl/npgl/vpath.testpar/src/backend'
make[6]: Nothing to be done for `submake-errcodes'.
make[6]: Leaving directory `/home/pgl/npgl/vpath.testpar/src/backend'
make[5]: Leaving directory `/home/pgl/npgl/vpath.testpar/src/port'
'/usr/bin/perl' 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/parse.pl 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc < 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/../../../backend/parser/gram.y > preproc.y
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/include/utils 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=3  -c -o 
dt_common.o 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/dt_common.c -MMD 
-MP -MF .deps/dt_common.Po
'/usr/bin/perl' 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/check_rules.pl 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/../../../backend/parser/gram.y
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/include/utils 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=3  -c -o 
timestamp.o 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/timestamp.c -MMD 
-MP -MF .deps/timestamp.Po
/usr/bin/flex  -o'pgc.c' 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/pgc.l
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I. 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc -DMAJOR_VERSION=4 
-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include 
-I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE 
-I/usr/include/libxml2   -c -o type.o 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/type.c -MMD -MP -MF 
.deps/type.Po
rm -f pgstrcasecmp.c && ln -s 
/home/andrew/pgl/pg_head/src/port/pgstrcasecmp.c .
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/include/utils 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=3  -c -o 
interval.o 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/interval.c -MMD 
-MP -MF .deps/interval.Po
( echo '{ global:'; gawk '/^[^#]/ {printf "%s;\n",$1}' 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/exports.txt; 
echo ' local: *; };' ) >exports.list
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/include/utils 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=3  -c -o 
pgstrcasecmp.o pgstrcasecmp.c -MMD -MP -MF .deps/pgstrcasecmp.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I. 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc -DMAJOR_VERSION=4 
-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include 
-I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE 
-I/usr/include/libxml2   -c -o ecpg.o 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/ecpg.c -MMD -MP -MF 
.deps/ecpg.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I. 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc -DMAJOR_VERSION=4 
-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include 
-I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE 
-I/usr/include/libxml2   -c -o output.o 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/output.c -MMD -MP 
-MF .deps/output.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I. 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc -DMAJOR_VERSION=4 
-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include 
-I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE 
-I/usr/include/libxml2   -c -o parser.o 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/parser.c -MMD -MP 
-MF .deps/parser.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I. 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc -DMAJOR_VERSION=4 
-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include 
-I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE 
-I/usr/include/libxml2   -c -o keywords.o 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/keywords.c -MMD -MP 
-MF .deps/keywords.Po
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/parser.c:25:21: 
fatal error: preproc.h: No such file or directory
compilation terminated.
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/keywords.c:20:21: 
fatal error: preproc.h: No such file or directory
compilation terminated.
make[4]: *** [parser.o] Error 1
make[4]: *** Waiting for unfinished jobs....
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -shared -Wl,-soname,libpgtypes.so.3 
-Wl,--version-script=exports.list -o libpgtypes.so.3.4 numeric.o 
datetime.o common.o dt_common.o timestamp.o interval.o pgstrcasecmp.o  
-L../../../../src/port -Wl,--as-needed 
-Wl,-rpath,'/home/andrew/pgl/inst.testpar.5701/lib',--enable-new-dtags -lm
ar crs libpgtypes.a numeric.o datetime.o common.o dt_common.o 
timestamp.o interval.o pgstrcasecmp.o
ranlib libpgtypes.a
rm -f libpgtypes.so.3
ln -s libpgtypes.so.3.4 libpgtypes.so.3
rm -f libpgtypes.so
ln -s libpgtypes.so.3.4 libpgtypes.so
make[4]: Leaving directory 
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/pgtypeslib'
make -C ecpglib all
make[4]: *** [keywords.o] Error 1
make[4]: Leaving directory 
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/preproc'
make[3]: *** [all-preproc-recurse] Error 2
make[3]: *** Waiting for unfinished jobs....
make[4]: Entering directory 
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/ecpglib'
make -C ../../../../src/interfaces/libpq all
make -C ../../../../src/interfaces/ecpg/pgtypeslib all
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=6  -c -o 
execute.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/execute.c 
-MMD -MP -MF .deps/execute.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=6  -c -o 
typename.o 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/typename.c -MMD -MP 
-MF .deps/typename.Po
make[5]: Entering directory 
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/pgtypeslib'
make[5]: Nothing to be done for `all'.
make[5]: Leaving directory 
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/pgtypeslib'
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=6  -c -o 
descriptor.o 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/descriptor.c -MMD 
-MP -MF .deps/descriptor.Po
make[5]: Entering directory 
`/home/pgl/npgl/vpath.testpar/src/interfaces/libpq'
make[5]: Nothing to be done for `all'.
make[5]: Leaving directory 
`/home/pgl/npgl/vpath.testpar/src/interfaces/libpq'
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=6  -c -o 
sqlda.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/sqlda.c 
-MMD -MP -MF .deps/sqlda.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=6  -c -o data.o 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/data.c -MMD -MP -MF 
.deps/data.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=6  -c -o 
error.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/error.c 
-MMD -MP -MF .deps/error.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=6  -c -o 
prepare.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/prepare.c 
-MMD -MP -MF .deps/prepare.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=6  -c -o 
memory.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/memory.c 
-MMD -MP -MF .deps/memory.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=6  -c -o 
connect.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/connect.c 
-MMD -MP -MF .deps/connect.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=6  -c -o misc.o 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/misc.c -MMD -MP -MF 
.deps/misc.Po
rm -f path.c && ln -s /home/andrew/pgl/pg_head/src/port/path.c .
rm -f pgstrcasecmp.c && ln -s 
/home/andrew/pgl/pg_head/src/port/pgstrcasecmp.c .
rm -f strlcpy.c && ln -s /home/andrew/pgl/pg_head/src/port/strlcpy.c .
rm -f thread.c && ln -s /home/andrew/pgl/pg_head/src/port/thread.c .
( echo '{ global:'; gawk '/^[^#]/ {printf "%s;\n",$1}' 
/home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/exports.txt; echo ' 
local: *; };' ) >exports.list
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=6  -c -o path.o 
path.c -MMD -MP -MF .deps/path.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=6  -c -o 
pgstrcasecmp.o pgstrcasecmp.c -MMD -MP -MF .deps/pgstrcasecmp.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=6  -c -o 
strlcpy.o strlcpy.c -MMD -MP -MF .deps/strlcpy.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include 
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include 
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port 
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include 
-D_GNU_SOURCE -I/usr/include/libxml2  -DSO_MAJOR_VERSION=6  -c -o 
thread.o thread.c -MMD -MP -MF .deps/thread.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith 
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute 
-Wformat-security -fno-strict-aliasing -fwrapv 
-fexcess-precision=standard -g -pthread  -D_REENTRANT -D_THREAD_SAFE 
-D_POSIX_PTHREAD_SEMANTICS -fpic -shared -Wl,-soname,libecpg.so.6 
-Wl,--version-script=exports.list -o libecpg.so.6.5 execute.o typename.o 
descriptor.o sqlda.o data.o error.o prepare.o memory.o connect.o misc.o 
path.o pgstrcasecmp.o strlcpy.o thread.o -L../../../../src/port 
-Wl,--as-needed 
-Wl,-rpath,'/home/andrew/pgl/inst.testpar.5701/lib',--enable-new-dtags 
-L../pgtypeslib -lpgtypes -L../../../../src/interfaces/libpq -lpq -lm 
-lpthread
ar crs libecpg.a execute.o typename.o descriptor.o sqlda.o data.o 
error.o prepare.o memory.o connect.o misc.o path.o pgstrcasecmp.o 
strlcpy.o thread.o
ranlib libecpg.a
rm -f libecpg.so.6
ln -s libecpg.so.6.5 libecpg.so.6
rm -f libecpg.so
ln -s libecpg.so.6.5 libecpg.so
make[4]: Leaving directory 
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/ecpglib'
make[3]: Leaving directory 
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg'
make[2]: *** [all-ecpg-recurse] Error 2
make[2]: Leaving directory `/home/pgl/npgl/vpath.testpar/src/interfaces'
make[1]: *** [all-interfaces-recurse] Error 2
make[1]: Leaving directory `/home/pgl/npgl/vpath.testpar/src'
make: *** [all-src-recurse] Error 2

Andrew Dunstan <andrew@dunslane.net> writes:
> Well, it looks like it's always failing on ecpg, with preproc.h not 
> being made in the right order. Here is the last bit of a make log 
> starting from when it starts on ecpg. This is pretty repeatable.

Hmph.  I can't reproduce it at all on my Fedora 16 box.  What version
of make are you using?
        regards, tom lane

On 09/07/2012 08:43 PM, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>> Well, it looks like it's always failing on ecpg, with preproc.h not
>> being made in the right order. Here is the last bit of a make log
>> starting from when it starts on ecpg. This is pretty repeatable.
> Hmph.  I can't reproduce it at all on my Fedora 16 box.  What version
> of make are you using?


$ make -v
GNU Make 3.82
Built for x86_64-redhat-linux-gnu


cheers

andrew

On 09/07/2012 09:55 PM, Andrew Dunstan wrote:
>
> On 09/07/2012 08:43 PM, Tom Lane wrote:
>> Andrew Dunstan <andrew@dunslane.net> writes:
>>> Well, it looks like it's always failing on ecpg, with preproc.h not
>>> being made in the right order. Here is the last bit of a make log
>>> starting from when it starts on ecpg. This is pretty repeatable.
>> Hmph.  I can't reproduce it at all on my Fedora 16 box.  What version
>> of make are you using?
>
>
> $ make -v
> GNU Make 3.82
> Built for x86_64-redhat-linux-gnu
>
>
>


OK, I just tried on a different F16 machine and it didn't happen. I 
wonder what's different.

cheers

andrew

On 09/07/2012 10:46 PM, Andrew Dunstan wrote:
>
> On 09/07/2012 09:55 PM, Andrew Dunstan wrote:
>>
>> On 09/07/2012 08:43 PM, Tom Lane wrote:
>>> Andrew Dunstan <andrew@dunslane.net> writes:
>>>> Well, it looks like it's always failing on ecpg, with preproc.h not
>>>> being made in the right order. Here is the last bit of a make log
>>>> starting from when it starts on ecpg. This is pretty repeatable.
>>> Hmph.  I can't reproduce it at all on my Fedora 16 box.  What version
>>> of make are you using?
>>
>>
>> $ make -v
>> GNU Make 3.82
>> Built for x86_64-redhat-linux-gnu
>>
>>
>>
>
>
> OK, I just tried on a different F16 machine and it didn't happen. I 
> wonder what's different.

This seems totally stupid, but it happens when the path to the current 
directory includes a cross-device symlink. If I cd following the link, 
then this effect doesn't happen. Weird.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:
> This seems totally stupid, but it happens when the path to the current 
> directory includes a cross-device symlink. If I cd following the link, 
> then this effect doesn't happen. Weird.

Huh.  So maybe a gmake bug, or maybe there's something wrong with our
make rules for the case that `pwd` doesn't match what gmake thinks the
current path is.  Could you specify the setup more fully?
        regards, tom lane

On 09/08/2012 11:06 AM, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>> This seems totally stupid, but it happens when the path to the current
>> directory includes a cross-device symlink. If I cd following the link,
>> then this effect doesn't happen. Weird.
> Huh.  So maybe a gmake bug, or maybe there's something wrong with our
> make rules for the case that `pwd` doesn't match what gmake thinks the
> current path is.  Could you specify the setup more fully?
>
>             

Scratch that theory, that was just a transient. If anything it looks 
like it is related to system load. When almost nothing was running on 
the machine it worked fine. When I started up a Browser and an MUA the 
problem occurred. This VM has 4 CPUs and 4Gb of memory and a load 
average around 0.4 right now. I'm a bit perplexed.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:
> Scratch that theory, that was just a transient. If anything it looks 
> like it is related to system load. When almost nothing was running on 
> the machine it worked fine. When I started up a Browser and an MUA the 
> problem occurred. This VM has 4 CPUs and 4Gb of memory and a load 
> average around 0.4 right now. I'm a bit perplexed.

Hm ... you weren't using the -l (--max-load) option were you?  That
would make system load affect gmake's scheduling.  Although it's clear
when I test it that it is waiting for the bison run to finish before
launching the dependent builds, so it still seems like it must be a bug
if your copy isn't waiting for that.
        regards, tom lane

On 09/08/2012 04:46 PM, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>> Scratch that theory, that was just a transient. If anything it looks
>> like it is related to system load. When almost nothing was running on
>> the machine it worked fine. When I started up a Browser and an MUA the
>> problem occurred. This VM has 4 CPUs and 4Gb of memory and a load
>> average around 0.4 right now. I'm a bit perplexed.
> Hm ... you weren't using the -l (--max-load) option were you?  That
> would make system load affect gmake's scheduling.  Although it's clear
> when I test it that it is waiting for the bison run to finish before
> launching the dependent builds, so it still seems like it must be a bug
> if your copy isn't waiting for that.
>
>             

No. just "make -j 4"

And it's the stock Fedora build of make.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:
> And it's the stock Fedora build of make.

Which Fedora branch exactly?

The package version I was trying to reproduce with here is
make-3.82-8.fc16.x86_64.
        regards, tom lane

On 09/08/2012 04:54 PM, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>> And it's the stock Fedora build of make.
> Which Fedora branch exactly?
>
> The package version I was trying to reproduce with here is
> make-3.82-8.fc16.x86_64.

Same:
   $ rpm -q make   make-3.82-8.fc16.x86_64

cheers

andrew

On 09/08/2012 04:52 PM, Andrew Dunstan wrote:
>
> On 09/08/2012 04:46 PM, Tom Lane wrote:
>> Andrew Dunstan <andrew@dunslane.net> writes:
>>> Scratch that theory, that was just a transient. If anything it looks
>>> like it is related to system load. When almost nothing was running on
>>> the machine it worked fine. When I started up a Browser and an MUA the
>>> problem occurred. This VM has 4 CPUs and 4Gb of memory and a load
>>> average around 0.4 right now. I'm a bit perplexed.
>> Hm ... you weren't using the -l (--max-load) option were you? That
>> would make system load affect gmake's scheduling.  Although it's clear
>> when I test it that it is waiting for the bison run to finish before
>> launching the dependent builds, so it still seems like it must be a bug
>> if your copy isn't waiting for that.
>>
>>
>
> No. just "make -j 4"
>
> And it's the stock Fedora build of make.


I have just repeated this on an absolutely fresh up to date F17 machine, 
with no symlink stuff in play.

Steps to recreate:
   CC="ccache gcc" ../postgres/configure --enable-depend --enable-debug   --enable-cassert --with-perl --with-python
--with-tcl--with-libxml   --with-libxslt -with-openssl --with-gssapi --with-pam --with-ldap   make -j 4
 


cheers

andrew

q

Andrew Dunstan <andrew@dunslane.net> writes:
> I have just repeated this on an absolutely fresh up to date F17 machine, 
> with no symlink stuff in play.

> Steps to recreate:

>     CC="ccache gcc" ../postgres/configure --enable-depend --enable-debug
>     --enable-cassert --with-perl --with-python --with-tcl --with-libxml
>     --with-libxslt -with-openssl --with-gssapi --with-pam --with-ldap
>     make -j 4

Huh ... that recipe works (er, fails) for me too, at least some of the
time.  I wonder what exactly is the key difference between the working
and failing cases?

Anyway, what I notice is that I get different types of failures, but
they are all under ecpg/.  What I think we need to do is insert
.NOTPARALLEL in ecpg/Makefile, because there are several reasons not
to run its sub-makes in parallel:

* preproc/Makefile casually does this:

../ecpglib/typename.o: ../ecpglib/typename.c$(MAKE) -C $(dir $@) $(notdir $@)

which is very likely to screw up any make proceeding in parallel in
ecpglib.

* compatlib and ecpglib will equally casually invoke "make all" in
other subdirectories; ditto.

And that's not even counting the bison-output problem you were seeing.
I'm not entirely sure what's causing that, but I'm suspicious that the
ultimate cause is the extra rules for the "all...recurse" targets in
ecpg/Makefile, which look like they could result in additional instances
of multiple make processes running in the same subdirectory.

After adding the .NOTPARALLEL marker, I don't see these failures
anymore.
        regards, tom lane

On 09/08/2012 07:54 PM, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>> I have just repeated this on an absolutely fresh up to date F17 machine,
>> with no symlink stuff in play.
>> Steps to recreate:
>>      CC="ccache gcc" ../postgres/configure --enable-depend --enable-debug
>>      --enable-cassert --with-perl --with-python --with-tcl --with-libxml
>>      --with-libxslt -with-openssl --with-gssapi --with-pam --with-ldap
>>      make -j 4
> Huh ... that recipe works (er, fails) for me too, at least some of the
> time.  I wonder what exactly is the key difference between the working
> and failing cases?
>
> Anyway, what I notice is that I get different types of failures, but
> they are all under ecpg/.  What I think we need to do is insert
> .NOTPARALLEL in ecpg/Makefile, because there are several reasons not
> to run its sub-makes in parallel:
>
> * preproc/Makefile casually does this:
>
> ../ecpglib/typename.o: ../ecpglib/typename.c
>     $(MAKE) -C $(dir $@) $(notdir $@)
>
> which is very likely to screw up any make proceeding in parallel in
> ecpglib.
>
> * compatlib and ecpglib will equally casually invoke "make all" in
> other subdirectories; ditto.
>
> And that's not even counting the bison-output problem you were seeing.
> I'm not entirely sure what's causing that, but I'm suspicious that the
> ultimate cause is the extra rules for the "all...recurse" targets in
> ecpg/Makefile, which look like they could result in additional instances
> of multiple make processes running in the same subdirectory.
>
> After adding the .NOTPARALLEL marker, I don't see these failures
> anymore.
>
>             

Well, I'm glad it's not just me. :-)

This fix works for me too.

I guess it should be applied back to 9.1, when it looks like we started 
using .NOTPARALLEL


cheers

andrew

On Sat, 2012-09-08 at 19:54 -0400, Tom Lane wrote:
> Anyway, what I notice is that I get different types of failures, but
> they are all under ecpg/.  What I think we need to do is insert
> .NOTPARALLEL in ecpg/Makefile,

I'd hate that, because the ecpg build is one of the slowest parts of the
build, so de-parallelizing it would slow down everything quite a bit.

>  because there are several reasons not
> to run its sub-makes in parallel:
> 
> * preproc/Makefile casually does this:
> 
> ../ecpglib/typename.o: ../ecpglib/typename.c
>     $(MAKE) -C $(dir $@) $(notdir $@)
> 
> which is very likely to screw up any make proceeding in parallel in
> ecpglib.

That should probably be fixed by symlinking the source file and building
it in the preproc directory.

> And that's not even counting the bison-output problem you were seeing.
> I'm not entirely sure what's causing that, but I'm suspicious that the
> ultimate cause is the extra rules for the "all...recurse" targets in
> ecpg/Makefile, which look like they could result in additional instances
> of multiple make processes running in the same subdirectory.

I think the point of these targets is exactly to prevent that.

Peter Eisentraut <peter_e@gmx.net> writes:
> On Sat, 2012-09-08 at 19:54 -0400, Tom Lane wrote:
>> Anyway, what I notice is that I get different types of failures, but
>> they are all under ecpg/.  What I think we need to do is insert
>> .NOTPARALLEL in ecpg/Makefile,

> I'd hate that, because the ecpg build is one of the slowest parts of the
> build, so de-parallelizing it would slow down everything quite a bit.

There's only one bit of it that's slow, which is the bison build +
preproc.c compile, which is necessarily serial anyway.  So I think
trying to avoid .NOTPARALLEL there is a complete waste of effort.
But if you wanna fix it some other way, step right up.

(This all does remind me of the "Recursive Make Considered Harmful"
thread from awhile back.  Is anyone prepared to go down that path
now?  I'm not terribly excited by that, as yet --- yeah, it would
be better, but the work involved seems out of proportion to the
benefit.)
        regards, tom lane

On 09/09/2012 03:29 AM, Tom Lane wrote:
> Peter Eisentraut <peter_e@gmx.net> writes:
>> On Sat, 2012-09-08 at 19:54 -0400, Tom Lane wrote:
>>> Anyway, what I notice is that I get different types of failures, but
>>> they are all under ecpg/.  What I think we need to do is insert
>>> .NOTPARALLEL in ecpg/Makefile,
>> I'd hate that, because the ecpg build is one of the slowest parts of the
>> build, so de-parallelizing it would slow down everything quite a bit.
> There's only one bit of it that's slow, which is the bison build +
> preproc.c compile, which is necessarily serial anyway.  So I think
> trying to avoid .NOTPARALLEL there is a complete waste of effort.
> But if you wanna fix it some other way, step right up.


Yeah. I am going to add a config parameter to the buildfarm to allow 
parallelism for the "make" and "make contrib" stages, but I'm not going 
to release it until this is fixed.

(I suppose this is also a lesson to me that I should not ignore things 
like this that annoy me persistently as I did with these failures until 
Robert Creager started this discussion. It just didn't seem important 
enough.)

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:
> On 09/09/2012 03:29 AM, Tom Lane wrote:
>> Peter Eisentraut <peter_e@gmx.net> writes:
>>> On Sat, 2012-09-08 at 19:54 -0400, Tom Lane wrote:
>>>> Anyway, what I notice is that I get different types of failures, but
>>>> they are all under ecpg/.  What I think we need to do is insert
>>>> .NOTPARALLEL in ecpg/Makefile,

>>> I'd hate that, because the ecpg build is one of the slowest parts of the
>>> build, so de-parallelizing it would slow down everything quite a bit.

>> There's only one bit of it that's slow, which is the bison build +
>> preproc.c compile, which is necessarily serial anyway.  So I think
>> trying to avoid .NOTPARALLEL there is a complete waste of effort.
>> But if you wanna fix it some other way, step right up.

> Yeah. I am going to add a config parameter to the buildfarm to allow 
> parallelism for the "make" and "make contrib" stages, but I'm not going 
> to release it until this is fixed.

Well, why don't we stick .NOTPARALLEL in there for the moment, and then
if Peter thinks of a better solution, he can revert that change in favor
of something cleaner.

I assume we need this for all active branches, if the buildfarm is
going to be stressing it?
        regards, tom lane

On 09/09/2012 11:31 AM, Tom Lane wrote:
>> Yeah. I am going to add a config parameter to the buildfarm to allow
>> parallelism for the "make" and "make contrib" stages, but I'm not going
>> to release it until this is fixed.
> Well, why don't we stick .NOTPARALLEL in there for the moment, and then
> if Peter thinks of a better solution, he can revert that change in favor
> of something cleaner.


Works for me

>
> I assume we need this for all active branches, if the buildfarm is
> going to be stressing it?
>
>             

I can restrict it to only modern branches. Didn't we supposedly improve 
support for this during the 9.1 cycle? That seems like a sensible place 
to go back to.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:
> On 09/09/2012 11:31 AM, Tom Lane wrote:
>> I assume we need this for all active branches, if the buildfarm is
>> going to be stressing it?

> I can restrict it to only modern branches. Didn't we supposedly improve 
> support for this during the 9.1 cycle? That seems like a sensible place 
> to go back to.

Mmm, yeah, the commit history shows several different fixes for parallel
make back in the 9.1 time frame.  I doubt we want to try back-porting
all that work.  Let's just say 9.1 and up, then.
        regards, tom lane

And the answer is ... it's a gmake bug.  Apparently introduced in 3.82.

http://savannah.gnu.org/bugs/?30653
https://bugzilla.redhat.com/show_bug.cgi?id=835424

So I think .NOTPARALLEL is just masking the true problem, but
nonetheless it's a problem.  And given that the bug report on savannah
has been ignored for two years, we should not hold our breath for a fix
to appear upstream (much less propagate to everyone using 3.82).
        regards, tom lane

On 09/09/2012 02:05 PM, Tom Lane wrote:
> And the answer is ... it's a gmake bug.  Apparently introduced in 3.82.
>
> http://savannah.gnu.org/bugs/?30653
> https://bugzilla.redhat.com/show_bug.cgi?id=835424
>
> So I think .NOTPARALLEL is just masking the true problem, but
> nonetheless it's a problem.  And given that the bug report on savannah
> has been ignored for two years, we should not hold our breath for a fix
> to appear upstream (much less propagate to everyone using 3.82).
>
>             

Thanks for pursuing this. Whether or not it masks the underlying 
problem, it's still something we should do, no? In fact, it seems to me 
like this makes it even less worth trying to do anything better.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:
> On 09/09/2012 02:05 PM, Tom Lane wrote:
>> And the answer is ... it's a gmake bug.

> Thanks for pursuing this. Whether or not it masks the underlying 
> problem, it's still something we should do, no? In fact, it seems to me 
> like this makes it even less worth trying to do anything better.

Yeah, exactly.
        regards, tom lane

On Sun, 2012-09-09 at 14:05 -0400, Tom Lane wrote:
> And the answer is ... it's a gmake bug.  Apparently introduced in 3.82.
> 
> http://savannah.gnu.org/bugs/?30653
> https://bugzilla.redhat.com/show_bug.cgi?id=835424
> 
> So I think .NOTPARALLEL is just masking the true problem, but
> nonetheless it's a problem.  And given that the bug report on savannah
> has been ignored for two years, we should not hold our breath for a fix
> to appear upstream (much less propagate to everyone using 3.82).

So does the issue go away when using GNU make 3.81?

On Sun, 2012-09-09 at 14:57 -0400, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
> > On 09/09/2012 02:05 PM, Tom Lane wrote:
> >> And the answer is ... it's a gmake bug.
> 
> > Thanks for pursuing this. Whether or not it masks the underlying 
> > problem, it's still something we should do, no? In fact, it seems to me 
> > like this makes it even less worth trying to do anything better.
> 
> Yeah, exactly.

But then the answer could be, if you want to use parallel make, use a
version that's not broken.

Peter Eisentraut <peter_e@gmx.net> writes:
> On Sun, 2012-09-09 at 14:05 -0400, Tom Lane wrote:
>> So I think .NOTPARALLEL is just masking the true problem, but
>> nonetheless it's a problem.  And given that the bug report on savannah
>> has been ignored for two years, we should not hold our breath for a fix
>> to appear upstream (much less propagate to everyone using 3.82).

> So does the issue go away when using GNU make 3.81?

I couldn't reproduce it in a few tries on my Mac laptop, which has 3.81.
But that's not necessarily a comparable test, given the different
hardware, OS, and compiler version.  I'd put more faith in the multiple
statements in the upstream bugs that people weren't seeing this with
3.81.
        regards, tom lane

Peter Eisentraut <peter_e@gmx.net> writes:
> But then the answer could be, if you want to use parallel make, use a
> version that's not broken.

That's not a terribly practical answer for people who use the "make"
supplied by their OS vendor, which is approximately 99.9% of people.
It's even less practical for packagers, who don't have a choice about
what tool set to use.

Even if I wanted to use a locally-patched make, I'm not sure I'd trust a
patch that doesn't seem to have been signed off on by any actual gmake
developer or maintainer.  That sort of cure is frequently worse than the
disease.
        regards, tom lane

On 09/09/2012 05:00 PM, Tom Lane wrote:
> Peter Eisentraut <peter_e@gmx.net> writes:
>> But then the answer could be, if you want to use parallel make, use a
>> version that's not broken.
> That's not a terribly practical answer for people who use the "make"
> supplied by their OS vendor, which is approximately 99.9% of people.
> It's even less practical for packagers, who don't have a choice about
> what tool set to use.
>
> Even if I wanted to use a locally-patched make, I'm not sure I'd trust a
> patch that doesn't seem to have been signed off on by any actual gmake
> developer or maintainer.  That sort of cure is frequently worse than the
> disease.
>
>             


Right.

Meanwhile, I have buildfarm animal crake building with the experimental 
buildfarm code, and so far the results look good.

cheers

andrew

I wrote:
> And the answer is ... it's a gmake bug.  Apparently introduced in 3.82.
> http://savannah.gnu.org/bugs/?30653
> https://bugzilla.redhat.com/show_bug.cgi?id=835424

> So I think .NOTPARALLEL is just masking the true problem, but
> nonetheless it's a problem.  And given that the bug report on savannah
> has been ignored for two years, we should not hold our breath for a fix
> to appear upstream (much less propagate to everyone using 3.82).

So no sooner do I complain about that, than the upstream maintainers
wake up and commit it:
http://lists.gnu.org/archive/html/bug-make/2012-09/msg00016.html

No idea when a fixed release might appear, but at least somebody who
knows the gmake code has signed off on the fix now.
        regards, tom lane

On 09/10/2012 02:44 PM, Tom Lane wrote:
> I wrote:
>> And the answer is ... it's a gmake bug.  Apparently introduced in 3.82.
>> http://savannah.gnu.org/bugs/?30653
>> https://bugzilla.redhat.com/show_bug.cgi?id=835424
>> So I think .NOTPARALLEL is just masking the true problem, but
>> nonetheless it's a problem.  And given that the bug report on savannah
>> has been ignored for two years, we should not hold our breath for a fix
>> to appear upstream (much less propagate to everyone using 3.82).
> So no sooner do I complain about that, than the upstream maintainers
> wake up and commit it:
> http://lists.gnu.org/archive/html/bug-make/2012-09/msg00016.html
>
> No idea when a fixed release might appear, but at least somebody who
> knows the gmake code has signed off on the fix now.
>
>             

When it does appear in a release I guess we could make the .NOTPARALLEL 
conditional on make version. If not, we'll have to wait a long time 
before removing it.

cheers

andrew