Thread: Should we use make -k on the buildfarm?

Should we use make -k on the buildfarm?

From
Andrew Dunstan
Date:
Peter Eisentraut has suggested that we should run "make -k" instead of 
plain "make" for most or all of the buildfarm steps. This flag 
essentially instructs make to keep going rather than fail at the first 
error. We haven't done that for the last five or six years that the 
buildfarm has been running, and nobody up to now has complained (that I 
recall). I don't have any great objection, but before I make this change 
I thought it might be as well to canvas a wider range of opinion.

So, does anyone else have thoughts about it?

cheers

andrew


Re: Should we use make -k on the buildfarm?

From
Tom Lane
Date:
Andrew Dunstan <andrew@dunslane.net> writes:
> Peter Eisentraut has suggested that we should run "make -k" instead of 
> plain "make" for most or all of the buildfarm steps. This flag 
> essentially instructs make to keep going rather than fail at the first 
> error. We haven't done that for the last five or six years that the 
> buildfarm has been running, and nobody up to now has complained (that I 
> recall). I don't have any great objection, but before I make this change 
> I thought it might be as well to canvas a wider range of opinion.

> So, does anyone else have thoughts about it?

I don't really care about make -k as such.  What I *have* occasionally
wished for is that the buildfarm script would act more like make -k with
respect to the various test stages.  That is, not abandon the whole test
after one stage fails, but allow stages that don't logically depend on
the failed one to proceed.  But I'm not sure how hard that would be ---
quite aside from coding complexity, it would mean that you could have
more than one failing stage, and I don't know how you'd show that in the
dashboard.

Anyway, no objection to Peter's request; but there's another TODO item
for your buildfarm list, if you want to accept it.
        regards, tom lane


Re: Should we use make -k on the buildfarm?

From
Andrew Dunstan
Date:
<br /><br /> On 11/06/2010 01:07 PM, Tom Lane wrote: <blockquote cite="mid:21127.1289063239@sss.pgh.pa.us"
type="cite"><prewrap="">What I *have* occasionally
 
wished for is that the buildfarm script would act more like make -k with
respect to the various test stages.  That is, not abandon the whole test
after one stage fails, but allow stages that don't logically depend on
the failed one to proceed.  But I'm not sure how hard that would be ---
quite aside from coding complexity, it would mean that you could have
more than one failing stage, and I don't know how you'd show that in the
dashboard. 
</pre></blockquote><br /> That would be a significant architectural change. I'm not sure how many steps we could run
thisway. <br /><br /> Here's the list of tests from a recent run, leaving out stopping and starting the installed
postmaster,and locale specifiers:<br /><br /><blockquote>SCM-checkout      <br /> configure      <br /> make      <br
/>check      <br /> make-contrib<br /> make-install     <br /> install-contrib     <br /> initdb     <br />
install-check    <br /> pl-install-check     <br /> contrib-install-check     <br /> ecpg-check<br /></blockquote>
Currently,the implied dependency list is in this order. We could have "make-contrib" depend only on "make" rather than
"check","pl-install-check"  and "contrib-install-check" depend on "initdb", and "ecpg-check" depend on "make" rather
thananything that comes after. I think that's about the limit of what we could sensibly relax<br /><br /> I'm not sure
thatwould be a great advance. Certainly, right now I'm going to be putting effort into the FTS stuff which I think
shouldbe much higher up your list of wants.<br /><br /> cheers<br /><br /> andrew<br /> 

Re: Should we use make -k on the buildfarm?

From
Tom Lane
Date:
Andrew Dunstan <andrew@dunslane.net> writes:
> On 11/06/2010 01:07 PM, Tom Lane wrote:
>> What I *have* occasionally
>> wished for is that the buildfarm script would act more like make -k with
>> respect to the various test stages.

> I'm not sure that would be a great advance. Certainly, right now I'm 
> going to be putting effort into the FTS stuff which I think should be 
> much higher up your list of wants.

Agreed, that would be far more useful.
        regards, tom lane


Re: Should we use make -k on the buildfarm?

From
Stefan Kaltenbrunner
Date:
Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>> On 11/06/2010 01:07 PM, Tom Lane wrote:
>>> What I *have* occasionally
>>> wished for is that the buildfarm script would act more like make -k with
>>> respect to the various test stages.
> 
>> I'm not sure that would be a great advance. Certainly, right now I'm 
>> going to be putting effort into the FTS stuff which I think should be 
>> much higher up your list of wants.
> 
> Agreed, that would be far more useful.

FTS stuff? Would that also include additions to the current SOAP 
interface of the buildfarm?

lg

Stefan


Re: Should we use make -k on the buildfarm?

From
Andrew Dunstan
Date:

On 11/08/2010 10:27 AM, Stefan Kaltenbrunner wrote:
> Tom Lane wrote:
>> Andrew Dunstan <andrew@dunslane.net> writes:
>>> On 11/06/2010 01:07 PM, Tom Lane wrote:
>>>> What I *have* occasionally
>>>> wished for is that the buildfarm script would act more like make -k 
>>>> with
>>>> respect to the various test stages.
>>
>>> I'm not sure that would be a great advance. Certainly, right now I'm 
>>> going to be putting effort into the FTS stuff which I think should 
>>> be much higher up your list of wants.
>>
>> Agreed, that would be far more useful.
>
> FTS stuff? Would that also include additions to the current SOAP 
> interface of the buildfarm?
>
>

Possibly. I haven't looked at how we'll use it yet.

cheers

andrew


Re: Should we use make -k on the buildfarm?

From
Peter Eisentraut
Date:
On lör, 2010-11-06 at 14:45 -0400, Andrew Dunstan wrote:
> Here's the list of tests from a recent run, leaving out stopping and 
> starting the installed postmaster, and locale specifiers:
> 
>     SCM-checkout
>     configure
>     make
>     check
>     make-contrib
>     make-install
>     install-contrib
>     initdb
>     install-check
>     pl-install-check
>     contrib-install-check
>     ecpg-check
> 
> Currently, the implied dependency list is in this order. We could have
> "make-contrib" depend only on "make" rather than "check", 
> "pl-install-check"  and "contrib-install-check" depend on "initdb",
> and "ecpg-check" depend on "make" rather than anything that comes
> after. I think that's about the limit of what we could sensibly relax

In principle you could get this down to

SCM-checkout
configure
make -k world
make -k check-world   # target doesn't exist yet
make -k install-world
initdb
make -k installcheck-world

That way you don't have to update the buildfarm code every time a new
test suite is added.




Re: Should we use make -k on the buildfarm?

From
Andrew Dunstan
Date:

On 11/08/2010 02:49 PM, Peter Eisentraut wrote:
> On lör, 2010-11-06 at 14:45 -0400, Andrew Dunstan wrote:
>> Here's the list of tests from a recent run, leaving out stopping and
>> starting the installed postmaster, and locale specifiers:
>>
>>      SCM-checkout
>>      configure
>>      make
>>      check
>>      make-contrib
>>      make-install
>>      install-contrib
>>      initdb
>>      install-check
>>      pl-install-check
>>      contrib-install-check
>>      ecpg-check
>>
>> Currently, the implied dependency list is in this order. We could have
>> "make-contrib" depend only on "make" rather than "check",
>> "pl-install-check"  and "contrib-install-check" depend on "initdb",
>> and "ecpg-check" depend on "make" rather than anything that comes
>> after. I think that's about the limit of what we could sensibly relax
> In principle you could get this down to
>
> SCM-checkout
> configure
> make -k world
> make -k check-world   # target doesn't exist yet
> make -k install-world
> initdb
> make -k installcheck-world
>
> That way you don't have to update the buildfarm code every time a new
> test suite is added.
>

No, we can't do that for several reasons. Here are a couple.

First, the buildfarm doesn't build the docs. That's a deliberate 
decision, based on the fact that not every member has the required 
software installed. And second these targets only exist for 9.0 and/or 
later.


cheers

andrew


Re: Should we use make -k on the buildfarm?

From
Peter Eisentraut
Date:
On mån, 2010-11-08 at 15:01 -0500, Andrew Dunstan wrote:
> First, the buildfarm doesn't build the docs. That's a deliberate 
> decision, based on the fact that not every member has the required 
> software installed. And second these targets only exist for 9.0 and/or
> later.

I'm aware of those issues.  I'm just saying you could consolidate things
along those lines in the long term.



Re: Should we use make -k on the buildfarm?

From
Tom Lane
Date:
Peter Eisentraut <peter_e@gmx.net> writes:
> On mån, 2010-11-08 at 15:01 -0500, Andrew Dunstan wrote:
>> First, the buildfarm doesn't build the docs. That's a deliberate 
>> decision, based on the fact that not every member has the required 
>> software installed. And second these targets only exist for 9.0 and/or
>> later.

> I'm aware of those issues.  I'm just saying you could consolidate things
> along those lines in the long term.

Probably not: your proposal depends on having "make" available, which it
won't be for Windows builds.  I doubt Andrew wishes to implement
make-equivalent logic in the buildfarm script.
        regards, tom lane


Re: Should we use make -k on the buildfarm?

From
Andrew Dunstan
Date:

On 11/08/2010 03:38 PM, Tom Lane wrote:
> Peter Eisentraut<peter_e@gmx.net>  writes:
>> On mån, 2010-11-08 at 15:01 -0500, Andrew Dunstan wrote:
>>> First, the buildfarm doesn't build the docs. That's a deliberate
>>> decision, based on the fact that not every member has the required
>>> software installed. And second these targets only exist for 9.0 and/or
>>> later.
>> I'm aware of those issues.  I'm just saying you could consolidate things
>> along those lines in the long term.
> Probably not: your proposal depends on having "make" available, which it
> won't be for Windows builds.  I doubt Andrew wishes to implement
> make-equivalent logic in the buildfarm script.

No, and what is more, it runs counter to the philosophy of the 
buildfarm, which is basically to do by automation what a human would do 
by hand in building and testing Postgres.

cheers

andrew


Re: Should we use make -k on the buildfarm?

From
Peter Eisentraut
Date:
On mån, 2010-11-08 at 15:48 -0500, Andrew Dunstan wrote:
> No, and what is more, it runs counter to the philosophy of the 
> buildfarm, which is basically to do by automation what a human would
> do by hand in building and testing Postgres.

Yes, but a human would rarely actually do by hand all the 12 steps you
list, which is kind of part of the problem I'm trying to address.  It's
much simpler to run the 7 steps I list, which are easy-to-remember
top-level targets, than to remember to run 5 or 6 obscure test suites by
hand.

Anyway, doesn't matter, the buildfarm works fine, but your argument
above is backwards.