Thread: Should we use make -k on the buildfarm?
Peter Eisentraut has suggested that we should run "make -k" instead of plain "make" for most or all of the buildfarm steps. This flag essentially instructs make to keep going rather than fail at the first error. We haven't done that for the last five or six years that the buildfarm has been running, and nobody up to now has complained (that I recall). I don't have any great objection, but before I make this change I thought it might be as well to canvas a wider range of opinion. So, does anyone else have thoughts about it? cheers andrew
Andrew Dunstan <andrew@dunslane.net> writes:
> Peter Eisentraut has suggested that we should run "make -k" instead of
> plain "make" for most or all of the buildfarm steps. This flag
> essentially instructs make to keep going rather than fail at the first
> error. We haven't done that for the last five or six years that the
> buildfarm has been running, and nobody up to now has complained (that I
> recall). I don't have any great objection, but before I make this change
> I thought it might be as well to canvas a wider range of opinion.
> So, does anyone else have thoughts about it?
I don't really care about make -k as such. What I *have* occasionally
wished for is that the buildfarm script would act more like make -k with
respect to the various test stages. That is, not abandon the whole test
after one stage fails, but allow stages that don't logically depend on
the failed one to proceed. But I'm not sure how hard that would be ---
quite aside from coding complexity, it would mean that you could have
more than one failing stage, and I don't know how you'd show that in the
dashboard.
Anyway, no objection to Peter's request; but there's another TODO item
for your buildfarm list, if you want to accept it.
regards, tom lane
<br /><br /> On 11/06/2010 01:07 PM, Tom Lane wrote: <blockquote cite="mid:21127.1289063239@sss.pgh.pa.us" type="cite"><prewrap="">What I *have* occasionally wished for is that the buildfarm script would act more like make -k with respect to the various test stages. That is, not abandon the whole test after one stage fails, but allow stages that don't logically depend on the failed one to proceed. But I'm not sure how hard that would be --- quite aside from coding complexity, it would mean that you could have more than one failing stage, and I don't know how you'd show that in the dashboard. </pre></blockquote><br /> That would be a significant architectural change. I'm not sure how many steps we could run thisway. <br /><br /> Here's the list of tests from a recent run, leaving out stopping and starting the installed postmaster,and locale specifiers:<br /><br /><blockquote>SCM-checkout <br /> configure <br /> make <br />check <br /> make-contrib<br /> make-install <br /> install-contrib <br /> initdb <br /> install-check <br /> pl-install-check <br /> contrib-install-check <br /> ecpg-check<br /></blockquote> Currently,the implied dependency list is in this order. We could have "make-contrib" depend only on "make" rather than "check","pl-install-check" and "contrib-install-check" depend on "initdb", and "ecpg-check" depend on "make" rather thananything that comes after. I think that's about the limit of what we could sensibly relax<br /><br /> I'm not sure thatwould be a great advance. Certainly, right now I'm going to be putting effort into the FTS stuff which I think shouldbe much higher up your list of wants.<br /><br /> cheers<br /><br /> andrew<br />
Andrew Dunstan <andrew@dunslane.net> writes:
> On 11/06/2010 01:07 PM, Tom Lane wrote:
>> What I *have* occasionally
>> wished for is that the buildfarm script would act more like make -k with
>> respect to the various test stages.
> I'm not sure that would be a great advance. Certainly, right now I'm
> going to be putting effort into the FTS stuff which I think should be
> much higher up your list of wants.
Agreed, that would be far more useful.
regards, tom lane
Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: >> On 11/06/2010 01:07 PM, Tom Lane wrote: >>> What I *have* occasionally >>> wished for is that the buildfarm script would act more like make -k with >>> respect to the various test stages. > >> I'm not sure that would be a great advance. Certainly, right now I'm >> going to be putting effort into the FTS stuff which I think should be >> much higher up your list of wants. > > Agreed, that would be far more useful. FTS stuff? Would that also include additions to the current SOAP interface of the buildfarm? lg Stefan
On 11/08/2010 10:27 AM, Stefan Kaltenbrunner wrote: > Tom Lane wrote: >> Andrew Dunstan <andrew@dunslane.net> writes: >>> On 11/06/2010 01:07 PM, Tom Lane wrote: >>>> What I *have* occasionally >>>> wished for is that the buildfarm script would act more like make -k >>>> with >>>> respect to the various test stages. >> >>> I'm not sure that would be a great advance. Certainly, right now I'm >>> going to be putting effort into the FTS stuff which I think should >>> be much higher up your list of wants. >> >> Agreed, that would be far more useful. > > FTS stuff? Would that also include additions to the current SOAP > interface of the buildfarm? > > Possibly. I haven't looked at how we'll use it yet. cheers andrew
On lör, 2010-11-06 at 14:45 -0400, Andrew Dunstan wrote: > Here's the list of tests from a recent run, leaving out stopping and > starting the installed postmaster, and locale specifiers: > > SCM-checkout > configure > make > check > make-contrib > make-install > install-contrib > initdb > install-check > pl-install-check > contrib-install-check > ecpg-check > > Currently, the implied dependency list is in this order. We could have > "make-contrib" depend only on "make" rather than "check", > "pl-install-check" and "contrib-install-check" depend on "initdb", > and "ecpg-check" depend on "make" rather than anything that comes > after. I think that's about the limit of what we could sensibly relax In principle you could get this down to SCM-checkout configure make -k world make -k check-world # target doesn't exist yet make -k install-world initdb make -k installcheck-world That way you don't have to update the buildfarm code every time a new test suite is added.
On 11/08/2010 02:49 PM, Peter Eisentraut wrote: > On lör, 2010-11-06 at 14:45 -0400, Andrew Dunstan wrote: >> Here's the list of tests from a recent run, leaving out stopping and >> starting the installed postmaster, and locale specifiers: >> >> SCM-checkout >> configure >> make >> check >> make-contrib >> make-install >> install-contrib >> initdb >> install-check >> pl-install-check >> contrib-install-check >> ecpg-check >> >> Currently, the implied dependency list is in this order. We could have >> "make-contrib" depend only on "make" rather than "check", >> "pl-install-check" and "contrib-install-check" depend on "initdb", >> and "ecpg-check" depend on "make" rather than anything that comes >> after. I think that's about the limit of what we could sensibly relax > In principle you could get this down to > > SCM-checkout > configure > make -k world > make -k check-world # target doesn't exist yet > make -k install-world > initdb > make -k installcheck-world > > That way you don't have to update the buildfarm code every time a new > test suite is added. > No, we can't do that for several reasons. Here are a couple. First, the buildfarm doesn't build the docs. That's a deliberate decision, based on the fact that not every member has the required software installed. And second these targets only exist for 9.0 and/or later. cheers andrew
On mån, 2010-11-08 at 15:01 -0500, Andrew Dunstan wrote: > First, the buildfarm doesn't build the docs. That's a deliberate > decision, based on the fact that not every member has the required > software installed. And second these targets only exist for 9.0 and/or > later. I'm aware of those issues. I'm just saying you could consolidate things along those lines in the long term.
Peter Eisentraut <peter_e@gmx.net> writes:
> On mån, 2010-11-08 at 15:01 -0500, Andrew Dunstan wrote:
>> First, the buildfarm doesn't build the docs. That's a deliberate
>> decision, based on the fact that not every member has the required
>> software installed. And second these targets only exist for 9.0 and/or
>> later.
> I'm aware of those issues. I'm just saying you could consolidate things
> along those lines in the long term.
Probably not: your proposal depends on having "make" available, which it
won't be for Windows builds. I doubt Andrew wishes to implement
make-equivalent logic in the buildfarm script.
regards, tom lane
On 11/08/2010 03:38 PM, Tom Lane wrote: > Peter Eisentraut<peter_e@gmx.net> writes: >> On mån, 2010-11-08 at 15:01 -0500, Andrew Dunstan wrote: >>> First, the buildfarm doesn't build the docs. That's a deliberate >>> decision, based on the fact that not every member has the required >>> software installed. And second these targets only exist for 9.0 and/or >>> later. >> I'm aware of those issues. I'm just saying you could consolidate things >> along those lines in the long term. > Probably not: your proposal depends on having "make" available, which it > won't be for Windows builds. I doubt Andrew wishes to implement > make-equivalent logic in the buildfarm script. No, and what is more, it runs counter to the philosophy of the buildfarm, which is basically to do by automation what a human would do by hand in building and testing Postgres. cheers andrew
On mån, 2010-11-08 at 15:48 -0500, Andrew Dunstan wrote: > No, and what is more, it runs counter to the philosophy of the > buildfarm, which is basically to do by automation what a human would > do by hand in building and testing Postgres. Yes, but a human would rarely actually do by hand all the 12 steps you list, which is kind of part of the problem I'm trying to address. It's much simpler to run the 7 steps I list, which are easy-to-remember top-level targets, than to remember to run 5 or 6 obscure test suites by hand. Anyway, doesn't matter, the buildfarm works fine, but your argument above is backwards.