Thread: make world and install-world without docs
I've been thinking about rationalizing some of the buildfarm code, which
has grown somewhat like Topsy over the years. One useful thing would be
to run all the "make" and "install" pieces together. When the buildfarm
started we didn't have world targets, but they are now almost ancient
history themselves, so it would be nice to leverage them.
However, not all buildfarm animals are set up to build the docs, and not
all owners necessarily want to. Moreover, we have provision for testing
various docs formats (PDF, epub etc). So I'd like to be able to build
and install all the world EXCEPT the docs. Rather than specify yet more
targets in the Makefile, it seemed to me a better way would be to
provide a SKIPDOCS option that could be set on the command line like this:
make SKIPDOCS=1 world
make SKIPDOCS=1 install-world
The attached very small patch is intended to provide for that.
Incidentally, this is exactly what the MSVC build system's 'build.bat'
and 'install.bat' do.
I should add that quite apart from the buildfarm considerations this is
something I've long wanted, and I suspect other developers would find it
useful too.
Obviously to be useful to the buildfarm it would need to be backpatched.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
Attachment
On Mon, May 31, 2021, at 16:16, Andrew Dunstan wrote:
However, not all buildfarm animals are set up to build the docs, and notall owners necessarily want to. Moreover, we have provision for testingvarious docs formats (PDF, epub etc). So I'd like to be able to buildand install all the world EXCEPT the docs.
Why would someone not always want to test building the docs?
What makes the docs special?
/Joel
"Joel Jacobson" <joel@compiler.org> writes:
> On Mon, May 31, 2021, at 16:16, Andrew Dunstan wrote:
>> However, not all buildfarm animals are set up to build the docs, and not
>> all owners necessarily want to. Moreover, we have provision for testing
>> various docs formats (PDF, epub etc). So I'd like to be able to build
>> and install all the world EXCEPT the docs.
> Why would someone not always want to test building the docs?
> What makes the docs special?
Toolchain requirements, cf [1]. Per Andrew's comment, requiring all
that stuff to be installed would move the goalposts quite a ways for
buildfarm owners, and not all of the older systems we have in the farm
would be able to do it easily. (If you don't have access to prebuilt
packages, you're looking at a lot of work to get that stuff
installed.)
It was a good deal worse when we used the TeX-based toolchain
to make PDFs, but it's still not something I want to foist on
buildfarm owners. Especially since there's no real reason
to think that there are platform dependencies that would make
it valuable to run such builds on a spectrum of machines.
We do have a couple of machines that have opted-in to building
the docs, and that seems sufficient. I feel no urge to make
it be opt-out instead.
regards, tom lane
[1] https://www.postgresql.org/docs/devel/docguide-toolsets.html
On 2021-May-31, Andrew Dunstan wrote: > However, not all buildfarm animals are set up to build the docs, and not > all owners necessarily want to. Moreover, we have provision for testing > various docs formats (PDF, epub etc). So I'd like to be able to build > and install all the world EXCEPT the docs. Rather than specify yet more > targets in the Makefile, it seemed to me a better way would be to > provide a SKIPDOCS option that could be set on the command line like this: > > make SKIPDOCS=1 world > make SKIPDOCS=1 install-world I could use this feature. +1 > +ifndef SKIPDOCS > $(call recurse,world,doc src config contrib,all) > world: > +@echo "PostgreSQL, contrib, and documentation successfully made. Ready to install." > +else > +$(call recurse,world,src config contrib,all) > +world: > + +@echo "PostgreSQL and contrib successfully made. Ready to install." > +endif I was going to suggest that instead of repeating the $(call) line you could do something like $(call recurse,world,src config contrib,all) ifndef SKIPDOCS $(call recurse,world,doc,all) endif ... however, this makes the echoed string be wrong, and the whole thing looks uglier if you use a second "ifndef" to generate the string, so I think your proposal is okay. I do wonder if these echoed strings are really all that necessary. -- Álvaro Herrera 39°49'30"S 73°17'W
On 31.05.21 16:16, Andrew Dunstan wrote: > make SKIPDOCS=1 world > make SKIPDOCS=1 install-world Maybe this should be configure option? That's generally where you set what you want to build or not build. (That might also make the buildfarm integration easier, since there are already facilities to specify and report configure options.)
Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
> On 31.05.21 16:16, Andrew Dunstan wrote:
>> make SKIPDOCS=1 world
>> make SKIPDOCS=1 install-world
> Maybe this should be configure option? That's generally where you set
> what you want to build or not build. (That might also make the
> buildfarm integration easier, since there are already facilities to
> specify and report configure options.)
Hmm, I think I prefer Andrew's way. The fact that I don't want
to build the docs right now doesn't mean I won't want to do so
later --- in fact, that sequence is pretty exactly what I do
whenever I'm working on a patch. It'd be annoying to have
to re-configure to make that work.
regards, tom lane
On 6/1/21 6:23 PM, Tom Lane wrote: > Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes: >> On 31.05.21 16:16, Andrew Dunstan wrote: >>> make SKIPDOCS=1 world >>> make SKIPDOCS=1 install-world >> Maybe this should be configure option? That's generally where you set >> what you want to build or not build. (That might also make the >> buildfarm integration easier, since there are already facilities to >> specify and report configure options.) > Hmm, I think I prefer Andrew's way. The fact that I don't want > to build the docs right now doesn't mean I won't want to do so > later --- in fact, that sequence is pretty exactly what I do > whenever I'm working on a patch. It'd be annoying to have > to re-configure to make that work. > > Yes, agreed. If you don't like the SKIPDOCS=1 mechanism, let's just invent a couple of new targets instead, say `world-bin` and `install-world-bin`. I'm inclined to agree with Alvaro that the messages are at best an oddity. Standard Unix practice is to be silent on success. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
Andrew Dunstan <andrew@dunslane.net> writes:
> I'm inclined to agree with Alvaro that the messages are at best an
> oddity. Standard Unix practice is to be silent on success.
We've been steadily moving towards less chatter during builds.
I'd be good with dropping these messages in HEAD, but doing so
in the back branches might be inadvisable.
regards, tom lane
On 6/2/21 4:21 PM, Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: >> I'm inclined to agree with Alvaro that the messages are at best an >> oddity. Standard Unix practice is to be silent on success. > We've been steadily moving towards less chatter during builds. > I'd be good with dropping these messages in HEAD, but doing so > in the back branches might be inadvisable. > > OK, I think on reflection new targets will be cleaner. What I suggest is the attached, applied to all branches, followed by removal of the four noise messages in just HEAD. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
Attachment
Andrew Dunstan <andrew@dunslane.net> writes:
> OK, I think on reflection new targets will be cleaner. What I suggest is
> the attached, applied to all branches, followed by removal of the four
> noise messages in just HEAD.
Shouldn't these new targets be documented somewhere?
regards, tom lane
On 7/1/21 10:50 AM, Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: >> OK, I think on reflection new targets will be cleaner. What I suggest is >> the attached, applied to all branches, followed by removal of the four >> noise messages in just HEAD. > Shouldn't these new targets be documented somewhere? > > Good point. See attached. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
Attachment
Andrew Dunstan <andrew@dunslane.net> writes:
>> Shouldn't these new targets be documented somewhere?
> Good point. See attached.
+1, but spell check: "documantation"
regards, tom lane
On 01.07.21 16:47, Andrew Dunstan wrote: > > On 6/2/21 4:21 PM, Tom Lane wrote: >> Andrew Dunstan <andrew@dunslane.net> writes: >>> I'm inclined to agree with Alvaro that the messages are at best an >>> oddity. Standard Unix practice is to be silent on success. >> We've been steadily moving towards less chatter during builds. >> I'd be good with dropping these messages in HEAD, but doing so >> in the back branches might be inadvisable. > OK, I think on reflection new targets will be cleaner. What I suggest is > the attached, applied to all branches, followed by removal of the four > noise messages in just HEAD. This naming approach is a bit problematic. For example, we have "install-bin" in src/backend/, which is specifically for only installing binaries, not data files etc. (hence the name). Your proposal would confuse this scheme. I think we should also take a step back here and consider: We had "all", which wasn't "all" enough, then we had "world", now we have "world-minus-a-bit", but it's still more than "all". It's like we are trying to prove the continuum hypothesis here. I think we had consensus on the make variable approach, so I'm confused why a different solution was committed and backpatched without discussion.
On 7/1/21 3:39 PM, Peter Eisentraut wrote: > On 01.07.21 16:47, Andrew Dunstan wrote: >> >> On 6/2/21 4:21 PM, Tom Lane wrote: >>> Andrew Dunstan <andrew@dunslane.net> writes: >>>> I'm inclined to agree with Alvaro that the messages are at best an >>>> oddity. Standard Unix practice is to be silent on success. >>> We've been steadily moving towards less chatter during builds. >>> I'd be good with dropping these messages in HEAD, but doing so >>> in the back branches might be inadvisable. > >> OK, I think on reflection new targets will be cleaner. What I suggest is >> the attached, applied to all branches, followed by removal of the four >> noise messages in just HEAD. > > This naming approach is a bit problematic. For example, we have > "install-bin" in src/backend/, which is specifically for only > installing binaries, not data files etc. (hence the name). Your > proposal would confuse this scheme. > > I think we should also take a step back here and consider: We had > "all", which wasn't "all" enough, then we had "world", now we have > "world-minus-a-bit", but it's still more than "all". It's like we are > trying to prove the continuum hypothesis here. > > I think we had consensus on the make variable approach, so I'm > confused why a different solution was committed and backpatched > without discussion. In fact the names and approach were suggested in my email of June 21st. The make variable approach just felt klunky in the end. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
On 01.07.21 22:22, Andrew Dunstan wrote: > > On 7/1/21 3:39 PM, Peter Eisentraut wrote: >> On 01.07.21 16:47, Andrew Dunstan wrote: >>> >>> On 6/2/21 4:21 PM, Tom Lane wrote: >>>> Andrew Dunstan <andrew@dunslane.net> writes: >>>>> I'm inclined to agree with Alvaro that the messages are at best an >>>>> oddity. Standard Unix practice is to be silent on success. >>>> We've been steadily moving towards less chatter during builds. >>>> I'd be good with dropping these messages in HEAD, but doing so >>>> in the back branches might be inadvisable. >> >>> OK, I think on reflection new targets will be cleaner. What I suggest is >>> the attached, applied to all branches, followed by removal of the four >>> noise messages in just HEAD. >> >> This naming approach is a bit problematic. For example, we have >> "install-bin" in src/backend/, which is specifically for only >> installing binaries, not data files etc. (hence the name). Your >> proposal would confuse this scheme. >> >> I think we should also take a step back here and consider: We had >> "all", which wasn't "all" enough, then we had "world", now we have >> "world-minus-a-bit", but it's still more than "all". It's like we are >> trying to prove the continuum hypothesis here. >> >> I think we had consensus on the make variable approach, so I'm >> confused why a different solution was committed and backpatched >> without discussion. > > > In fact the names and approach were suggested in my email of June 21st. AFAICT this thread contains no email from June 21st or thereabouts. https://www.postgresql.org/message-id/flat/6a421136-d462-b043-a8eb-e75b2861f3df%40dunslane.net
On 7/1/21 4:29 PM, Peter Eisentraut wrote: > On 01.07.21 22:22, Andrew Dunstan wrote: >> >> On 7/1/21 3:39 PM, Peter Eisentraut wrote: >>> On 01.07.21 16:47, Andrew Dunstan wrote: >>>> >>>> On 6/2/21 4:21 PM, Tom Lane wrote: >>>>> Andrew Dunstan <andrew@dunslane.net> writes: >>>>>> I'm inclined to agree with Alvaro that the messages are at best an >>>>>> oddity. Standard Unix practice is to be silent on success. >>>>> We've been steadily moving towards less chatter during builds. >>>>> I'd be good with dropping these messages in HEAD, but doing so >>>>> in the back branches might be inadvisable. >>> >>>> OK, I think on reflection new targets will be cleaner. What I >>>> suggest is >>>> the attached, applied to all branches, followed by removal of the four >>>> noise messages in just HEAD. >>> >>> This naming approach is a bit problematic. For example, we have >>> "install-bin" in src/backend/, which is specifically for only >>> installing binaries, not data files etc. (hence the name). Your >>> proposal would confuse this scheme. >>> >>> I think we should also take a step back here and consider: We had >>> "all", which wasn't "all" enough, then we had "world", now we have >>> "world-minus-a-bit", but it's still more than "all". It's like we are >>> trying to prove the continuum hypothesis here. >>> >>> I think we had consensus on the make variable approach, so I'm >>> confused why a different solution was committed and backpatched >>> without discussion. >> >> >> In fact the names and approach were suggested in my email of June 21st. > > AFAICT this thread contains no email from June 21st or thereabouts. > > https://www.postgresql.org/message-id/flat/6a421136-d462-b043-a8eb-e75b2861f3df%40dunslane.net > Apologies. June 2nd. One day American style dates will stop playing havoc with my head - it's only been 25 years or so. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com