Thread: BUG #5595: Documentation is not installs from VPATH build.

The following bug has been logged online:

Bug reference:      5595
Logged by:          Dmtiriy Igrishin
Email address:      dmitigr@gmail.com
PostgreSQL version: 9.0 beta4
Operating system:   Ubuntu Linux 9.04 (2.6.31-22 x86_64)
Description:        Documentation is not installs from VPATH build.
Details:

When 'configure' executed in a directory outside the source tree the
documentation is not installs later nevertheless the "gmake install-docs" or
"gmake install-world" (I tried to build with "gmake world" also) typed to
install PostgreSQL.
When 'configure' executed inside the source tree - all is okay.

"Dmtiriy Igrishin" <dmitigr@gmail.com> writes:
> Description:        Documentation is not installs from VPATH build.

> When 'configure' executed in a directory outside the source tree the
> documentation is not installs later nevertheless the "gmake install-docs" or
> "gmake install-world" (I tried to build with "gmake world" also) typed to
> install PostgreSQL.
> When 'configure' executed inside the source tree - all is okay.

Hmm ... works for me, on a Fedora 13 box.  Maybe a bug in your copy of
gmake?

            regards, tom lane

On 04/08/10 03:35, Tom Lane wrote:
> "Dmtiriy Igrishin"<dmitigr@gmail.com>  writes:
>
>> Description:        Documentation is not installs from VPATH build.
>>
>
>> When 'configure' executed in a directory outside the source tree the
>> documentation is not installs later nevertheless the "gmake install-docs" or
>> "gmake install-world" (I tried to build with "gmake world" also) typed to
>> install PostgreSQL.
>> When 'configure' executed inside the source tree - all is okay.
>>
> Hmm ... works for me, on a Fedora 13 box.  Maybe a bug in your copy of
> gmake?
>
>
>

Hmm, I'm on Ubuntu 10.04 - works for me only if I do the following in
the src tree first:

$ cd pgsql-9.0beta3
$ ./configure
$ make maintainer-clean

... and then do a VPATH configure and build somewhere else. I'm guessing
that if I used a checked out src tree instead of a downloaded one then
it work work fine (suspect Tom used a checked out tree) Tom?

Cheers

Mark

Mark Kirkwood <mark.kirkwood@catalyst.net.nz> writes:
> On 04/08/10 03:35, Tom Lane wrote:
>> Hmm ... works for me, on a Fedora 13 box.  Maybe a bug in your copy of
>> gmake?

> ... and then do a VPATH configure and build somewhere else. I'm guessing
> that if I used a checked out src tree instead of a downloaded one then
> it work work fine (suspect Tom used a checked out tree) Tom?

You're right, I did.  Perhaps the presence of prebuilt docs in the
source tree confuses something --- anybody wanna test?

            regards, tom lane

On 04/08/10 16:55, Tom Lane wrote:
>
> You're right, I did.  Perhaps the presence of prebuilt docs in the
> source tree confuses something --- anybody wanna test?
>
>

The files that seem to be causing the confusion are:

<src tree>/doc/src/sgml/html-stamp
<src tree>/doc/src/sgm/man-stamp

A src tree 'maintainer-clean' removes then, but any lesser level of
clean doesn't. Hmm - shouldn't a VPATH build look at its *own*
doc/src/sgml/*-stamp files to see if it needs to build the docs? Note
that it does *create* them in there after a successful build...

Cheers

Mark

Excerpts from Mark Kirkwood's message of mié ago 04 19:14:07 -0400 2010:
> On 04/08/10 16:55, Tom Lane wrote:
> >
> > You're right, I did.  Perhaps the presence of prebuilt docs in the
> > source tree confuses something --- anybody wanna test?
> >
> >
>
> The files that seem to be causing the confusion are:
>
> <src tree>/doc/src/sgml/html-stamp
> <src tree>/doc/src/sgm/man-stamp
>
> A src tree 'maintainer-clean' removes then, but any lesser level of
> clean doesn't. Hmm - shouldn't a VPATH build look at its *own*
> doc/src/sgml/*-stamp files to see if it needs to build the docs? Note
> that it does *create* them in there after a successful build...

I think the VPATH mechanism in gmake looks for files in srcdir and then
in builddir, by default.  Not sure if that can be overridden easily.

Maybe those two files should be deleted from the tarball.

--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Alvaro Herrera <alvherre@commandprompt.com> writes:
> I think the VPATH mechanism in gmake looks for files in srcdir and then
> in builddir, by default.  Not sure if that can be overridden easily.

> Maybe those two files should be deleted from the tarball.

That's not going to work: it would mean that make would try to rebuild
the docs from scratch instead of installing the prebuilt ones like it's
supposed to.

I'm inclined to think the right fix is that the docs should be built
into $srcdir even in a VPATH build.  I'm no make guru though.  Peter?

            regards, tom lane

On tis, 2010-08-03 at 14:36 +0000, Dmtiriy Igrishin wrote:
> When 'configure' executed in a directory outside the source tree the
> documentation is not installs later nevertheless the "gmake
> install-docs" or
> "gmake install-world" (I tried to build with "gmake world" also) typed
> to
> install PostgreSQL.
> When 'configure' executed inside the source tree - all is okay.

I think I can explain this behavior.  When building outside the source
tree, in looks in the build tree and then in the source tree for
documentation to install.  This is done by looking for a 'html'
directory.  When you already have an 'html' directory in the source tree
before you run configure, an 'html' directory is also created in the
build tree, because the build tree is created by making an empty copy of
every directory in the source tree.  So then it thinks the documentation
is present in the build tree and installs that, but the directory is
empty, so nothing is installed.

We could fix this in several ways.  Either hardcode exceptions for the
'html' directory in prep_buildtree, or only create directories that
contain a makefile, which could also be achieved by pruning empty
directories at the end of prep_buildtree.  All of these might have other
side effects I'm not seeing right now.

Peter Eisentraut <peter_e@gmx.net> writes:
> I think I can explain this behavior.  When building outside the source
> tree, in looks in the build tree and then in the source tree for
> documentation to install.  This is done by looking for a 'html'
> directory.  When you already have an 'html' directory in the source tree
> before you run configure, an 'html' directory is also created in the
> build tree, because the build tree is created by making an empty copy of
> every directory in the source tree.  So then it thinks the documentation
> is present in the build tree and installs that, but the directory is
> empty, so nothing is installed.

Hm, but prep_buildtree doesn't create an html_stamp file in the build
tree, so this explanation doesn't seem to hold water completely.  Why
didn't it try to build new docs in the build tree?

            regards, tom lane

Peter Eisentraut <peter_e@gmx.net> writes:
> We could fix this in several ways.  Either hardcode exceptions for the
> 'html' directory in prep_buildtree, or only create directories that
> contain a makefile, which could also be achieved by pruning empty
> directories at the end of prep_buildtree.  All of these might have other
> side effects I'm not seeing right now.

I looked at this some more and now understand the failure mechanism;
I misunderstood your explanation before.  I think that the simplest
and most reliable fix is your first alternative, ie, hack prep_buildtree
to not auto-create an html directory (we might need the same for the
man dirs, not sure).  This should work for all cases, both building
docs in a VPATH and using pre-built docs from the source tree (but
I will check that).  This is a bit ugly, for sure, but anything that
might have side-effects elsewhere in the build tree will require
more testing than we have time for.  Since the 9.0RC1 wrap deadline
is hard upon us, I'm going to go implement the simple fix.

            regards, tom lane

On tor, 2010-08-26 at 14:06 -0400, Tom Lane wrote:
> Peter Eisentraut <peter_e@gmx.net> writes:
> > We could fix this in several ways.  Either hardcode exceptions for the
> > 'html' directory in prep_buildtree, or only create directories that
> > contain a makefile, which could also be achieved by pruning empty
> > directories at the end of prep_buildtree.  All of these might have other
> > side effects I'm not seeing right now.
>
> I looked at this some more and now understand the failure mechanism;
> I misunderstood your explanation before.  I think that the simplest
> and most reliable fix is your first alternative, ie, hack prep_buildtree
> to not auto-create an html directory (we might need the same for the
> man dirs, not sure).  This should work for all cases, both building
> docs in a VPATH and using pre-built docs from the source tree (but
> I will check that).  This is a bit ugly, for sure, but anything that
> might have side-effects elsewhere in the build tree will require
> more testing than we have time for.  Since the 9.0RC1 wrap deadline
> is hard upon us, I'm going to go implement the simple fix.

Yes, man pages need the same treatment.

If you have "find" superpowers, maybe you can tell it not to process
anything below doc/src/sgml/.

Peter Eisentraut <peter_e@gmx.net> writes:
> If you have "find" superpowers, maybe you can tell it not to process
> anything below doc/src/sgml/.

I just hardwired some rmdir's ;-).  Pretty is not the goal right now.

            regards, tom lane

Excerpts from Peter Eisentraut's message of jue ago 26 14:25:31 -0400 2010:

> If you have "find" superpowers, maybe you can tell it not to process
> anything below doc/src/sgml/.

line 25:

find "$sourcetree" -type d \( -name CVS -prune -o -path "*doc/src/sgml/*" -prune -o -print \)

--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Alvaro Herrera <alvherre@commandprompt.com> writes:
> Excerpts from Peter Eisentraut's message of jue ago 26 14:25:31 -0400 2010:
>> If you have "find" superpowers, maybe you can tell it not to process
>> anything below doc/src/sgml/.

> line 25:

> find "$sourcetree" -type d \( -name CVS -prune -o -path "*doc/src/sgml/*" -prune -o -print \)

Those wildcards look a bit scary, although I suppose only a really
perverse person would put their PG source tree underneath a path
containing that.

            regards, tom lane

Excerpts from Tom Lane's message of jue ago 26 16:38:07 -0400 2010:
> Alvaro Herrera <alvherre@commandprompt.com> writes:
> > Excerpts from Peter Eisentraut's message of jue ago 26 14:25:31 -0400 2010:
> >> If you have "find" superpowers, maybe you can tell it not to process
> >> anything below doc/src/sgml/.
>
> > line 25:
>
> > find "$sourcetree" -type d \( -name CVS -prune -o -path "*doc/src/sgml/*" -prune -o -print \)
>
> Those wildcards look a bit scary, although I suppose only a really
> perverse person would put their PG source tree underneath a path
> containing that.

Hmm, perhaps it can be
    -path "$sourcetree/doc/src/sgml/*"

--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Excerpts from Alvaro Herrera's message of jue ago 26 16:46:42 -0400 2010:
> Excerpts from Tom Lane's message of jue ago 26 16:38:07 -0400 2010:
> > Alvaro Herrera <alvherre@commandprompt.com> writes:
> > > Excerpts from Peter Eisentraut's message of jue ago 26 14:25:31 -0400 2010:
> > >> If you have "find" superpowers, maybe you can tell it not to process
> > >> anything below doc/src/sgml/.
> >
> > > line 25:
> >
> > > find "$sourcetree" -type d \( -name CVS -prune -o -path "*doc/src/sgml/*" -prune -o -print \)
> >
> > Those wildcards look a bit scary, although I suppose only a really
> > perverse person would put their PG source tree underneath a path
> > containing that.
>
> Hmm, perhaps it can be
>     -path "$sourcetree/doc/src/sgml/*"

Yeah, this works for me (i.e. it creates doc/src/sgml but not anything
under it), though I didn't test whether this fixes the original problem.

*** config/prep_buildtree    10 Sep 2001 23:28:59 -0000    1.5
--- config/prep_buildtree    26 Aug 2010 20:56:40 -0000
***************
*** 22,35 ****

  buildtree=`cd ${2:-'.'} && pwd`

! for item in `find "$sourcetree" -type d \( -name CVS -prune -o -print \)`; do
      subdir=`expr "$item" : "$sourcetree\(.*\)"`
      if test ! -d "$buildtree/$subdir"; then
          mkdir -p "$buildtree/$subdir" || exit 1
      fi
  done

! for item in `find "$sourcetree" -name Makefile -print -o -name GNUmakefile -print`; do
      filename=`expr "$item" : "$sourcetree\(.*\)"`
      if test ! -f "${item}.in"; then
          if cmp "$item" "$buildtree/$filename" >/dev/null 2>&1; then : ; else
--- 22,35 ----

  buildtree=`cd ${2:-'.'} && pwd`

! for item in `find "$sourcetree" -type d \( -name CVS -prune -o -path "$sourcetree/doc/src/sgml/*" -prune -o -print
\)`;do 
      subdir=`expr "$item" : "$sourcetree\(.*\)"`
      if test ! -d "$buildtree/$subdir"; then
          mkdir -p "$buildtree/$subdir" || exit 1
      fi
  done

! for item in `find "$sourcetree" \( -name Makefile -o -name GNUmakefile \) -print `; do
      filename=`expr "$item" : "$sourcetree\(.*\)"`
      if test ! -f "${item}.in"; then
          if cmp "$item" "$buildtree/$filename" >/dev/null 2>&1; then : ; else

--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Excerpts from Alvaro Herrera's message of jue ago 26 17:01:14 -0400 2010:
> Excerpts from Alvaro Herrera's message of jue ago 26 16:46:42 -0400 2010:
> > Excerpts from Tom Lane's message of jue ago 26 16:38:07 -0400 2010:

> > > Those wildcards look a bit scary, although I suppose only a really
> > > perverse person would put their PG source tree underneath a path
> > > containing that.
> >
> > Hmm, perhaps it can be
> >     -path "$sourcetree/doc/src/sgml/*"
>
> Yeah, this works for me (i.e. it creates doc/src/sgml but not anything
> under it), though I didn't test whether this fixes the original problem.

I confirmed last night that it fixes the problem (install docs on a
VPATH built from a tarball), so I committed it and removed the rmdirs.

Served nicely as a test case for my git pull/push workflow.

--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support