Thread: HEAD build failure on win32 mingw

HEAD build failure on win32 mingw

From
ITAGAKI Takahiro
Date:
Hello,

HEAD is failed to be built on win32 mingw.
It requires manual 'mkdir man7' or so.

Are there any changes in build process?
Or am I missing something required to build?

----
$ make
/bin/sh.exe: man7/.timestamp: No such file or directory

$ mkdir doc/man7            <- HERE
$ make
(succeeded to build)

$ make install
../config/install-sh: man7/*.7 does not exist.

$ touch doc/man7/dummy.7    <- HERE
$ make install
(succeeded to install)

----

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center



Re: HEAD build failure on win32 mingw

From
Devrim GÜNDÜZ
Date:
On Thu, 2008-11-20 at 08:51 +0900, ITAGAKI Takahiro wrote:
> HEAD is failed to be built on win32 mingw.
> It requires manual 'mkdir man7' or so.
>
> Are there any changes in build process?

It is probably because of this commit:

http://archives.postgresql.org/pgsql-committers/2008-11/msg00169.php

--
Devrim GÜNDÜZ, RHCE
devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr                  http://www.gunduz.org

Re: HEAD build failure on win32 mingw

From
Tom Lane
Date:
Devrim GÜNDÜZ <devrim@gunduz.org> writes:
> On Thu, 2008-11-20 at 08:51 +0900, ITAGAKI Takahiro wrote:
>> HEAD is failed to be built on win32 mingw.
>> It requires manual 'mkdir man7' or so.
>> 
>> Are there any changes in build process?

> It is probably because of this commit:
> http://archives.postgresql.org/pgsql-committers/2008-11/msg00169.php

Since all the buildfarm machines are still green, a report that "HEAD is
broken on mingw" is entirely useless.  Takahiro-san needs to expend some
effort on determining why his machine is yielding different results from
those ones.
        regards, tom lane


Re: HEAD build failure on win32 mingw

From
Peter Eisentraut
Date:
ITAGAKI Takahiro wrote:
> HEAD is failed to be built on win32 mingw.
> It requires manual 'mkdir man7' or so.
> 
> Are there any changes in build process?
> Or am I missing something required to build?
> 
> ----
> $ make
> /bin/sh.exe: man7/.timestamp: No such file or directory
> 
> $ mkdir doc/man7            <- HERE
> $ make
> (succeeded to build)
> 
> $ make install
> ../config/install-sh: man7/*.7 does not exist.
> 
> $ touch doc/man7/dummy.7    <- HERE
> $ make install
> (succeeded to install)
> 
> ----

This code would only be executed if you have a man.tar.gz in the doc 
directory.  If you do, it is probably an old one that indeed does not 
contain the man7 directory.  So delete the man.tar.gz (and build a new 
one if you are so inclined).


Re: HEAD build failure on win32 mingw

From
ITAGAKI Takahiro
Date:
Peter Eisentraut <peter_e@gmx.net> wrote:

> This code would only be executed if you have a man.tar.gz in the doc 
> directory.  If you do, it is probably an old one that indeed does not 
> contain the man7 directory.  So delete the man.tar.gz (and build a new 
> one if you are so inclined).

I used a nightly snapshot(snapshot/postgresql-snapshot.tar.bz2)
and it includes *.gz files. It would be the cause.

Thanks. I'll delete the file before building.

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center




Re: HEAD build failure on win32 mingw

From
"Jaime Casanova"
Date:
On Thu, Nov 20, 2008 at 4:31 AM, ITAGAKI Takahiro
<itagaki.takahiro@oss.ntt.co.jp> wrote:
>
> I used a nightly snapshot(snapshot/postgresql-snapshot.tar.bz2)
> and it includes *.gz files. It would be the cause.
>

i saw the same on a nightly snapshot a week ago, then used cvs and
everything was good... i thouhgt it was a problem already solved...
could be a problem in the nightly builds?

--
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Asesoría y desarrollo de sistemas
Guayaquil - Ecuador
Cel. +59387171157


Re: HEAD build failure on win32 mingw

From
Andrew Dunstan
Date:

Jaime Casanova wrote:
> On Thu, Nov 20, 2008 at 4:31 AM, ITAGAKI Takahiro
> <itagaki.takahiro@oss.ntt.co.jp> wrote:
>   
>> I used a nightly snapshot(snapshot/postgresql-snapshot.tar.bz2)
>> and it includes *.gz files. It would be the cause.
>>
>>     
>
> i saw the same on a nightly snapshot a week ago, then used cvs and
> everything was good... i thouhgt it was a problem already solved...
> could be a problem in the nightly builds?
>
>   

Yeah. Buildfarm doesn't do docs at all.

Looks to me at first glance like this line in the Makefile:
       $(mkinstalldirs) man$(sqlmansectnum)

should be moved up so it comes immediately after gzip line. That 
certainly seems to make it work properly for me.

cheers

andrew


Re: HEAD build failure on win32 mingw

From
Tom Lane
Date:
Andrew Dunstan <andrew@dunslane.net> writes:
> Jaime Casanova wrote:
>> i saw the same on a nightly snapshot a week ago, then used cvs and
>> everything was good... i thouhgt it was a problem already solved...
>> could be a problem in the nightly builds?

> Yeah. Buildfarm doesn't do docs at all.

> Looks to me at first glance like this line in the Makefile:
>         $(mkinstalldirs) man$(sqlmansectnum)
> should be moved up so it comes immediately after gzip line.

No, that is just a kluge that prevents the visible failure.

It looks to me like the problem is that the makefile expects the
man.tar.gz file to contain stuff in man1/ and man7/, but the file
that is actually being delivered in snapshots still contains man1/
and manl/.  So that file needs to be regenerated.  Peter?
        regards, tom lane


Re: HEAD build failure on win32 mingw

From
Andrew Dunstan
Date:

Tom Lane wrote:
> It looks to me like the problem is that the makefile expects the
> man.tar.gz file to contain stuff in man1/ and man7/, but the file
> that is actually being delivered in snapshots still contains man1/
> and manl/.  So that file needs to be regenerated.  Peter?
>
>             
>   


Shouldn't it be part of the nightly snapshot creation to make that file? 
In the snapshot I just downloaded its date is yesterday.

cheers

andrew





Re: HEAD build failure on win32 mingw

From
Tom Lane
Date:
Andrew Dunstan <andrew@dunslane.net> writes:
> Tom Lane wrote:
>> ... the file
>> that is actually being delivered in snapshots still contains man1/
>> and manl/.  So that file needs to be regenerated.  Peter?

> Shouldn't it be part of the nightly snapshot creation to make that file? 
> In the snapshot I just downloaded its date is yesterday.

The tar file's own date might be yesterday, but its contents are dated
2003-11-02 ... and indeed look to be that old.  So something's bollixed
in the snapshot generation process.

Historically the man.tar.gz files were created manually because there
were some manual fixups needed to the generated man files.  I'm not sure
what vestiges of that still remain --- Peter's generally been the one to
take care of it.  But we definitely aren't shipping a freshly generated
copy in the nightly snapshot right now.

If we do have a fully automated process now, it's probably fair to ask
why there's an internal tarball involved at all, rather than just
shipping the built man1/ and man7/ subdirectories (and likewise for the
html).  Double compression of that data isn't going to be helpful.
        regards, tom lane


Re: HEAD build failure on win32 mingw

From
Peter Eisentraut
Date:
On Friday 21 November 2008 23:33:47 Tom Lane wrote:
> Historically the man.tar.gz files were created manually because there
> were some manual fixups needed to the generated man files.  I'm not sure
> what vestiges of that still remain --- Peter's generally been the one to
> take care of it.  But we definitely aren't shipping a freshly generated
> copy in the nightly snapshot right now.

I don't actually know how the snapshots are built, so I don't know how to fix 
it.

> If we do have a fully automated process now, it's probably fair to ask
> why there's an internal tarball involved at all, rather than just
> shipping the built man1/ and man7/ subdirectories (and likewise for the
> html).  Double compression of that data isn't going to be helpful.

Fixing all this is definitely my plan for 8.4, but we are not there yet.


Re: HEAD build failure on win32 mingw

From
Alvaro Herrera
Date:
Peter Eisentraut wrote:
> On Friday 21 November 2008 23:33:47 Tom Lane wrote:
> > Historically the man.tar.gz files were created manually because there
> > were some manual fixups needed to the generated man files.  I'm not sure
> > what vestiges of that still remain --- Peter's generally been the one to
> > take care of it.  But we definitely aren't shipping a freshly generated
> > copy in the nightly snapshot right now.
> 
> I don't actually know how the snapshots are built, so I don't know how to fix 
> it.

According to Stefan Kaltenbrunner, who took the time to delve into
developer.pg.org guts, the script used to generate the snapshots looks
like this

/usr/bin/cvs -q export -rREL8_3_STABLE pgsql
cd pgsql
./configure
cd doc/src
gmake postgres.tar.gz
mv postgres.tar.gz ..
gmake postgres.tar.gz
mv postgres.tar.gz ..
gmake man.tar.gz
mv man.tar.gz ..
cd sgml
gmake HISTORY INSTALL
mv -f HISTORY INSTALL ../../..
cd ../../..
gmake VERSION=snapshot dist
gmake maintainer-clean

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


Re: HEAD build failure on win32 mingw

From
Tom Lane
Date:
Alvaro Herrera <alvherre@commandprompt.com> writes:
> According to Stefan Kaltenbrunner, who took the time to delve into
> developer.pg.org guts, the script used to generate the snapshots looks
> like this

> /usr/bin/cvs -q export -rREL8_3_STABLE pgsql
> cd pgsql
> ./configure
> cd doc/src
> gmake postgres.tar.gz
> mv postgres.tar.gz ..
> gmake postgres.tar.gz
> mv postgres.tar.gz ..
> gmake man.tar.gz
> mv man.tar.gz ..
> cd sgml
> gmake HISTORY INSTALL
> mv -f HISTORY INSTALL ../../..
> cd ../../..
> gmake VERSION=snapshot dist
> gmake maintainer-clean

That can't be the right script --- the final "maintainer-clean" would
wipe out all the derived files made by the dist step.  Also, it's
*definitely* not the script being used to generate HEAD-branch
snapshots ...
        regards, tom lane


Re: HEAD build failure on win32 mingw

From
Stefan Kaltenbrunner
Date:
Tom Lane wrote:
> Alvaro Herrera <alvherre@commandprompt.com> writes:
>> According to Stefan Kaltenbrunner, who took the time to delve into
>> developer.pg.org guts, the script used to generate the snapshots looks
>> like this
> 
>> /usr/bin/cvs -q export -rREL8_3_STABLE pgsql
>> cd pgsql
>> ./configure
>> cd doc/src
>> gmake postgres.tar.gz
>> mv postgres.tar.gz ..
>> gmake postgres.tar.gz
>> mv postgres.tar.gz ..
>> gmake man.tar.gz
>> mv man.tar.gz ..
>> cd sgml
>> gmake HISTORY INSTALL
>> mv -f HISTORY INSTALL ../../..
>> cd ../../..
>> gmake VERSION=snapshot dist
>> gmake maintainer-clean
> 
> That can't be the right script --- the final "maintainer-clean" would
> wipe out all the derived files made by the dist step.  Also, it's
> *definitely* not the script being used to generate HEAD-branch
> snapshots ...

no it is not the -HEAD situation is significantly worse because what we 
are actually shipping as the manpages there are the ones from 7.4 ...

set -e
export PATH=/usr/local/bin:/usr/bin:/bin:/sbin
export CVSROOT=:ext:scrappy@cvs.postgresql.org:/cvsroot
cd /usr/local/pgsql/snapshot
if [ -d pgsql ]
then  mv pgsql pgsql.dead
fi
if [ -d pgsql.dead ]
then  rm -rf pgsql.dead &
fi
/usr/bin/cvs -q export -rHEAD pgsql
cd pgsql
./configure
cd doc/src
gmake postgres.tar.gz
mv postgres.tar.gz ..
cp ~ftp/pub/dev/doc/man-7.4.tar.gz ../man.tar.gz
cd sgml
gmake HISTORY INSTALL
mv -f HISTORY INSTALL ../../..
cd ../../..
gmake VERSION=snapshot dist
gmake maintainer-clean

for x in *.tar.gz; do  md5 $x > $x.md5
done

cp *.tar.gz *.tar.gz.md5 /var/spool/ftp/pub/snapshot/dev

gunzip *.tar.gz
bzip2 -f *.tar

for x in *.tar.bz2; do  md5 $x > $x.md5
done

mv *.tar.bz2 *.tar.bz2.md5 /var/spool/ftp/pub/snapshot/dev



Stefan


Re: HEAD build failure on win32 mingw

From
Alvaro Herrera
Date:
Tom Lane wrote:
> Alvaro Herrera <alvherre@commandprompt.com> writes:

> > gmake VERSION=snapshot dist
> > gmake maintainer-clean
> 
> That can't be the right script --- the final "maintainer-clean" would
> wipe out all the derived files made by the dist step.  Also, it's
> *definitely* not the script being used to generate HEAD-branch
> snapshots ...

Stefan already posted the correct one, but to clarify why it runs with
the same "make dist ; make maintainer-clean", what's happening is that
dist creates a tarball, so by the time maintainer-clean runs, the files
have already been packed.

The derived files are created by distprep AFAIK.

BTW most of that script is just duplicating the stuff done in distdir ...

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.


Re: HEAD build failure on win32 mingw

From
Alvaro Herrera
Date:
Stefan Kaltenbrunner wrote:

> no it is not the -HEAD situation is significantly worse because what we  
> are actually shipping as the manpages there are the ones from 7.4 ...
>

So how do we get Marc to fix this?

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.


Re: HEAD build failure on win32 mingw

From
Tom Lane
Date:
Alvaro Herrera <alvherre@commandprompt.com> writes:
> So how do we get Marc to fix this?

Actually I think the ball is in Peter's court now.  Marc updated the
snapshot build script, but all the attempts are dying now because the
new docbook2man part of the Makefiles doesn't work on svr1.
        regards, tom lane


Re: HEAD build failure on win32 mingw

From
Peter Eisentraut
Date:
On Friday 28 November 2008 18:25:23 Tom Lane wrote:
> Alvaro Herrera <alvherre@commandprompt.com> writes:
> > So how do we get Marc to fix this?
>
> Actually I think the ball is in Peter's court now.  Marc updated the
> snapshot build script, but all the attempts are dying now because the
> new docbook2man part of the Makefiles doesn't work on svr1.

As far as I know, the snapshot and release building scripts have always taken 
the existing man.tar.gz from the FTP server and put it into the release 
tarball.  This is why the mismatch appeared in the first place, after all.  I 
have uploaded an updated and matching man.tar.gz a while ago already, so 
things should be working again.

This has nothing to do with the new man page build stuff.  Anything official 
should still be using the old stuff.  And the snapshot build shouldn't be 
building anything either way.


Re: HEAD build failure on win32 mingw

From
Tom Lane
Date:
Peter Eisentraut <peter_e@gmx.net> writes:
> On Friday 28 November 2008 18:25:23 Tom Lane wrote:
>> Actually I think the ball is in Peter's court now.  Marc updated the
>> snapshot build script, but all the attempts are dying now because the
>> new docbook2man part of the Makefiles doesn't work on svr1.

> As far as I know, the snapshot and release building scripts have always taken 
> the existing man.tar.gz from the FTP server and put it into the release 
> tarball.  This is why the mismatch appeared in the first place, after all.  I 
> have uploaded an updated and matching man.tar.gz a while ago already, so 
> things should be working again.

> This has nothing to do with the new man page build stuff.  Anything official 
> should still be using the old stuff.  And the snapshot build shouldn't be 
> building anything either way.

Are you sure you're not remembering How Things Used To Be?

AFAICT, we are building both man.tar.gz and postgres.tar.gz on-the-fly
in current releases.  At least, both of them have internal timestamps
supporting that theory in the official 8.3.5 and 8.2.11 tarballs, and
the contained files reflect recent source changes.  It looks to me like
the dev snapshot build process has failed to track the official releases.
        regards, tom lane


Re: HEAD build failure on win32 mingw

From
Peter Eisentraut
Date:
On Saturday 29 November 2008 00:52:03 Tom Lane wrote:
> AFAICT, we are building both man.tar.gz and postgres.tar.gz on-the-fly
> in current releases.  At least, both of them have internal timestamps
> supporting that theory in the official 8.3.5 and 8.2.11 tarballs, and
> the contained files reflect recent source changes.  It looks to me like
> the dev snapshot build process has failed to track the official releases.

Well, Stefan showed upthread that the snapshot script contains this line:

cp ~ftp/pub/dev/doc/man-7.4.tar.gz ../man.tar.gz

This needs to be changed to copy man.tar.gz instead.

Releases might work differently, but that is not really the problem at hand.