Thread: Distribution making
I've been collecting a few ideas for integrating the distribution making process into the build system. (I did take a look at the mk-release and mk-snapshot scripts on hub.org as well.) I have a trial implementation which works well, except that it doesn't build the documentation. 1. Instead of the release_prep script we have a new target `distprep'. Those makefiles that want to build something for the distribution can simply implement this target. This is nice because you have the distprep and the correspondng distclean target in the same file. To make a distribution-ready tree, you do ./configure make distprep make distclean 2. In order to get rid of the CVS directories (and anything else you don't want to ship, like _deadcode) and get the directory name straight, *without* clobbering your checked out tree, I create a directory "postgresql-$(VERSION)" and copy the files I want to distribute in there. The conceptual procedure is # use your already configured tree make distprep mkdir postgresql-$(VERSION) copy all files from . into postgresql-$(VERSION), expect those you don't want make -C postgresql-$(VERSION) distclean This is done by `make distdir'. 3. `make dist' depends on distdir, tars up the prepared tree, and leaves a file postgresql-$(VERSION).tar.gz, which you can give to your friends. 4. If you got an extra half hour you can run `make distcheck', which: * makes a distribution, using make dist * unpacks the distribution * runs configure * runs make -q distprep, to check whether the files you just prepared for distribution are really still up to date * builds and installs everything * runs make uninstall and checks whether it really uninstalled everything * makes another distribution from this test tree * checks whether this distribution is sufficiently similar to the previous one (i.e., same files, same size) This approach should guard against the common tarball making problems: misnamed top-level directory, missing files, funky timestamps, etc. If there's any interest in this I can commit it so you can take a look. It doesn't affect anything else (including release_prep). -- Peter Eisentraut Sernanders väg 10:115 peter_e@gmx.net 75262 Uppsala http://yi.org/peter-e/ Sweden
On Mon, 10 Jul 2000, Peter Eisentraut wrote:
> 2. In order to get rid of the CVS directories (and anything else you don't
> want to ship, like _deadcode) and get the directory name straight,
Need we still "_deadcode", if we have CVS? BTW. --- who compile backend
with "#define NOT_USED"?
> 3. `make dist' depends on distdir, tars up the prepared tree, and leaves a
> file postgresql-$(VERSION).tar.gz, which you can give to your friends.
And what ".orig.tar.gz"? We Debian's user love it :-) (and IMHO it is
really not bad idea.)
Karel
PS. not flame please...
Sounds pretty good to me... but Marc's the guy who would normally be
using it, so it's his wishes you gotta cater to...
regards, tom lane
Karel Zak writes: > > 3. `make dist' depends on distdir, tars up the prepared tree, and leaves a > > file postgresql-$(VERSION).tar.gz, which you can give to your friends. > > And what ".orig.tar.gz"? We Debian's user love it :-) (and IMHO it is > really not bad idea.) What's that? -- Peter Eisentraut Sernanders väg 10:115 peter_e@gmx.net 75262 Uppsala http://yi.org/peter-e/ Sweden
Sounds great to me ... what we have now is a major kludge ... Basically, what I'd have to do for a release is CVSUP the latest code on that branch, do a 'make dist' and that's it? Is this something that can be 'extended' to do the RPM also? :) On Mon, 10 Jul 2000, Peter Eisentraut wrote: > I've been collecting a few ideas for integrating the distribution making > process into the build system. (I did take a look at the mk-release and > mk-snapshot scripts on hub.org as well.) I have a trial implementation > which works well, except that it doesn't build the documentation. > > 1. Instead of the release_prep script we have a new target `distprep'. > Those makefiles that want to build something for the distribution can > simply implement this target. This is nice because you have the distprep > and the correspondng distclean target in the same file. To make a > distribution-ready tree, you do > > ./configure > make distprep > make distclean > > 2. In order to get rid of the CVS directories (and anything else you don't > want to ship, like _deadcode) and get the directory name straight, > *without* clobbering your checked out tree, I create a directory > "postgresql-$(VERSION)" and copy the files I want to distribute in there. > The conceptual procedure is > > # use your already configured tree > make distprep > mkdir postgresql-$(VERSION) > copy all files from . into postgresql-$(VERSION), expect those you don't want > make -C postgresql-$(VERSION) distclean > > This is done by `make distdir'. > > 3. `make dist' depends on distdir, tars up the prepared tree, and leaves a > file postgresql-$(VERSION).tar.gz, which you can give to your friends. > > 4. If you got an extra half hour you can run `make distcheck', which: > > * makes a distribution, using make dist > * unpacks the distribution > * runs configure > * runs make -q distprep, to check whether the files you just prepared for > distribution are really still up to date > * builds and installs everything > * runs make uninstall and checks whether it really uninstalled everything > * makes another distribution from this test tree > * checks whether this distribution is sufficiently similar to the previous > one (i.e., same files, same size) > > This approach should guard against the common tarball making problems: > misnamed top-level directory, missing files, funky timestamps, etc. > > > If there's any interest in this I can commit it so you can take a look. It > doesn't affect anything else (including release_prep). > > -- > Peter Eisentraut Sernanders v�g 10:115 > peter_e@gmx.net 75262 Uppsala > http://yi.org/peter-e/ Sweden > Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy Systems Administrator @ hub.org primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
The Hermit Hacker wrote:
> Is this something that can be 'extended' to do the RPM also? :)
Yes. Do you want that to happen? It won't be too hard -- the
requirements would be simply having an 'RPM' directory in the tarball
that would contain the spec file and the required patches and ancilliary
programs. Then, you simply issue (on a RedHat system :-)) the
appropriate 'rpm -tb' (or 'rpm -ta') command. The binary RPM's will be
built to the usual place, and all is well with the world.
Although the spec file might need to be in the tarball's top-level
directory to work -- I'll have to investigate this.
So, the RPM-building would go like:
download postgresql-$version.tar.gz to %{_topdir}/SOURCES
(in %{_topdir}/SOURCES, execute 'rpm -tb' (or -ta).
Pick up your RPM's in %{_topdir}/RPMS/%{_arch}
Install them with rpm -Uvh.
No sweat. See, the RPM building process is already highly automated --
once the patches are made, that is.
Although, I must admit -- it would be very nice to not have those
patches.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11
On Mon, 10 Jul 2000, Lamar Owen wrote:
> The Hermit Hacker wrote:
> > Is this something that can be 'extended' to do the RPM also? :)
>
> Yes. Do you want that to happen? It won't be too hard -- the
> requirements would be simply having an 'RPM' directory in the tarball
> that would contain the spec file and the required patches and ancilliary
> programs. Then, you simply issue (on a RedHat system :-)) the
> appropriate 'rpm -tb' (or 'rpm -ta') command. The binary RPM's will be
> built to the usual place, and all is well with the world.
>
> Although the spec file might need to be in the tarball's top-level
> directory to work -- I'll have to investigate this.
>
> So, the RPM-building would go like:
> download postgresql-$version.tar.gz to %{_topdir}/SOURCES
> (in %{_topdir}/SOURCES, execute 'rpm -tb' (or -ta).
> Pick up your RPM's in %{_topdir}/RPMS/%{_arch}
> Install them with rpm -Uvh.
>
> No sweat. See, the RPM building process is already highly automated --
> once the patches are made, that is.
>
> Although, I must admit -- it would be very nice to not have those
> patches.
If this can be worked in along side/on top of what Peter is doing, I think
that would be great ...
On Tue, 11 Jul 2000, Peter Eisentraut wrote:
> Karel Zak writes:
>
> > > 3. `make dist' depends on distdir, tars up the prepared tree, and leaves a
> > > file postgresql-$(VERSION).tar.gz, which you can give to your friends.
> >
> > And what ".orig.tar.gz"? We Debian's user love it :-) (and IMHO it is
> > really not bad idea.)
>
> What's that?
This is the original source package. But 'tar.gz' is some common archive
file. See any debian's ftp....
Karel
On Mon, 10 Jul 2000, The Hermit Hacker wrote:
> Is this something that can be 'extended' to do the RPM also? :)
Into standard *source* tree? I was long time glad of PG's devevelopment,
that create dateless and OS independent source.
Well, we will have support for RPM, do you want support for Debian too?
If yes, do you accept /Debian directory in top source tree?
Do you accept different directory philosophy for several dists.?
...etc.
IMHO to *source* tree belong to matter for binary making only. A
distribution must be out of source.
Karel
PS. is it this equation right: "OS = Linux = RH"? I hope that not.
Peter Eisentraut wrote: >Karel Zak writes: > >> > 3. `make dist' depends on distdir, tars up the prepared tree, and leaves > a >> > file postgresql-$(VERSION).tar.gz, which you can give to your friends. >> >> And what ".orig.tar.gz"?We Debian's user love it :-) (and IMHO it is >> really not bad idea.) > >What's that? A Debian source package consists of the upstream source (named package.orig.tar.gz) together with a diff file and a control file. These should be all that are needed to rebuild the binary package from the source. Some upstream developers make it hard for us, though. :-( -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight http://www.lfix.co.uk/oliver PGP: 1024R/32B8FAA1: 97 EA 1D 47 72 3F 28 47 6B 7E 39 CC 56 E4 C1 47 GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C ======================================== "I waited patiently for the LORD; and he inclined unto me, and heard my cry.He brought me up also out of an horrible pit, out of the miry clay, and set my feet upon a rock, and establishedmy goings. And he hath put a new song in my mouth, even praise unto our God. Many shall see it, and fear,and shall trust in the LORD." Psalms 40:1-3
The Hermit Hacker wrote: > On Mon, 10 Jul 2000, Lamar Owen wrote: > > The Hermit Hacker wrote: > > > Is this something that can be 'extended' to do the RPM also? :) > > Yes. Do you want that to happen? It won't be too hard -- the > If this can be worked in along side/on top of what Peter is doing, I think > that would be great ... Well, it can be done. However, the RPM' have historically had a tighter cycle than the official release -- We have 7.0.2-1 and -2 RPM's (soon to be up to -6 or -7) a problems are found in the packaging. Of course, as I try to synchronize things, maybe that cycle time can go away. :-) Now, I know that, in the past, feature patches were released on postgresql.org (as well as bugfix patches) that were inbetween releases -- that could be done for the RPM stuff easily enough. But, I'd want some comments from RPM distribution users before going that route. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Karel Zak wrote: > On Mon, 10 Jul 2000, The Hermit Hacker wrote: > > Is this something that can be 'extended' to do the RPM also? :) > Into standard *source* tree? I was long time glad of PG's devevelopment, > that create dateless and OS independent source. > Well, we will have support for RPM, do you want support for Debian too? > If yes, do you accept /Debian directory in top source tree? > Do you accept different directory philosophy for several dists.? I don't mind one bit to have multiple distribution support in the tarball. > ...etc. > IMHO to *source* tree belong to matter for binary making only. A > distribution must be out of source. There are more files in the RPM set, for instance, than just the binaries -- there is the spec file, which controls the building of the RPM's; there is a patch file (to patch around some madness in the source that breaks the RPM package in some way); there are a couple of scripts (startup and upgrade); there is a man page for the upgrade script; as well as other things. Now, getting the *source* of the RPM distribution packaging into the tarball might be OK -- I'm certainly not advocating packaging binaries in the tarball! RPM is used on many more systems than just RedHat Linux -- and I am actively trying to get the *source* RPM to be buildable on as many RPM-based systems as possible -- including being buildable on a Solaris system that might have RPM installed, or a Tru64 system with RPM installed (they DO exist). Unfortunately there are a number of RedHat-isms in the current RPMset that are having to be worked around -- although, this situation promises to improve in the near future, if I have anything to do with it. Therefore, a separate source RPM would not need to be distributed -- a person can just download the tarball, execute a single command, and have properly built RPM's for their system ready to install. Can't get much easier than that! > PS. is it this equation right: "OS = Linux = RH"? I hope that not. No, and neither is RPM == "RedHat Linux Only." -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
On Tue, 11 Jul 2000, Lamar Owen wrote:
> > IMHO to *source* tree belong to matter for binary making only. A
> > distribution must be out of source.
>
> There are more files in the RPM set, for instance, than just the
> binaries -- there is the spec file, which controls the building of the
> RPM's; there is a patch file (to patch around some madness in the source
> that breaks the RPM package in some way); there are a couple of scripts
> (startup and upgrade); there is a man page for the upgrade script; as
> well as other things. Now, getting the *source* of the RPM distribution
> packaging into the tarball might be OK -- I'm certainly not advocating
> packaging binaries in the tarball!
I good known how act RH packaging. And I probably understant you.
> Therefore, a separate source RPM would not need to be distributed -- a
> person can just download the tarball, execute a single command, and have
> properly built RPM's for their system ready to install. Can't get much
> easier than that!
Yes, I know. But it expect that in the *common-original-source* must be
.spec file. Or not?
I'm not enemy of RH, I only not sure if is good "foul" original source.
Karel
The Hermit Hacker writes: > Basically, what I'd have to do for a release is CVSUP the latest code on > that branch, do a 'make dist' and that's it? Yes. You'll just have to copy the documentation files in there first. Of course eventually that should work better as well. > Is this something that can be 'extended' to do the RPM also? :) I'd have to look at how the RPMs are made, but in theory why not. -- Peter Eisentraut Sernanders väg 10:115 peter_e@gmx.net 75262 Uppsala http://yi.org/peter-e/ Sweden
From: Karel Zak <zakkr@zf.jcu.cz> Date: Tue, 11 Jul 2000 18:22:30 +0200 (CEST) Hi Karel, > Yes, I know. But it expect that in the *common-original-source* must > be .spec file. Or not? not, of course. But if it is there, you can easily built RPM via rpm -ta source*tar.gz. Only Source RPMS (aka SRPM) must have .spec file built in. > I'm not enemy of RH, I only not sure if is good "foul" original > source. The same for me. I do hate packagers who just rename my package.tar.gz to package.orig.tar.gz and built tens of patches into one big .diff.gz which is crazy to maintain :-)) No flamewars please. BTW - Searching of mailing-lists is not working on postgresql.org. Try this very long link :-)) http://www.postgresql.org/mhonarc/pgsql-hackers/search.cgi?ps=10&ul=http%3A%2F%2Fwww.postgresql.org%2Fmhonarc%2Fpgsql-hackers%2F%25&q=CREATE+SNAPSHOT&ps=10&o=0&m=all -- Pavel Janík ml. Pavel.Janik@linux.cz