Thread: pgxs default installation + various fixes
Dear patchers,
This patch addresses open item "make pgxs install by default" in Bruce's
list.
(1) "make intall" installs everything.
(2) "make light-install" does not install pgxs and server dev stuff.
this is the previous version of "make install". This target maybe
of interest of packagers. If you want any other name, just tell
me what you want!
(3) the installation.sgml documentation reflects the new status...
but only with my poor English.
(4) I noticed that some pgxs related files were incidently installed
under "light-install", so I make sure that they are not.
(5) a minor stylistic fix in the pgxs makefile, so that it does not
have a double slash (/some/directory//sub/directory) in some paths.
What might be consider for future improvements,
but ISTM not mandatory for 8.0?
- maybe rename "install-all-headers" target, as it also install
scripts, makefiles and libraries useful for server developpement.
I would suggest "install-server-dev".
- make the contrib regressions work with pgxs. I'm not sure about what
is needed for that.
I'm available to fix any problem with this patch.
Have a nice day,
--
Fabien Coelho - coelho@cri.ensmp.fr
Attachment
Am Donnerstag, 2. September 2004 10:16 schrieb Fabien COELHO: > (1) "make intall" installs everything. > > (2) "make light-install" does not install pgxs and server dev stuff. > this is the previous version of "make install". If we do that, we should remove install-all-headers. It's very confusing otherwise. > This target maybe of interest of packagers. This I don't understand. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Dear Peter, > > (2) "make light-install" does not install pgxs and server dev stuff. > > this is the previous version of "make install". > > If we do that, we should remove install-all-headers. It's very confusing > otherwise. > > > This target maybe of interest of packagers. > > This I don't understand. I meant: "This target may be of interest for packagers", sorry for my poor English writing and typing skills. This is to take into account Tom's view that distributions are often split with a separate "dev" package for developpements. For instance under Debian there are: postgresql, postgresql-client, postgresql-dev, postgresql-doc... So this target would help checking what files belong to what part of such split installation for the package maintainer. I guess that could possibly help administrators that would like to install a small-footprint database. Now, if everyone agree that having a distinct installation for server headers and other stuff is useless, then indeed we can just keep the install target and drop "install-all-headers" altogether, I agree with you. That would indeed simplify the patch and the various makefiles, and be less confusing. If there is a consensus, I can remove both "light-install" and "install -all-headers" targets in another submission. Have a nice day, -- Fabien Coelho - coelho@cri.ensmp.fr
Am Donnerstag, 2. September 2004 14:31 schrieb Fabien COELHO: > I meant: "This target may be of interest for packagers", sorry for my poor > English writing and typing skills. I understood that. :) > This is to take into account Tom's view that distributions are often split > with a separate "dev" package for developpements. For instance under > Debian there are: postgresql, postgresql-client, postgresql-dev, > postgresql-doc... So this target would help checking what files belong to > what part of such split installation for the package maintainer. No, the way this works is that the package building script installs everything in one tree and then has file lists (with wildcards) that determine which files belong in which package. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Dear Peter, > > This is to take into account Tom's view that distributions are often split > > with a separate "dev" package for developpements. For instance under > > Debian there are: postgresql, postgresql-client, postgresql-dev, > > postgresql-doc... So this target would help checking what files belong to > > what part of such split installation for the package maintainer. > > No, the way this works is that the package building script installs > everything in one tree and then has file lists (with wildcards) that > determine which files belong in which package. Ok. Thus it suggests that the only use for the two targets is whether someone is interested in having a smaller footprint version... The other reason why I sticked to it in the submission is to be upward compatible with the previous state with 2 differents targets. I feel it is a "political" decision to shift to a single target install, and I'm not the one to take such a decision... I'm only really interested in having server headers and pgxs stuff installed by default, so that extensions can be reliably added on a default installation. So, the question remains the same: is there a consensus to drop install-all-headers/light-install targets for a simple and full "install"? If so, I'll make a new submission. -- Fabien Coelho - coelho@cri.ensmp.fr
On Thu, Sep 02, 2004 at 02:31:14PM +0200, Fabien COELHO wrote: > This is to take into account Tom's view that distributions are often split > with a separate "dev" package for developpements. For instance under > Debian there are: postgresql, postgresql-client, postgresql-dev, > postgresql-doc... So this target would help checking what files belong to > what part of such split installation for the package maintainer. What they do AFAIK is to run the biggest install target and then classify in different packages using file and glob lists hardcoded in the package specification. Having more make targets does not help them. At least that's how RPM packages are done. -- Alvaro Herrera (<alvherre[a]dcc.uchile.cl>) "I call it GNU/Linux. Except the GNU/ is silent." (Ben Reiter)
On Thu, Sep 02, 2004 at 02:10:13PM +0200, Peter Eisentraut wrote: > Am Donnerstag, 2. September 2004 10:16 schrieb Fabien COELHO: > > (1) "make intall" installs everything. > > > > (2) "make light-install" does not install pgxs and server dev stuff. > > this is the previous version of "make install". > > If we do that, we should remove install-all-headers. It's very confusing > otherwise. Why not get rid of install-all-headers and light-install? Just have one target that installs all that's needed, and call it "install". Are we trying to save disk space or what? People who are short on disk space because they are trying to run on a small device has a lot of manual work to do anyway. > > This target maybe of interest of packagers. > > This I don't understand. I think the argument is that packagers may want to have different make targets for the main package and the "development" package. AFAIK, actually they don't. What was the argument for having an "install-all-headers" target in the first place? It creates a lot of unnecessary pain, just to save some disk space. -- Alvaro Herrera (<alvherre[a]dcc.uchile.cl>) "I think my standards have lowered enough that now I think 'good design' is when the page doesn't irritate the living f*ck out of me." (JWZ)
Alvaro Herrera <alvherre@dcc.uchile.cl> writes:
> What was the argument for having an "install-all-headers" target in the
> first place?
By my count it adds about 2 megabytes to the installed footprint, which
perhaps is not so much now as it was at the time.
A larger problem from my point of view is that the installation process
seems quite inefficient. "make install-all-headers" takes a full minute
on my devel machine, which is above my threshold of pain for something
that I often do more than once a day.
regards, tom lane
Dear Alvaro, > > As I'm into these files, I can say that one of the reason for that is that > > the shell scripts in the makefile looks inefficient, with nested for-loops > > and one-at-a-time config/install-sh forked-script copies for 350 header > > files, on the 971 files of a standard installation. > > Also the install-sh script apparently is way more complex than it needs > to be. Maybe. I guess the options are there because they might be useful sometimes? > There's probably a lot of that complexity (and subsequent slowness) that > install-all-headers doesn't need. Maybe. I don't have a clear view about portability issues that I guess justify this script. > A lot of time goes into processing the script itself rather than doing > useful work. Yes. > Is there an objection to trying to convert it to a simpler, faster > alternative? Maybe even one that receives multiple files as arguments, > which would reduce the number of times it is called by an order of > magnitude. Yes, handling several files at a time could indeed improve the stuff. But this means changing the syntax somehow, and fixing makefiles... Also, most unix box have an "install" program which might be more efficient and which handles several files. I do not know whether it has all the required facilities. For instance, apache looks for a "bsd" install program at configuration time... and a slow but compatible shell substitute is used instead if none is available. Maybe this can be reused quite simply by postgresql, with their kind permission. As apache is quite portable, it might be good enough for pg. This seems a reasonnable todo objective, but I'm not sure it should be done for 8.0 as it changes the installation procedure significantly? Well, maybe it could be done quite quiclky. It does not look too difficult to implement with apache example at hand. -- Fabien Coelho - coelho@cri.ensmp.fr
On Fri, Sep 03, 2004 at 10:42:30AM +0200, Fabien COELHO wrote: > As I'm into these files, I can say that one of the reason for that is that > the shell scripts in the makefile looks inefficient, with nested for-loops > and one-at-a-time config/install-sh forked-script copies for 350 header > files, on the 971 files of a standard installation. Also the install-sh script apparently is way more complex than it needs to be. There's probably a lot of that complexity (and subsequent slowness) that install-all-headers doesn't need. A lot of time goes into processing the script itself rather than doing useful work. Is there an objection to trying to convert it to a simpler, faster alternative? Maybe even one that receives multiple files as arguments, which would reduce the number of times it is called by an order of magnitude. -- Alvaro Herrera (<alvherre[a]dcc.uchile.cl>) <Schwern> It does it in a really, really complicated way <crab> why does it need to be complicated? <Schwern> Because it's MakeMaker.
Dear Tom, dear Peter, > By my count it adds about 2 megabytes to the installed footprint, which > perhaps is not so much now as it was at the time. Ok. I agree. I'm investigating on how to do that, and the choice will be to committers. > A larger problem from my point of view is that the installation process > seems quite inefficient. "make install-all-headers" takes a full minute > on my devel machine, which is above my threshold of pain for something > that I often do more than once a day. As I'm into these files, I can say that one of the reason for that is that the shell scripts in the makefile looks inefficient, with nested for-loops and one-at-a-time config/install-sh forked-script copies for 350 header files, on the 971 files of a standard installation. I guess it a deliberate portability choice that only standard shell and file wildcards are used. I'm not sure there is a simple and *portable* performance fix on the installation. Some files are copied more that once: for instance, pg_config.h is copied both in include/ and include/server, and there are others examples file that. In fact, all files and directories under include/ but include/server seems also copied into include/server. It seems to be a 200KB and 28 files replication. Not really big deal, but not really clean either. If there is only one installation target which includes all header files, ISTM that the include/server subdirectory does not make much sense anymore? On the other hand, it might help packagers to have a clear list of the use of all headers files, in order that they can figure out where they belong to. Opinions? -- Fabien Coelho - coelho@cri.ensmp.fr
Fabien COELHO <coelho@cri.ensmp.fr> writes:
> For instance, apache looks for a "bsd" install program at configuration
> time... and a slow but compatible shell substitute is used instead if none
> is available.
We used to do that, and gave up on it because there was too much
variability in the available install programs.
regards, tom lane
Where are we on this? --------------------------------------------------------------------------- Fabien COELHO wrote: > > Dear patchers, > > This patch addresses open item "make pgxs install by default" in Bruce's > list. > > (1) "make intall" installs everything. > > (2) "make light-install" does not install pgxs and server dev stuff. > this is the previous version of "make install". This target maybe > of interest of packagers. If you want any other name, just tell > me what you want! > > (3) the installation.sgml documentation reflects the new status... > but only with my poor English. > > (4) I noticed that some pgxs related files were incidently installed > under "light-install", so I make sure that they are not. > > (5) a minor stylistic fix in the pgxs makefile, so that it does not > have a double slash (/some/directory//sub/directory) in some paths. > > > > What might be consider for future improvements, > but ISTM not mandatory for 8.0? > > - maybe rename "install-all-headers" target, as it also install > scripts, makefiles and libraries useful for server developpement. > I would suggest "install-server-dev". > > - make the contrib regressions work with pgxs. I'm not sure about what > is needed for that. > > I'm available to fix any problem with this patch. > > Have a nice day, > > -- > Fabien Coelho - coelho@cri.ensmp.fr Content-Description: [ Attachment, skipping... ] > > ---------------------------(end of broadcast)--------------------------- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Where are we on this?
I think we're waiting on Peter to review it.
regards, tom lane
> Bruce Momjian <pgman@candle.pha.pa.us> writes: > > Where are we on this? > > I think we're waiting on Peter to review it. Yes, indeed. I've made two different submissions on this issue. It is up to the reviewer/committer to chose, and I'm willing to solve any issue with these patches if needed. -- Fabien Coelho - coelho@cri.ensmp.fr
Well, we don't have forever to wait on this, and I don't plan to require patches to be held until one specific person can review them. Fabien, which version is your best? I will put it in the patch queue. --------------------------------------------------------------------------- Fabien COELHO wrote: > > > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > > Where are we on this? > > > > I think we're waiting on Peter to review it. > > Yes, indeed. > > I've made two different submissions on this issue. It is up to the > reviewer/committer to chose, and I'm willing to solve any issue with these > patches if needed. > > -- > Fabien Coelho - coelho@cri.ensmp.fr > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Tom Lane wrote: > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > Where are we on this? > > I think we're waiting on Peter to review it. I have no technical issue with the patch, except that as I have previously mentioned, I don't particularly care for what it does. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Peter Eisentraut wrote: > Tom Lane wrote: > > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > > Where are we on this? > > > > I think we're waiting on Peter to review it. > > I have no technical issue with the patch, except that as I have > previously mentioned, I don't particularly care for what it does. Yea, I am sure it isn't 100% ideal but what other options to we have if we want this type of functionality, which I think we have agreed as a group we do? -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Fabien COELHO wrote: > Thus it suggests that the only use for the two targets is whether someone > is interested in having a smaller footprint version... > > The other reason why I sticked to it in the submission is to be upward > compatible with the previous state with 2 differents targets. I feel it is > a "political" decision to shift to a single target install, and I'm not > the one to take such a decision... I'm only really interested in having > server headers and pgxs stuff installed by default, so that extensions can > be reliably added on a default installation. > > So, the question remains the same: is there a consensus to drop > install-all-headers/light-install targets for a simple and full "install"? This is the 8.0 release --- let's remove those targets. Our logic has usually been to remove confusing options and document their removal rather than keep them around and continue confusing people forever. Would you resubmit with adjustments for comments made? -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Dear Bruce, >> So, the question remains the same: is there a consensus to drop >> install-all-headers/light-install targets for a simple and full "install"? > > This is the 8.0 release --- let's remove those targets. Our logic has > usually been to remove confusing options and document their removal > rather than keep them around and continue confusing people forever. > > Would you resubmit with adjustments for comments made? What comments do you want me to adjust? Do you mean I should add a paragraph is the documentation about that now there is only one install target. I updated the documentation on both patch submission, but if more is required I'll do what I ask, just be more specific. The v2 version is the one which has a single and simple "install" target everywhere. -- Fabien.
Fabien COELHO wrote: > > Dear Bruce, > > >> So, the question remains the same: is there a consensus to drop > >> install-all-headers/light-install targets for a simple and full "install"? > > > > This is the 8.0 release --- let's remove those targets. Our logic has > > usually been to remove confusing options and document their removal > > rather than keep them around and continue confusing people forever. > > > > Would you resubmit with adjustments for comments made? > > What comments do you want me to adjust? > > Do you mean I should add a paragraph is the documentation about > that now there is only one install target. I updated the documentation > on both patch submission, but if more is required I'll do what I ask, > just be more specific. > > The v2 version is the one which has a single and simple "install" target > everywhere. Ah, I see v2 now. It was a little farther down in my mailbox. It is in the patch queue now. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073