Thread: Trimming the Fat, Part Deux ...
Okay ... since this is pretty much going to be 'one camp for, one camp against' without anything to really back up either camps perspectives / arguments, I did some research on CVS in order to find a nice, effective middle ground ... and it actually works quite sweet ... Basically, CVS let's you "merge" modules into one mega-module, which is what I've just done ... I pulled out two sub-directories from the pgsql CVS repository ... contrib/earthdistance and src/interfaces/libpqxx ... if you do: cvs checkout -P libpqxx you'll see *just* the libpqxx code ... same with doing a checkout of 'earthdistance' ... If you checkout pgsql, it will get the source tree *minus* libpqxx and earthdistance ... And, finally, if you want it all: cvs checkout -P pgsql-all Now, for those that are going to yell out about already checked out code ... don't, I've already tested that too ... From within your checked out code, remove the above two directories, then from the top level (ie. if you have your src in src/pgsql, go to the src directory, *not* the root of the source tree itself) and type: cvs checkout -P contrib interfaces and you will be 'in sync' ... Next thing I'm going to do is modify the scripts that build the .tar.gz packages so that they now build a libpqxx.tar.gz and earthdistance.tar.gz distribution, so that they can be downloaded seperately, as well as pulling out the README files so that ppl know what they are useful for in the first place ... For those that like to work on the 'mega source tree', nothing has changed ... you check out pgsql-all, and commits will commit to the right places in the repository ... If ppl want to download everything, including the kitchen sink, then they will have that option ... but if ppl want to just download what they need, that option is now open to them as well ... Now comes the 'tricky part, which I'm hoping someone like Peter might know how to do ... I know there is a way of writing configure to 'run configure' in sub projects .. for instance, with libpqxx ... at the top level of pgsql, one could type: ./configure --enable-libpqxx and it would run the appropriate configure in the libpqxx subdirectory and add libpqxx to the 'targets' in src/interfaces/Makefile ... If we can get *that* done, removing something like libpq++ would be a simple matter of removing the option from configure.in at the top level, and removing its inclusion in the pgsql-all meta on the CVS repository ... nice and clean ... I've only done libpqxx and earthdistance right now ... none of the history is lost, but I want to get the packaging issue worked out using those two first, before I dive into the others ... also, want to make sure that I haven't overlooked anything in my testing ... *Eventually*, a simple checkout of 'pgsql' should result in a "server only" distribution that we can pull bits and pieces into transparently ... the really cool thing is that using PHP, I could probably come up with a nice simple interface that ppl could create a custom download bundle: "I want the base server + jdbc + the earth distance module" and it would package that up and present it to them to download ... then everyone can package whatever they want and download it ...
> Okay ... since this is pretty much going to be 'one camp for, one camp > against' without anything to really back up either camps perspectives / > arguments, I did some research on CVS in order to find a nice, effective > middle ground ... and it actually works quite sweet ... Personally, I'd like to be able to have a "client-only" distro, but other than that I'm not fussed. I like the convenience of having all the contribs in contrib, however some sort of CPgAN thing would be sweet as. Currently, the guy who maintains the FreeBSD postgres port has hacked up a client only version that does all the building, but only installs the client stuff. Chris
On Thu, 1 Aug 2002, Christopher Kings-Lynne wrote: > > Okay ... since this is pretty much going to be 'one camp for, one camp > > against' without anything to really back up either camps perspectives / > > arguments, I did some research on CVS in order to find a nice, effective > > middle ground ... and it actually works quite sweet ... > > Personally, I'd like to be able to have a "client-only" distro, but other > than that I'm not fussed. I like the convenience of having all the contribs > in contrib, however some sort of CPgAN thing would be sweet as. > > Currently, the guy who maintains the FreeBSD postgres port has hacked up a > client only version that does all the building, but only installs the client > stuff. Ya, which is what the MySQL port does also ... but you still have to download the complete distro ... the other thing to consider is that we have how many mirrors right now? That are paying for the bandwidth to mirror the software and provide it to the community? If 100 ppl that just need, for instance, the libpq libraries could download *just* the libpq.tar.gz file and install it, how much bandwidth does that save overall? Bruce figured that the libpq.tar.gz stuff he did for me saved ~1/10 the space of the whole distro ... so if the distro is ~8Meg, that means that the libpq is ~800k ... so we're talking about a saving of, what, ~720Meg of bandwidth? It doesn't sound like alot when you aren't paying for it, but it definitely adds up quickly to those that are ...
...
> *Eventually*, a simple checkout of 'pgsql' should result in a "server
> only" distribution that we can pull bits and pieces into transparently ...
I'm still not quite sure where this is headed or why, but if nothing
else pgsql could and should be the whole thing, and pgsql-server could
be the server subset. Server-only makes sense for only some users, and
nothing apparently makes sense for everyone. But we may as well have the
old names do the old things...
- Thomas
On Wed, 31 Jul 2002, Thomas Lockhart wrote: > ... > > *Eventually*, a simple checkout of 'pgsql' should result in a "server > > only" distribution that we can pull bits and pieces into transparently ... > > I'm still not quite sure where this is headed or why, but if nothing > else pgsql could and should be the whole thing, and pgsql-server could > be the server subset. Server-only makes sense for only some users, and > nothing apparently makes sense for everyone. But we may as well have the > old names do the old things... will make those changes in the morning ...
On Wednesday 31 July 2002 09:38 pm, Marc G. Fournier wrote: > Okay ... since this is pretty much going to be 'one camp for, one camp > against' without anything to really back up either camps perspectives / > arguments, I did some research on CVS in order to find a nice, effective > middle ground ... and it actually works quite sweet ... MAn, did I ever pick a bad day to be offline for a whole day. :-) If anyone cares to look, or for that matter cares at all, something similar is already being done in the binary RPMs. I have split, sliced, and diced this distribution for over three years. So, let me share some of the experiences learned from that exercise. > Now comes the 'tricky part, which I'm hoping someone like Peter might know > how to do ... I know there is a way of writing configure to 'run > configure' in sub projects .. for instance, with libpqxx ... at the top > level of pgsql, one could type: > ./configure --enable-libpqxx I like this idea, up to an extent. I guess it boils down to this: 1.) What is the minimum build environment necessary to build anything in the source distribution; 2.) What degree of granularity is desired; 3.) We must not assume the presence of a full source tree to build _anything_, only the minimum build system, which can then handle everything else as a module. Then I could much more easily package a 'postgresql-devel' package that would allow, for instance, PostGIS to be built as a server module without having to have the entire source distribution tree in place (which is currently required). As PostGIS is a separately developed and distributed module, it seems reasonable that it should be buildable without the full source tree in place. As to getting rid of bloat, I'm all for splitting out contrib modules, client libraries, and the like into separate projects. If the build system is a unit, and is not dependent upon the server or libpq source code, then it is easier and more consistent to just require it to be in place to build ANY contrib or client module. We're too big and too spread out. It was said (by Andrew) that we're hard to install -- why is that? Is it possible that it is _because_ of the number of pieces we include? And the sooner our very old perl client goes away, the better I like it. It is a good client, don't get me wrong: but DBD:Pg is the standard now. But, if you are an RPM user, you can already just download the pieces for a minimal client-side system. And you have been able to do so for right at three years, give or take. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
> And the sooner our very old perl client goes away, the better I > like it. It > is a good client, don't get me wrong: but DBD:Pg is the standard now. > This may sound like a dumb question, but DBD::Pg == DBI right ? not pg.pm I code perl about 25 hours a week, and DBI has never failed me yet. Jeff.
On Thursday 01 August 2002 12:05 pm, Jeff MacDonald wrote: > > And the sooner our very old perl client goes away, the better I > > like it. It > > is a good client, don't get me wrong: but DBD:Pg is the standard now. > This may sound like a dumb question, but DBD::Pg == DBI right ? not pg.pm Right. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Lamar Owen wrote: > And the sooner our very old perl client goes away, the better I like it. It > is a good client, don't get me wrong: but DBD:Pg is the standard now. I have been in contact with Edmund about moving DBD into our CVS and updating CPAN ourselves. Should have a final answer soon. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Bruce Momjian wrote: > Lamar Owen wrote: > > And the sooner our very old perl client goes away, the better I like it. It > > is a good client, don't get me wrong: but DBD:Pg is the standard now. > > I have been in contact with Edmund about moving DBD into our CVS and > updating CPAN ourselves. Should have a final answer soon. OK, I got the go-ahead from Edmund. We will have DBD:pg in the 7.3 tarball. I will add it to CVS today or tomorrow. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
On Thu, 1 Aug 2002, Bruce Momjian wrote: > Bruce Momjian wrote: > > Lamar Owen wrote: > > > And the sooner our very old perl client goes away, the better I like it. It > > > is a good client, don't get me wrong: but DBD:Pg is the standard now. > > > > I have been in contact with Edmund about moving DBD into our CVS and > > updating CPAN ourselves. Should have a final answer soon. > > OK, I got the go-ahead from Edmund. We will have DBD:pg in the 7.3 > tarball. I will add it to CVS today or tomorrow. Hmm, according README from DBD-Pg ( 1.13 ), it's maintained now by Jeffrey W. Baker (jwbaker@acm.org) DBD::Pg -- a PostgreSQL interface for Perl 5. $Id: README,v 1.3 2002/04/10 02:01:38 jwb Exp $ Copyright (c) 1997,1998,1999,2000 Edmund Mergl Copyright (c) 2002 Jeffrey W. Baker Portions Copyright (c) 1994,1995,1996,1997Tim Bunce I'm a little bit aware about DBD::Pg because it needs many changes and testing to be 7.3 compliant. Jeffrey, wake up :-) > > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
Oleg Bartunov wrote: > On Thu, 1 Aug 2002, Bruce Momjian wrote: > > > Bruce Momjian wrote: > > > Lamar Owen wrote: > > > > And the sooner our very old perl client goes away, the better I like it. It > > > > is a good client, don't get me wrong: but DBD:Pg is the standard now. > > > > > > I have been in contact with Edmund about moving DBD into our CVS and > > > updating CPAN ourselves. Should have a final answer soon. > > > > OK, I got the go-ahead from Edmund. We will have DBD:pg in the 7.3 > > tarball. I will add it to CVS today or tomorrow. > > Hmm, > > according README from DBD-Pg ( 1.13 ), it's maintained now by > Jeffrey W. Baker (jwbaker@acm.org) > > DBD::Pg -- a PostgreSQL interface for Perl 5. > > $Id: README,v 1.3 2002/04/10 02:01:38 jwb Exp $ > > Copyright (c) 1997,1998,1999,2000 Edmund Mergl > Copyright (c) 2002 Jeffrey W. Baker > Portions Copyright (c) 1994,1995,1996,1997 Tim Bunce > > I'm a little bit aware about DBD::Pg because it needs many changes and > testing to be 7.3 compliant. > > Jeffrey, wake up :-) Oh, that's strange. I wonder why Edmund didn't mention that. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
On Thu, 1 Aug 2002, Bruce Momjian wrote: > Bruce Momjian wrote: > > Lamar Owen wrote: > > > And the sooner our very old perl client goes away, the better I like it. It > > > is a good client, don't get me wrong: but DBD:Pg is the standard now. > > > > I have been in contact with Edmund about moving DBD into our CVS and > > updating CPAN ourselves. Should have a final answer soon. > > OK, I got the go-ahead from Edmund. We will have DBD:pg in the 7.3 > tarball. I will add it to CVS today or tomorrow. If you know how to, please import it as a seperate module, do not add it to the pgsql tree itself ... if you don't know how, please just forward it onto me and I'll get it into the repository ...
Ummm ... stupid question, but can we even bring this into the 'core'? You may distribute under the terms of either the GNU General Public License or the Artistic License, as specified in thePerl README file. On Thu, 1 Aug 2002, Oleg Bartunov wrote: > On Thu, 1 Aug 2002, Bruce Momjian wrote: > > > Bruce Momjian wrote: > > > Lamar Owen wrote: > > > > And the sooner our very old perl client goes away, the better I like it. It > > > > is a good client, don't get me wrong: but DBD:Pg is the standard now. > > > > > > I have been in contact with Edmund about moving DBD into our CVS and > > > updating CPAN ourselves. Should have a final answer soon. > > > > OK, I got the go-ahead from Edmund. We will have DBD:pg in the 7.3 > > tarball. I will add it to CVS today or tomorrow. > > Hmm, > > according README from DBD-Pg ( 1.13 ), it's maintained now by > Jeffrey W. Baker (jwbaker@acm.org) > > DBD::Pg -- a PostgreSQL interface for Perl 5. > > $Id: README,v 1.3 2002/04/10 02:01:38 jwb Exp $ > > Copyright (c) 1997,1998,1999,2000 Edmund Mergl > Copyright (c) 2002 Jeffrey W. Baker > Portions Copyright (c) 1994,1995,1996,1997 Tim Bunce > > I'm a little bit aware about DBD::Pg because it needs many changes and > testing to be 7.3 compliant. > > Jeffrey, wake up :-) > > > > > > > > Regards, > Oleg > _____________________________________________________________ > Oleg Bartunov, sci.researcher, hostmaster of AstroNet, > Sternberg Astronomical Institute, Moscow University (Russia) > Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ > phone: +007(095)939-16-83, +007(095)939-23-83 > >
Marc G. Fournier wrote: > > Ummm ... stupid question, but can we even bring this into the 'core'? > > You may distribute under the terms of either the GNU General Public > License or the Artistic License, as specified in the Perl README file. Artistic License is fine, I think. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
On Thursday 01 August 2002 02:21 pm, Bruce Momjian wrote: > Bruce Momjian wrote: > > Lamar Owen wrote: > > > And the sooner our very old perl client goes away, the better I like > > > it. It is a good client, don't get me wrong: but DBD:Pg is the > > > standard now. > > I have been in contact with Edmund about moving DBD into our CVS and > > updating CPAN ourselves. Should have a final answer soon. > OK, I got the go-ahead from Edmund. We will have DBD:pg in the 7.3 > tarball. I will add it to CVS today or tomorrow. Um, is putting it into our tarball necessary, or even desireable? It is separately maintained as part of CPAN. Is there some fundamental reason that we _must_ ship a perl client (the same goes for tcl/tk/python/c++ as well) if it is adequately maintained in the standard location? Perlers know to go to CPAN. Likewise Pythonistas have their own place. And that's the crux of Marc's message, I believe -- why can't we minimize here? -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Bruce Momjian writes: > Artistic License is fine, I think. The Artistic License doesn't even qualify as Free Software as far as the FSF is concerned. More generally, it is a different license, and that is a problem. -- Peter Eisentraut peter_e@gmx.net
Bruce Momjian writes: > OK, I got the go-ahead from Edmund. We will have DBD:pg in the 7.3 > tarball. I will add it to CVS today or tomorrow. Pleeeeease, no more Perl modules in our CVS! The ones we have are already messy enough to build. I thought we were talking about trimming the source tree, not adding more. Why not put it on gborg or somewhere else? -- Peter Eisentraut peter_e@gmx.net
On Thursday 01 August 2002 04:37 pm, Peter Eisentraut wrote: > I thought we were talking about trimming the source tree, not adding more. > Why not put it on gborg or somewhere else? It's already in CPAN. A link to CPAN should suffice, IMHO. I also thought we were discussing trimming the tree; and that was a good feeling. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
On Thu, 1 Aug 2002, Lamar Owen wrote: > On Thursday 01 August 2002 04:37 pm, Peter Eisentraut wrote: > > I thought we were talking about trimming the source tree, not adding more. > > Why not put it on gborg or somewhere else? > > It's already in CPAN. A link to CPAN should suffice, IMHO. > > I also thought we were discussing trimming the tree; and that was a good > feeling. Oh thank you to both you and Peter .... I was feeling sooooo alone out here ...
Peter Eisentraut wrote: > Bruce Momjian writes: > > > Artistic License is fine, I think. > > The Artistic License doesn't even qualify as Free Software as far as the > FSF is concerned. > > More generally, it is a different license, and that is a problem. Well, our ODBC is LGPL. I wonder if Edmund can/would change the license. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Lamar Owen wrote: > On Thursday 01 August 2002 04:37 pm, Peter Eisentraut wrote: > > I thought we were talking about trimming the source tree, not adding more. > > Why not put it on gborg or somewhere else? > > It's already in CPAN. A link to CPAN should suffice, IMHO. > > I also thought we were discussing trimming the tree; and that was a good > feeling. Lamar, you said earlier today: > And the sooner our very old perl client goes away, the better I like it. It > is a good client, don't get me wrong: but DBD:Pg is the standard now. So I assumed you wanted DBD:Pg. DBD:Pg is a good example of an interface that hasn't advanced a quickly as it would have had it been in our CVS tree. I have received a number of bug reports for it, and have them in my mailbox. I have no idea if they made it into the CPAN version. Moving interfaces out can be a problem too unless there is a large enough group to grow it. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
On Thursday 01 August 2002 05:22 pm, Bruce Momjian wrote: > Lamar Owen wrote: > > It's already in CPAN. A link to CPAN should suffice, IMHO. > > I also thought we were discussing trimming the tree; and that was a good > > feeling. > Lamar, you said earlier today: > > And the sooner our very old perl client goes away, the better I like it. > > It is a good client, don't get me wrong: but DBD:Pg is the standard now. > So I assumed you wanted DBD:Pg. I'm sorry I gave that impression; I was advocating removing the old Pg module in favor of people using the DBD::Pg module, which is distributed separately in CPAN. I wasn't advocating bringing DBD::Pg into our distribution; my apologies for giving the wrong impression. DBD::Pg is typically distributed separately even in things such as Red Hat Linux, where it lives as a separate RPM. There's also qt-PostgreSQL, php-pgsql, mod_auth_pgsql, and others that are doing quite OK outside our CVS. The OpenACS/AOLserver PostgreSQL driver is a good example of a client outside our CVS that is being very well maintained by its group. We should be providing the core client library, the backend, and documentation. Contrib modules (earthdistance, etc), other clients, and things that don't fit in the core should be separately tarballed -- not necessarily separately CVS'd -- the AOLserver CVS, for instance, has a number of modules, all of which are somewhat independent. Those modules then need to be buildable with a set of headers and makefile includes ONLY. Without assuming any paths. > DBD:Pg is a good example of an > interface that hasn't advanced a quickly as it would have had it been in > our CVS tree. I have received a number of bug reports for it, and have > them in my mailbox. I have no idea if they made it into the CPAN > version. Moving interfaces out can be a problem too unless there is a > large enough group to grow it. Even if it's in a CVS at postgresql.org it doesn't necessarily need to be in the main tarball. Even stuff in our CVS can languish -- witness the pgaccess revival outside the CVS tree. The main tarball needs dramatic splitting into independent pieces, with a build framework that can deal with the pieces. If I want a perl client, the backend, and PostGIS, I should be able to download the build system, the perl client, the backend, and PostGIS and make it work. And each module shouldn't require the source of the other modules to build -- just the necessary bits in headers. If I want just the python client, I should be able to download the client-side development headers, configure, and makefile, then download the python client, and build it. And if I already have headers installed as part of a RPM or from source, I shouldn't need config.guess, configure, makefile.global, or anything but a set of C headers to get the python client to build. IOW the python client should be somewhat independent. It CAN be done -- the AOLServer/OpenACS PostgreSQL driver does it -- all it needs is the path to the headers and to libpq and it's off to the races. I guess I'm saying that we're too big and too popular to include the kitchen sink in the tarball anymore. We need to think more modularly -- and not assume a source tree with those modules. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Someone said earlier cvsup would have problems but the anonymous cvs would work fine. Well I've just had a weirdness reconfiguring and rebuilding my few weeks old 7.3dev tree and so deleted it and tried using the anoncvs to get pgsql. Running configure gives me the error: ./configure: ./src/template/linux: No such file or directory and all ./src contains is: total 44 drwxr-xr-x 2 software software 4096 Aug 1 23:27 CVS -rw-r--r-- 1 software software 119 Jul 30 1999 DEVELOPERS -rw-r--r-- 1 software software 1039 Jul 30 18:47 Makefile -rw-r--r-- 1 software software 13288 Jul 27 21:10 Makefile.global.in -rw-r--r-- 1 software software 10853 Jul 27 21:10 Makefile.shlib drwxr-xr-x 23 software software 4096 Aug 1 23:27 backend -- Nigel J. Andrews Director --- Logictree Systems Limited Computer Consultants
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Marc G. Fournier wrote:
>> Ummm ... stupid question, but can we even bring this into the 'core'?
>>
>> You may distribute under the terms of either the GNU General Public
>> License or the Artistic License, as specified in the Perl README file.
> Artistic License is fine, I think.
The Artistic License is cool, but it's not BSD. Are we going to drop
our recent agreement to try to move the distribution to all-BSD terms
so easily?
regards, tom lane
On Fri, 2 Aug 2002, Tom Lane wrote: > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > Marc G. Fournier wrote: > >> Ummm ... stupid question, but can we even bring this into the 'core'? > >> > >> You may distribute under the terms of either the GNU General Public > >> License or the Artistic License, as specified in the Perl README file. > > > Artistic License is fine, I think. > > The Artistic License is cool, but it's not BSD. Are we going to drop > our recent agreement to try to move the distribution to all-BSD terms > so easily? that's kinda what I'm wondering ... but, as someone else already asked also, is there a reason why we are even bringing this into the distirbution, vs, say, moving it to GBorg? I do like the ability for us to modify the code itself, but haven't put it into the repository yet since at least one person (or was it two?) questioned whether it should even be in there ... Personally, I'd love to put it into GBorg and have someone assigned as a maintainer for any patch submissions, or someone that wants to actively work on it ...
Peter Eisentraut <peter_e@gmx.net> writes: > The Artistic License doesn't even qualify as Free Software as far as the > FSF is concerned. Errr, http://www.fsf.org/licenses/license-list.html lists it as not only Free Software but GPL-compatible. Mike.