Thread: Re: [GENERAL] SPI & file locations

Re: [GENERAL] SPI & file locations

From
Lamar Owen
Date:
On Fri, 26 May 2000, Tom Lane wrote:
> Lamar Owen <lamar.owen@wgcr.org> writes:
> > /lib/cpp -M -I. -I../backend executor/spi.h |xargs -n 1|grep \\W|grep -v ^/|grep -v spi.h | sort |cpio -pdu
$RPM_BUILD_ROOT/usr/include/pgsql
> > This could easily enough be included in the make install, couldn't it?
> > (Tom?  Anyone?)  I realize that GNU grepisms (or is \W common?) are
> > used above,
> That's just the tip of the iceberg of the portability problems in the
> above.  If you want to propose that we actually do something like that
> during 'make install', you're gonna have to work a lot harder.

:-)  I expect no less (than to work harder, of course....).  Why not do
something like that, other than the 'out-of-sight, out-of-mind' problem?  Or,
to put it far more bluntly, the SPI installation is very broken if the SPI
developer has to go around manually installing header files that make install
should have automatically taken care of.

> (However, as a non-portable trick for getting a list of files that need
> to be included in a hand-generated makefile, it's not bad.)

I take that as high praise :-).  No, really it IS a quick hack -- I got tired
of generating the list by hand, so I automated it.  And, when the list shrinks
down to nearly nothing (10-15 includes, maybe?), I won't have to change a thing.

> The more serious problem is "what else might an SPI module need besides
> spi.h".

What else indeed.  Isn't spi.h the exported SPI itself?  We seem to have a very
poorly defined line between exported and private interfaces -- back to my point
about the assumption that the whole source tree is available for any little
program's whim (such as the regression test suite, which was _interesting_ to
get working without a bunch of the source tree around.)  Mike Mascari may be
able to answer more about what SPI programs require, as he is doing SPI
development.

>  Also, it disturbs me a lot that spi.h pulls in 80+ include
> files in the first place --- there's got to be stuff in there that
> needn't/shouldn't be exported.

Such as lztext.h?

> rather than fall back on automation that will let the list bloat even
> more without anyone much noticing...

Part of that would be to think more like a packager or user, rather than a
developer.  That was the first thing I learned when tackling the RPM's, was,
that I was a packager -- while that does involve knowledge of the guts of the
package, it also requires a mindset very different from development -- more
user-oriented, I suppose.

A packager for RedHat will start thinking about what can be partitioned out,
saving the user's HD space.  A packager will start thinking about how the
package _can_ be partitioned, and logical partitioning (for instance, the
absolutely worst possible packaging for PostgreSQL in RPM form would be a
single RPM with everything in it -- thus, requiring a database server to have a
full-bore X11 installation for pgaccess (which requires tk and tcl, as well),
when that database server may have absolutely no need for pgaccess.)).  I am
now even considering splitting out pltcl into a separate package -- as pltcl
implicitly requires the server package, it makes the whole tcl package require
the server package -- and someone may have legitimate need for a postgresql
client machine _without_ the server AND need the tcl client.  Maybe a
pgaccess-based administration client, perhaps?). 

Those who build from the source have the configure options to eliminate (in
their opinion) the cruft they don't need -- RPM users don't have that option --
so, I build everything, and split the distribution.

Some issues I have seen along these lines are:
1.)     The header situation -- assumes a source tree somewhere so that you
can pull in things....
2.)    The regression tests -- you know, someone MIGHT want to _run_ those
tests on a database server in a production situation where there is no make, no
compiler, and no PostgreSQL source tree (where config.guess resides).  The
installation can and should prebuild all necessary binaries and should
preconfigure the results of a config.guess -- and the shell script should not
need to be invoked by make.  (Yes, the RPMset does all this (except eliminate
the need for config.guess), and it's packaged as the postgresql-test rpm, which
is an optional 1.5MB package -- I  wimped out and packaged the whole test
subtree....)

While I am all for source availability and the whole open source (free
software) model, I am also for easily installed and upgraded packages for users
who simply want to use the package in a production environment.  I've been
there and done that -- and was more than a little frustrated at the state of
the RPM packaging.  Thus my current situation :-).

--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11



Re: [GENERAL] SPI & file locations

From
Tom Lane
Date:
Lamar Owen <lamar.owen@wgcr.org> writes:
> :-) I expect no less (than to work harder, of course....).  Why not do
> something like that, other than the 'out-of-sight, out-of-mind'
> problem?  Or, to put it far more bluntly, the SPI installation is very
> broken if the SPI developer has to go around manually installing
> header files that make install should have automatically taken care
> of.

Agreed, but I'm worried about the 'out-of-sight, out-of-mind' aspect.

>> The more serious problem is "what else might an SPI module need besides
>> spi.h".

> What else indeed.  Isn't spi.h the exported SPI itself?  We seem to
> have a very poorly defined line between exported and private
> interfaces

Ah-hah, I think you and I are on exactly the same wavelength there.

My whole problem with the spi.h-imports-88-headers business is that it
exposes in gory detail the fact that *we have absolutely no idea* what
we consider an exported backend interface and what we don't.  I don't
like the idea of an automatic tool that exports whatever it thinks might
be needed, because if we let ourselves go down that path we will very
soon find ourselves trying to preserve backwards compatibility with
something that should never have been exported at all.

Basically I feel that this is a problem that requires some actual
thought and design judgment.  I don't want to see us substituting
a one-liner script hack for design judgment.  OTOH, I don't really
want to do the legwork myself (lame excuse: never having written
an SPI module myself, I have little idea what they need to see).
I'm just concerned about the long-term consequences of taking the
easy way out.
        regards, tom lane


Re: [GENERAL] SPI & file locations

From
Lamar Owen
Date:
On Sat, 27 May 2000, Tom Lane wrote:
> Lamar Owen <lamar.owen@wgcr.org> writes:
> > What else indeed.  Isn't spi.h the exported SPI itself?  We seem to
> > have a very poorly defined line between exported and private
> > interfaces
> Ah-hah, I think you and I are on exactly the same wavelength there.
> My whole problem with the spi.h-imports-88-headers business is that it
> exposes in gory detail the fact that *we have absolutely no idea* what
> we consider an exported backend interface and what we don't.  I don't

Yes, this is a problem.  And, yes, it needs to be taken care of in a designed,
methodical manner -- the spi.h header should only need a few things (I, like
you, have not done any SPI development yet to see what really _is_ needed....).

> Basically I feel that this is a problem that requires some actual
> thought and design judgment. 

Yes, most certainly.  Can someone who has actual SPI experience look at this?

> I'm just concerned about the long-term consequences of taking the
> easy way out.

I'll have to admit to taking the easy way out a little in  my RPM solution --
but, it's only there so that an advertised development interface can be
actually used.  Once a thorough look is taken at the whole header mess for SPI,
I still won't have to change anything, which is good both for me and for RPM
users, as I really don't keep track of every header file dependency change --
thus, RPM releases won't happen (again, as 7.0-1 was in error in this regard)
with broken header deps, requiring a bugfix package.

If no one else is forthcoming with a solution to this problem, I'll take a look
at it (possibly during the development cycle for 7.1 after the 7.0.x series has
stabilized, when the RPM cycle is at idle).

--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11


Re: Re: [GENERAL] SPI & file locations

From
Peter Vazsonyi
Date:
Hmmm...

I'm attached a small patch. This patch includes the list of dirs and
headers refered by spi.h (And _not_ more)

On Sat, 27 May 2000, Lamar Owen wrote:

> On Sat, 27 May 2000, Tom Lane wrote:
> > My whole problem with the spi.h-imports-88-headers business is that it
> > exposes in gory detail the fact that *we have absolutely no idea* what
> > we consider an exported backend interface and what we don't.  I don't

Yes, but _before_ somebody create the new (more structural?) header layout,
must all spi-developer search for this files?

> Yes, this is a problem.  And, yes, it needs to be taken care of in a
> designed, methodical manner -- the spi.h header should only need a few
> things (I, like you, have not done any SPI development yet to see what
> really _is_ needed....).


--nek;(