Thread: writing backend extensions using Visual Studio
Greetings, I am trying to port a library of backend functions that I've built and tested on linux to the windows version of postgresql (v8.0.1). I installed pgSQL on the windows box (win2k) using the binary installer a while ago and the base installation works fine, so long as I don't attempt to use my backend functions. Because the extension functions use a bunch of in-house libraries that are built with Visual Studio, I really need an approach that works with MSVC. After playing around for a while, I managed to get a DLL to build and work at least far enough that I can CREATE and call the functions from psql. However, it looks like every argument I pass into the functions comes in as null (e.g. PG_ARGISNULL returns true). This clearly is less than helpful. :-) Can anyone point me to documentation or example files for how to set up a project for building extensions functions for postgres on windows? Thanks for any help, -greg
Greg Landrum <greg.landrum@gmail.com> writes:
> After playing around for a while, I managed to get a DLL to build and
> work at least far enough that I can CREATE and call the functions from
> psql. However, it looks like every argument I pass into the functions
> comes in as null (e.g. PG_ARGISNULL returns true). This clearly is
> less than helpful. :-)
Sounds like MSVC is interpreting the layout of the FunctionCallInfo
struct differently than the MinGW compiler did. You should probably
look into whether there are "pragma pack" directives lurking in your
program. ("pragma pack" is evil...)
regards, tom lane
On Fri, 25 Mar 2005 17:57:07 -0500, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Greg Landrum <greg.landrum@gmail.com> writes:
> > After playing around for a while, I managed to get a DLL to build and
> > work at least far enough that I can CREATE and call the functions from
> > psql. However, it looks like every argument I pass into the functions
> > comes in as null (e.g. PG_ARGISNULL returns true). This clearly is
> > less than helpful. :-)
>
> Sounds like MSVC is interpreting the layout of the FunctionCallInfo
> struct differently than the MinGW compiler did. You should probably
> look into whether there are "pragma pack" directives lurking in your
> program. ("pragma pack" is evil...)
I've got the library stripped down to the bare minimum (just a couple
of demo functions) and I can't seem to find any uses of pragma pack,
so I don't think that's it.
-greg
Greg Landrum wrote:
>On Fri, 25 Mar 2005 17:57:07 -0500, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
>
>>Greg Landrum <greg.landrum@gmail.com> writes:
>>
>>
>>>After playing around for a while, I managed to get a DLL to build and
>>>work at least far enough that I can CREATE and call the functions from
>>>psql. However, it looks like every argument I pass into the functions
>>>comes in as null (e.g. PG_ARGISNULL returns true). This clearly is
>>>less than helpful. :-)
>>>
>>>
>>Sounds like MSVC is interpreting the layout of the FunctionCallInfo
>>struct differently than the MinGW compiler did. You should probably
>>look into whether there are "pragma pack" directives lurking in your
>>program. ("pragma pack" is evil...)
>>
>>
>
>I've got the library stripped down to the bare minimum (just a couple
>of demo functions) and I can't seem to find any uses of pragma pack,
>so I don't think that's it.
>
>
>
>
Do we actually support building backend extensions with anything other
than our standard build environment? I know we support building libpq
using MSVC, but that is for client use. I'm not even mildly surprised
that building a backend extension with MSVC breaks.
cheers
andrew
On Sat, 26 Mar 2005 11:25:27 -0500, Andrew Dunstan <andrew@dunslane.net> wrote: > > Do we actually support building backend extensions with anything other > than our standard build environment? I know we support building libpq > using MSVC, but that is for client use. I'm not even mildly surprised > that building a backend extension with MSVC breaks. This goes a long way towards answering my question. So given that I want to work with backend extensions on windows, I need to pull down a copy of mingw and use that? -greg
Greg Landrum wrote: > On Sat, 26 Mar 2005 11:25:27 -0500, Andrew Dunstan <andrew@dunslane.net> wrote: > > > > Do we actually support building backend extensions with anything other > > than our standard build environment? I know we support building libpq > > using MSVC, but that is for client use. I'm not even mildly surprised > > that building a backend extension with MSVC breaks. > > This goes a long way towards answering my question. > > So given that I want to work with backend extensions on windows, I > need to pull down a copy of mingw and use that? Yes, read the MinGW FAQ that explains the things you need. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Andrew Dunstan <andrew@dunslane.net> writes:
> Do we actually support building backend extensions with anything other
> than our standard build environment? I know we support building libpq
> using MSVC, but that is for client use. I'm not even mildly surprised
> that building a backend extension with MSVC breaks.
I'm not either, but given that he can get as far as building a library
that will load into the backend, it's surprising that it would fall down
on the small matter of passing function arguments. It seems worth
trying to understand exactly what's happening there.
Greg, how sure are you about the diagnosis of "PG_ARGISNULL always
returns true" --- have you stepped through your code with a debugger?
regards, tom lane
On Sat, 26 Mar 2005 13:55:06 -0500, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> I'm not either, but given that he can get as far as building a library
> that will load into the backend, it's surprising that it would fall down
> on the small matter of passing function arguments. It seems worth
> trying to understand exactly what's happening there.
>
> Greg, how sure are you about the diagnosis of "PG_ARGISNULL always
> returns true" --- have you stepped through your code with a debugger?
Nope, I haven't gone that far. I just had the code that was causing
the crash (from accessing bad memory) inside:
if(!PG_ARGISNULL(0)) {
}
and I'm sure that the function isn't actually being called with a null
argument (for one thing, it's declared 'strict').
-greg
When I switched to using mingw to compile the library containing my extension functions, everything works fine. This wasn't a trivial undertaking, and I'd certainly prefer to be able to use MSVC to build the libraries, but I now have a working extension library and that's the most important thing at the moment. For the sake of anyone following later: If you're using postgresql 8.0 under windows and you want to build extension libraries, it looks like you need to use mingw to build the libraries. You'll also need to build a local copy of postgresql itself in order to get the appropriate libraries and header files; it seems that the development files distributed with the win32 installer aren't sufficient to the task. Thanks to everyone for the help, -greg
> -----Original Message----- > From: pgsql-hackers-win32-owner@postgresql.org > [mailto:pgsql-hackers-win32-owner@postgresql.org] On Behalf > Of Greg Landrum > Sent: 28 March 2005 20:46 > To: pgsql-hackers-win32@postgresql.org > Subject: Re: [pgsql-hackers-win32] writing backend extensions > using Visual Studio > > it seems that the development > files distributed with the win32 installer aren't sufficient to the > task. They should be - what did we miss? Regards, Dave.
On Mon, 28 Mar 2005 22:05:58 +0100, Dave Page <dpage@vale-housing.co.uk> wrote: > > > > > it seems that the development > > files distributed with the win32 installer aren't sufficient to the > > task. > > They should be - what did we miss? Looking at a fresh installation, the libpostgres.a distributed with pginstaller seems to match the one I built, but the header files required for building backend apps are missing. (These are the "internal to Postgres" headers, to use the terminology of postgres_ext.h) -greg
> -----Original Message----- > From: Greg Landrum [mailto:greg.landrum@gmail.com] > Sent: 28 March 2005 23:31 > To: Dave Page; pgsql-hackers-win32@postgresql.org > Subject: Re: [pgsql-hackers-win32] writing backend extensions > using Visual Studio > > On Mon, 28 Mar 2005 22:05:58 +0100, Dave Page > <dpage@vale-housing.co.uk> wrote: > > > > > > > > it seems that the development > > > files distributed with the win32 installer aren't > sufficient to the > > > task. > > > > They should be - what did we miss? > > Looking at a fresh installation, the libpostgres.a distributed with > pginstaller seems to match the one I built, but the header files > required for building backend apps are missing. (These are the > "internal to Postgres" headers, to use the terminology of > postgres_ext.h) We install the following: $ ls c:/program\ files/postgresql/8.0/include ecpg_informix.h libpq-fe.h pgtypes_date.h pgtypes_timestamp.h ecpgerrno.h pg_config.h pgtypes_error.h postgres_ext.h ecpglib.h pg_config_manual.h pgtypes_interval.h sql3types.h ecpgtype.h pg_config_os.h pgtypes_numeric.h sqlca.h Which is what is installed by PostgreSQL's build system. Do backend extensions need to be built in the source tree? Regards, Dave
Dave Page said: > > >> -----Original Message----- >> From: Greg Landrum [mailto:greg.landrum@gmail.com] >> Sent: 28 March 2005 23:31 >> To: Dave Page; pgsql-hackers-win32@postgresql.org >> Subject: Re: [pgsql-hackers-win32] writing backend extensions >> using Visual Studio >> >> On Mon, 28 Mar 2005 22:05:58 +0100, Dave Page >> <dpage@vale-housing.co.uk> wrote: >> > >> > > >> > > it seems that the development >> > > files distributed with the win32 installer aren't >> sufficient to the >> > > task. >> > >> > They should be - what did we miss? >> >> Looking at a fresh installation, the libpostgres.a distributed with >> pginstaller seems to match the one I built, but the header files >> required for building backend apps are missing. (These are the >> "internal to Postgres" headers, to use the terminology of >> postgres_ext.h) > > We install the following: > > $ ls c:/program\ files/postgresql/8.0/include > ecpg_informix.h libpq-fe.h pgtypes_date.h > pgtypes_timestamp.h > ecpgerrno.h pg_config.h pgtypes_error.h postgres_ext.h > ecpglib.h pg_config_manual.h pgtypes_interval.h sql3types.h > ecpgtype.h pg_config_os.h pgtypes_numeric.h sqlca.h > > Which is what is installed by PostgreSQL's build system. Do backend > extensions need to be built in the source tree? > Don't you need the (fairly large) tree that is normally rooted at include/postgresql/server? This will contain things like funcapi.h and fmgr.h. cheers andrew
> -----Original Message----- > From: Andrew Dunstan [mailto:andrew@dunslane.net] > Sent: 29 March 2005 12:34 > To: Dave Page > Cc: greg.landrum@gmail.com; pgsql-hackers-win32@postgresql.org > Subject: Re: [pgsql-hackers-win32] writing backend extensions > using Visual Studio > > > Don't you need the (fairly large) tree that is normally rooted at > include/postgresql/server? This will contain things like funcapi.h and > fmgr.h. Yeah - these used to be installed by 'make install-all-headers' iirc, however that has gone now. I can't see anything in /INSTALL or the docs on what to use instead other than the source tree (which doesn't ship with pgInstaller of course). So, a question to anyone who knows, do we really need to ship *all* headers with the installer, or is there a useful subset? Regards, Dave
On Tue, 29 Mar 2005 13:32:46 +0100, Dave Page <dpage@vale-housing.co.uk> wrote: > > Yeah - these used to be installed by 'make install-all-headers' iirc, > however that has gone now. I can't see anything in /INSTALL or the docs > on what to use instead other than the source tree (which doesn't ship > with pgInstaller of course). > > So, a question to anyone who knows, do we really need to ship *all* > headers with the installer, or is there a useful subset? I'm too new to this whole area to provide anything like an authoritative answer, but I can at least speak to what headers I'm using: $PGSRC/include/postgres.h // should be first file included in backend modules $PGSRC/include/fmgr.h // required to define functions using version 1 calling conventions $PGSRC/include/libpq/pqformat.h /// required to work with byteas Clearly some of these #include a bunch of other files. This selection is based on the postgresql documentation and some web surfing. Maybe a good solution to this is to add an extra option in the installer's development category to include header files for backend development? That way most people (even those who do front-end development) don't have to deal with having that whole include tree installed, but those who want to do backend work can still get everything required without doing their own build/source installation. -greg
> -----Original Message----- > From: Greg Landrum [mailto:greg.landrum@gmail.com] > Sent: 29 March 2005 15:52 > To: Dave Page; pgsql-hackers-win32@postgresql.org > Subject: Re: [pgsql-hackers-win32] writing backend extensions > using Visual Studio > > Maybe a good solution to this is to add an extra option in the > installer's development category to include header files for backend > development? That way most people (even those who do front-end > development) don't have to deal with having that whole include tree > installed, but those who want to do backend work can still get > everything required without doing their own build/source installation. Yeah, that's quite straightforward to do - although, there are that many files that it would probably be easier to do a wildcard include anyway unless it's a fairly well defined set that's required. In reality I doubt most people would care whats in the include/ directory anyway - it's the download size I'm more worried about! Regards, Dave.
> > Maybe a good solution to this is to add an extra option in the > > installer's development category to include header files > for backend > > development? That way most people (even those who do front-end > > development) don't have to deal with having that whole include tree > > installed, but those who want to do backend work can still get > > everything required without doing their own build/source > installation. > > Yeah, that's quite straightforward to do - although, there > are that many files that it would probably be easier to do a > wildcard include anyway unless it's a fairly well defined set > that's required. In reality I doubt most people would care > whats in the include/ directory anyway - it's the download > size I'm more worried about! The full postgresql headers (from src/include) compresses to around 500Kb, so it's not that big a deal. But I don't think we need *all* those, so it would definitly be better to have a subset if possible. I still think a separate feature for iti s definitly the way to go. //Magnus
On Tue, 29 Mar 2005 15:54:54 +0100, Dave Page <dpage@vale-housing.co.uk> wrote: > > > > -----Original Message----- > > From: Greg Landrum [mailto:greg.landrum@gmail.com] > > Sent: 29 March 2005 15:52 > > To: Dave Page; pgsql-hackers-win32@postgresql.org > > Subject: Re: [pgsql-hackers-win32] writing backend extensions > > using Visual Studio > > > > Maybe a good solution to this is to add an extra option in the > > installer's development category to include header files for backend > > development? That way most people (even those who do front-end > > development) don't have to deal with having that whole include tree > > installed, but those who want to do backend work can still get > > everything required without doing their own build/source installation. > > Yeah, that's quite straightforward to do - although, there are that many > files that it would probably be easier to do a wildcard include anyway > unless it's a fairly well defined set that's required. In reality I > doubt most people would care whats in the include/ directory anyway - > it's the download size I'm more worried about! For what it's worth, if I zip up the $PGSRC/include directory, the archive is about 500K. That seems like a pretty minimal bump to the 17MB installer. -greg
> -----Original Message----- > From: Greg Landrum [mailto:greg.landrum@gmail.com] > Sent: 29 March 2005 16:05 > To: Dave Page; pgsql-hackers-win32@postgresql.org > Subject: Re: [pgsql-hackers-win32] writing backend extensions > using Visual Studio > > For what it's worth, if I zip up the $PGSRC/include directory, the > archive is about 500K. That seems like a pretty minimal bump to the > 17MB installer. Yeah, true - we just seem to keep adding 500Kb here and a megabyte there.... /D
"Dave Page" <dpage@vale-housing.co.uk> writes:
> Yeah - these used to be installed by 'make install-all-headers' iirc,
> however that has gone now. I can't see anything in /INSTALL or the docs
> on what to use instead other than the source tree (which doesn't ship
> with pgInstaller of course).
> So, a question to anyone who knows, do we really need to ship *all*
> headers with the installer, or is there a useful subset?
The reason 'make install-all-headers' went away is that we made a policy
decision to install *all* the headers. I don't think the Windows
packaging should be second-guessing that. It would be reasonable to
have an option to install *none* of the headers, for people who aren't
going to be doing any software development, but if you do install them
please don't be selective about it.
(FWIW I wasn't personally in favor of that decision, but it's a done
deal now --- having some packagings follow it and some not would be
the worst of all possible worlds.)
regards, tom lane
> -----Original Message----- > From: Tom Lane [mailto:tgl@sss.pgh.pa.us] > Sent: 29 March 2005 16:22 > To: Dave Page > Cc: Andrew Dunstan; greg.landrum@gmail.com; > pgsql-hackers-win32@postgresql.org > Subject: Re: [pgsql-hackers-win32] writing backend extensions > using Visual Studio > > > So, a question to anyone who knows, do we really need to ship *all* > > headers with the installer, or is there a useful subset? > > The reason 'make install-all-headers' went away is that we > made a policy > decision to install *all* the headers. I don't think the Windows > packaging should be second-guessing that. It would be reasonable to > have an option to install *none* of the headers, for people who aren't > going to be doing any software development, but if you do install them > please don't be selective about it. Ahh, I see the problem - the build script is only pulling in files in include/, not include/*/ etc. Will have to look at that. Thanks Tom. Ragrds, Dave.