Thread: Visual Studio 2005, C-language function - avoiding hacks?
<div class="Section1"><p class="MsoNormal">I have PostgreSQL 8.4 installed on Windows XP, and am using Visual Studio 2005to write a C-Language function. I have the most basic hello-world type example (just the ‘add_one’ function from <a href="http://www.postgresql.org/docs/8.4/interactive/xfunc-c.html">http://www.postgresql.org/docs/8.4/interactive/xfunc-c.html</a>) ina DLL, set to compile to C code rather than C++, and I can do CREATE FUNCTION on it then use it successfully. However,I had to do some questionable hacks to get it to build, so I’m guessing I’ve done something the wrong way. I’ve notfound anything categorical in the archives here, so would very much appreciate anyone who can tell me if there’s a keystep I’m missing that would make the hacks unnecessary. (For instance, I’ve come across a requirement in previous versionsto replace pg_config.h with pg_config.h.win32, but that doesn’t seem to apply to current versions – maybe somethingelse does.) I’m concerned the hacks might turn round and bite me otherwise once I’m implementing something non-trivial.<pclass="MsoNormal"> <p class="MsoNormal">Having added C:\Program Files\PostgreSQL\8.4\include\server to theinclude directories to pick up postgres.h and fmgr.h, the compiler complained about various missing include files, startingwith ‘libintl.h’. Having read the post at <a href="http://archives.postgresql.org/pgsql-general/2009-03/msg00332.php">http://archives.postgresql.org/pgsql-general/2009-03/msg00332.php</a> Icreated an empty libint.h in an include dir, along with a bunch of other empty dummy files that were expected: netdb.h,pwd.h, netinet/in.h and arpa/inet.h. <p class="MsoNormal"> <p class="MsoNormal">The code then compiles ok, but gives‘inconsistent dll linkage’ on the line with PG_FUNCTION_INFO_V1 and the one with PG_MODULE_MAGIC.<p class="MsoNormal"> <pclass="MsoNormal">So I’d like to ask:<p class="MsoNormal">* Is there some symbol I should be defining,or something, to avoid all those dummy header files and make the corresponding Visual Studio headers get pickedup?<p class="MsoNormal">* Is there something I should be doing to avoid the ‘inconsistent dll linkage’ warnings?<pclass="MsoNormal"> <p class="MsoNormal">(The C-Language functions are going to end up using some in-proc Windows-onlycomponents, so I’m hoping that building them using Visual Studio will be the least painful way to get at those.)<pclass="MsoNormal"> <p class="MsoNormal">Thanks in advance for any help<p class="MsoNormal"> <p class="MsoNormal">Kevin.<pclass="MsoNormal"> <p class="MsoNormal"> <p class="MsoNormal"> </div>
Kevin Flanagan wrote: > the compiler > complained about various missing include files, starting with > ‘libintl.h’. Having read the post at > http://archives.postgresql.org/pgsql-general/2009-03/msg00332.php I > created an empty libint.h in an include dir NFI why Pg for win32 doesn't bundle a copy of the libintl it was built against. I should poke the EDB guys about it, actually. > along with a bunch of other > empty dummy files that were expected: netdb.h, pwd.h, netinet/in.h and > arpa/inet.h. Those I wouldn't expect to be included if you're building for win32. Are you sure you're building with the win32 configuration? > The code then compiles ok, but gives ‘inconsistent dll linkage’ on the > line with PG_FUNCTION_INFO_V1 and the one with PG_MODULE_MAGIC. This would suggest that the macros that insert appropriate __declspec(dllimport) and __declspec(dllexport) attributes aren't being triggered - so again, it makes me wonder if Pg knows it's building for win32. -- Craig Ringer
On Fri, Mar 5, 2010 at 4:02 AM, Craig Ringer <craig@postnewspapers.com.au> wrote: > Kevin Flanagan wrote: > >> the compiler >> complained about various missing include files, starting with >> ‘libintl.h’. Having read the post at >> http://archives.postgresql.org/pgsql-general/2009-03/msg00332.php I >> created an empty libint.h in an include dir > > NFI why Pg for win32 doesn't bundle a copy of the libintl it was built > against. I should poke the EDB guys about it, actually. We do include the library. We don't include the headers or source for third party code though - that would be considered part of the build environment, just the same as the Windows SDK. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com PG East Conference: http://www.enterprisedb.com/community/nav-pg-east-2010.do
Dave Page wrote: > On Fri, Mar 5, 2010 at 4:02 AM, Craig Ringer > <craig@postnewspapers.com.au> wrote: >> Kevin Flanagan wrote: >> >>> the compiler >>> complained about various missing include files, starting with >>> ‘libintl.h’. Having read the post at >>> http://archives.postgresql.org/pgsql-general/2009-03/msg00332.php I >>> created an empty libint.h in an include dir >> NFI why Pg for win32 doesn't bundle a copy of the libintl it was built >> against. I should poke the EDB guys about it, actually. > > We do include the library. We don't include the headers or source for > third party code though - that would be considered part of the build > environment, just the same as the Windows SDK. The installer includes the headers for PostgreSQL its self (ie a "postgresql sdk"). For them to be useful for much you need libintl too. As on windows there's no "system" libintl, the user has to try to figure out what libintl Pg was built against and somehow obtain appropriate headers, including any config headers. There are many different win32 builds of libintl out there and many (most) of them won't match what Pg was built against. That seems unnecessarily painful. It seems pretty harmless to ship the gettext headers, as a separate download if not in the main installer package. How do _you_ go about building server extensions for Pg? Where do you get the headers for gettext etc? IMO if the gettext headers aren't in the main installer pkg then the Pg headers shouldn't be either, and a separate "sdk" installer should be provided with them both. Ditto any other headers required. I'm increasingly thinking the win32 package _should_ be split into server binary and separate headers+pdb+sources packages, with the sdk package including gettext headers and sources too. It'd be a LOT easier to develop with Pg on win32 this way. -- Craig Ringer
On Fri, Mar 5, 2010 at 9:05 AM, Craig Ringer <craig@postnewspapers.com.au> wrote: > How do _you_ go about building server extensions for Pg? Where do you > get the headers for gettext etc? Same place I get the binaries - gnuwin32 mostly. > I'm increasingly thinking the win32 package _should_ be split into > server binary and separate headers+pdb+sources packages, with the sdk > package including gettext headers and sources too. It'd be a LOT easier > to develop with Pg on win32 this way. How does breaking it up into multiple packages make it easier? -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com PG East Conference: http://www.enterprisedb.com/community/nav-pg-east-2010.do
Dave Page wrote: > On Fri, Mar 5, 2010 at 9:05 AM, Craig Ringer > <craig@postnewspapers.com.au> wrote: > >> How do _you_ go about building server extensions for Pg? Where do you >> get the headers for gettext etc? > > Same place I get the binaries - gnuwin32 mostly. > >> I'm increasingly thinking the win32 package _should_ be split into >> server binary and separate headers+pdb+sources packages, with the sdk >> package including gettext headers and sources too. It'd be a LOT easier >> to develop with Pg on win32 this way. > > How does breaking it up into multiple packages make it easier? What I was trying to say was "if you don't want to include gettext in the main download, perhaps splitting all the dev files into a separate package would permit you to add gettext and the rest". I don't much like the fact that presently users have to go hunting for the libraries, with not even a pointer included in the sources about where they should look to find headers matching the shipped libraries, and what version they need. Why _not_ distribute gettext headers, though? Sources I can understand for size reasons, but the headers are small and fuss free, and you need the _right_ _versions_ to build against the Pg backend. -- Craig Ringer
On Fri, Mar 5, 2010 at 9:31 AM, Craig Ringer <craig@postnewspapers.com.au> wrote: > Why _not_ distribute gettext headers, though? Sources I can understand > for size reasons, but the headers are small and fuss free, and you need > the _right_ _versions_ to build against the Pg backend. No reason, other than I didn't realise they were needed to build extension. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com PG East Conference: http://www.enterprisedb.com/community/nav-pg-east-2010.do
Dave Page wrote: > On Fri, Mar 5, 2010 at 9:31 AM, Craig Ringer > <craig@postnewspapers.com.au> wrote: > >> Why _not_ distribute gettext headers, though? Sources I can understand >> for size reasons, but the headers are small and fuss free, and you need >> the _right_ _versions_ to build against the Pg backend. > > No reason, other than I didn't realise they were needed to build extension. > Ah, fair enough. I read: > We do include the library. We don't include the headers or source for > third party code though - that would be considered part of the build > environment, just the same as the Windows SDK. as "we don't want to distribute third-party headers even if required by Pg's own headers" and thus thought you *did* know but by policy didn't want to distribute them. -- Craig Ringer
Ok, re "building with the win32 configuration" ... that sounds like just the thing I should know about. All I've done isdownloaded and installed the 1-click installer for Windows from http://www.enterprisedb.com/products/pgdownload.do#windows... so while I'm sure it knows it's running on Win32, is theresome other configuration change I should make for dev purposes to indicate that it's "the win32 configuration"? Or does"building with the win32 configuration" refer to those who are building the server from source, or something? Thanks Kevin -----Original Message----- From: Craig Ringer [mailto:craig@postnewspapers.com.au] Sent: 05 March 2010 04:02 To: Kevin Flanagan Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Visual Studio 2005, C-language function - avoiding hacks? Kevin Flanagan wrote: > the compiler > complained about various missing include files, starting with > ‘libintl.h’. Having read the post at > http://archives.postgresql.org/pgsql-general/2009-03/msg00332.php I > created an empty libint.h in an include dir NFI why Pg for win32 doesn't bundle a copy of the libintl it was built against. I should poke the EDB guys about it, actually. > along with a bunch of other > empty dummy files that were expected: netdb.h, pwd.h, netinet/in.h and > arpa/inet.h. Those I wouldn't expect to be included if you're building for win32. Are you sure you're building with the win32 configuration? > The code then compiles ok, but gives ‘inconsistent dll linkage’ on the > line with PG_FUNCTION_INFO_V1 and the one with PG_MODULE_MAGIC. This would suggest that the macros that insert appropriate __declspec(dllimport) and __declspec(dllexport) attributes aren't being triggered - so again, it makes me wonder if Pg knows it's building for win32. -- Craig Ringer
Kevin Flanagan wrote: > Ok, re "building with the win32 configuration" ... that sounds like just the thing I should know about. All I've done isdownloaded and installed the 1-click installer for Windows from http://www.enterprisedb.com/products/pgdownload.do#windows... so while I'm sure it knows it's running on Win32, is theresome other configuration change I should make for dev purposes to indicate that it's "the win32 configuration"? Or does"building with the win32 configuration" refer to those who are building the server from source, or something? I wasn't too specific because it's been a while since I did any coding against Pg on win32, and I couldn't remember exactly how it selected the right code to use for a given platform - whether it was a macro that must be defined, or what. Having had a look at the sources: It's done by header search path. You need to make sure that include/port/win32_msvc is on the header search path as well as the main include/ directory. I *think* port/win32 is for the MinGW win32 port and thus shouldn't be included in the search path for msvc builds, but I'm not 100% sure of that and a quick look doesn't reveal any documentation on the matter. -- Craig Ringer
On Fri, Mar 5, 2010 at 9:50 AM, Craig Ringer <craig@postnewspapers.com.au> wrote: > Dave Page wrote: >> On Fri, Mar 5, 2010 at 9:31 AM, Craig Ringer >> <craig@postnewspapers.com.au> wrote: >> >>> Why _not_ distribute gettext headers, though? Sources I can understand >>> for size reasons, but the headers are small and fuss free, and you need >>> the _right_ _versions_ to build against the Pg backend. >> >> No reason, other than I didn't realise they were needed to build extension. >> > > Ah, fair enough. I read: > >> We do include the library. We don't include the headers or source for >> third party code though - that would be considered part of the build >> environment, just the same as the Windows SDK. > > as "we don't want to distribute third-party headers even if required by > Pg's own headers" and thus thought you *did* know but by policy didn't > want to distribute them. I didn't know in this case, but was making a general statement about how I felt the policy should be. Plus I was feeling a little grumpy in my pre-coffee state. Sorry :-p -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com PG East Conference: http://www.enterprisedb.com/community/nav-pg-east-2010.do
Ok, that got me on the right track, thanks. I think the key points for this build scenario are these: 1. you have to define the symbol BUILDING_DLL in your code before including postgres.h (as that then means PGDLLIMPORT getsdefined right in pg_config_os.h). That makes the 'inconsistent dll linkage' warnings go away. 2. you have to have include\server\port\win32 in the include dirs list as well as include\server (as that provides a bunchof otherwise-missing headers such as netdb.h) However, that still leaves one missing include file - libintl.h, which c.h tries to include because ENABLE_NLS is defined(and that seems to get defined as 1 in pg_config.h whether you like it or not). And in fact, it seems Rostic Sheykhetposted at http://www.postgresql.org/docs/8.2/interactive/xfunc-c.html with the same problem (now that I know howto do it, I know what to Google for to, er, find out how to do it :) ). You can get round it by commenting out the includeor creating a dummy libintl.h. Just posting the results here in case they're relevant for anything. Kevin. -----Original Message----- From: Craig Ringer [mailto:craig@postnewspapers.com.au] Sent: 05 March 2010 10:05 To: Kevin Flanagan Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Visual Studio 2005, C-language function - avoiding hacks? Kevin Flanagan wrote: > Ok, re "building with the win32 configuration" ... that sounds like just the thing I should know about. All I've done isdownloaded and installed the 1-click installer for Windows from http://www.enterprisedb.com/products/pgdownload.do#windows... so while I'm sure it knows it's running on Win32, is theresome other configuration change I should make for dev purposes to indicate that it's "the win32 configuration"? Or does"building with the win32 configuration" refer to those who are building the server from source, or something? I wasn't too specific because it's been a while since I did any coding against Pg on win32, and I couldn't remember exactly how it selected the right code to use for a given platform - whether it was a macro that must be defined, or what. Having had a look at the sources: It's done by header search path. You need to make sure that include/port/win32_msvc is on the header search path as well as the main include/ directory. I *think* port/win32 is for the MinGW win32 port and thus shouldn't be included in the search path for msvc builds, but I'm not 100% sure of that and a quick look doesn't reveal any documentation on the matter. -- Craig Ringer
"Kevin Flanagan" <kevin-f@linkprior.com> wrote: > 1. you have to define the symbol BUILDING_DLL in your code before > including postgres.h No, BUILDING_DLL does not work. We use PGDLLIMPORT both exported global variables and PG_MODULE_MAGIC/PG_FUNCTION_INFO_V1 for now, but actually we should always use __declspec (dllexport) for the latter cases. They are exported from *user dlls*, not the postgres' one. I'd like to propose to define PGALWAYSEXPORT macro: #ifdef WIN32 #define PGALWAYSEXPORT __declspec (dllexport) #endif and modify PG_MODULE_MAGIC and PG_FUNCTION_INFO_V1 to use it instead of PGDLLEXPORT. Regards, --- Takahiro Itagaki NTT Open Source Software Center
Takahiro Itagaki <itagaki.takahiro@oss.ntt.co.jp> writes:
> I'd like to propose to define PGALWAYSEXPORT macro:
> #ifdef WIN32
> #define PGALWAYSEXPORT __declspec (dllexport)
> #endif
> and modify PG_MODULE_MAGIC and PG_FUNCTION_INFO_V1 to use it
> instead of PGDLLEXPORT.
This seems like change for the sake of change. The existing mechanism
works (as demonstrated by the fact that the contrib modules work on
Windows). I don't think we should invent a new mechanism that won't be
backwards-compatible.
regards, tom lane
Tom Lane <tgl@sss.pgh.pa.us> wrote: > Takahiro Itagaki <itagaki.takahiro@oss.ntt.co.jp> writes: > > I'd like to propose to define PGALWAYSEXPORT macro: > > #ifdef WIN32 > > #define PGALWAYSEXPORT __declspec (dllexport) > > #endif > > and modify PG_MODULE_MAGIC and PG_FUNCTION_INFO_V1 to use it > > instead of PGDLLEXPORT. > > This seems like change for the sake of change. The existing mechanism > works (as demonstrated by the fact that the contrib modules work on > Windows). I wonder why the contrib modules can be compiled correctly because: 1. PG_MODULE_MAGIC requires dllexport. 2. Other exportedvariables from postgres requires dllimport. 3. Exported functions from the contrib DLLs require dllexport, but they don't have any PGDLLEXPORT tags in their functions. Did we use non-standard tools except msvc in the build frameword for core code? And what should I do for an external project? Regards, --- Takahiro Itagaki NTT Open Source Software Center
2010/3/8 Takahiro Itagaki <itagaki.takahiro@oss.ntt.co.jp>: > > Tom Lane <tgl@sss.pgh.pa.us> wrote: > >> Takahiro Itagaki <itagaki.takahiro@oss.ntt.co.jp> writes: >> > I'd like to propose to define PGALWAYSEXPORT macro: >> > #ifdef WIN32 >> > #define PGALWAYSEXPORT __declspec (dllexport) >> > #endif >> > and modify PG_MODULE_MAGIC and PG_FUNCTION_INFO_V1 to use it >> > instead of PGDLLEXPORT. >> >> This seems like change for the sake of change. The existing mechanism >> works (as demonstrated by the fact that the contrib modules work on >> Windows). > > I wonder why the contrib modules can be compiled correctly because: > 1. PG_MODULE_MAGIC requires dllexport. > 2. Other exported variables from postgres requires dllimport. > 3. Exported functions from the contrib DLLs require dllexport, > but they don't have any PGDLLEXPORT tags in their functions. > > Did we use non-standard tools except msvc in the build frameword > for core code? And what should I do for an external project? Yes, we use mingw. In this particular case, it may be the non-standard behavior that mingw exports *all* symbols in a DLL. We have some scripts in the MSVC build system that does this - it auto-generates a .DEF file that lists all symbols inthe file, and makes sure those are all exported. In fact, this even requires us to remove warnings created by modern versions of Visual Studio, since you're not supposed to use both dllexport and DEF files for the same symbol, but we do. -- Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/
Magnus Hagander <magnus@hagander.net> wrote: > >> The existing mechanism > >> works (as demonstrated by the fact that the contrib modules work on > >> Windows). > > > > Did we use non-standard tools except msvc in the build framework > > for core code? And what should I do for an external project? > > Yes, we use mingw. Hmmm, it means the existing mechanism can only work on limited environments. Should we make the code to be more portable for standard developers? Third party developer might not build the postgres server, but they would want to build their extension modules without mingw. Regards, --- Takahiro Itagaki NTT Open Source Software Center
2010/3/9 Takahiro Itagaki <itagaki.takahiro@oss.ntt.co.jp>: > > Magnus Hagander <magnus@hagander.net> wrote: > >> >> The existing mechanism >> >> works (as demonstrated by the fact that the contrib modules work on >> >> Windows). >> > >> > Did we use non-standard tools except msvc in the build framework >> > for core code? And what should I do for an external project? >> >> Yes, we use mingw. > > Hmmm, it means the existing mechanism can only work on limited environments. > Should we make the code to be more portable for standard developers? That would definitely be good. > Third party developer might not build the postgres server, > but they would want to build their extension modules without mingw. Well, the tool for generating the full export of symbols is in the build tree, so perhaps we jus tneed to ship that. But usually you *don't* want that because, well, it's non-standard. And exporting all symbols makes the binary *much* larger (iirc, postgres.exe grows about 40%). It would perhaps be better to encourage (and document) how people create their own .DEF files to export the specific symbols they need. Because AFAIK, that will work if you do that - or do you know of issues with that method? -- Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/