Thread: Building Windows Server Extensions Using VC++ 2005
Hmm...this didn't seem to make it to the newsgroup. Apologies if its a repeat. Charlie ------------- Hi everyone, I've been able to successfully build server extension using Visual Studio 2005 for Windows Postgresql 8.1. However, it took a few tweaks which I thought I should document (maybe these issues could be fixed in future postgresql versions?): 1. There is no lib file for VC++ to link against. This can be created easily enough by going to src/backend directory and running: lib /MACHINE:x86 /NAME:postgres.exe /DEF:postgres.def Note the use of /NAME to tell VC++ it is linking against an executable and not a dll. It would be nice if this lib file was automatically installed on windows when installing postgresql. 2. Requirement on strings.h In c.h:69 there is this code: #ifdef HAVE_STRINGS_H #include <strings.h> #endif In pg_config.h:405 this is defined: /* Define to 1 if you have the <strings.h> header file. */ #define HAVE_STRINGS_H 1 However, Visual Studio 2005 does not include this file. For a workaround I simply added it but that's a bit of hack. 3. This is a bigger issue, and took a while to figure out. If you try to use the Version 1 calling convention, your function will be called but if you try to do anything with the passed in parameters a segmentation fault will occur. If you use the Version 0 calling convention things work fine. The problem is if you use PG_FUNCTION_INFO_V1 postgres does not see the generated function because it is not exported from the dll and thus assumes the Version 0 calling convention when in fact your function is using Version1. The problem is in fmgr.h:298 #define PG_FUNCTION_INFO_V1(funcname) \ extern Pg_finfo_record * CppConcat(pg_finfo_,funcname) (void); \ ... For windows to export this function it must be: extern __declspec(dllexport) Pg_finfo_record * CppConcat(pg_finfo_,funcname) (void); Would it be possible to add a DLLEXPORT macro here to fix this? 4. Last, and the biggest issue, if my function calls pfree it blows up. What is happening is that pfree somehow ends uppointing to a random spot in memory - thus when you try to call it you execute invalid code (in fact you never execute pfree at all as far as I can see). I worked around this by using pgport_pfree which does work. Haven't a clue why... Here is the assembly for successfully calling pgport_pfree: pgport_pfree(fileName); 100112D3 mov eax,dword ptr [ebp-0Ch] 100112D6 push eax 100112D7 call 100110C3 100112DC add esp,4 100110C3 jmp 1001131A pgport_pfree: 1001131A jmp dword ptr ds:[10016288h] 005CF140 push ebp 005CF141 mov ebp,esp 005CF143 sub esp,8 005CF146 mov eax,dword ptr [ebp+8] 005CF149 mov dword ptr [esp+4],eax 005CF14D mov eax,dword ptr ds:[006A9F94h] 005CF152 mov dword ptr [esp],eax 005CF155 call 005CF0D0 005CF15A leave 005CF15B ret And here is pfree. Note at the end the code tries to execute "db" at 005E1560 causing a segmentation fault. pfree( fileName ); 100112D3 mov eax,dword ptr [ebp-0Ch] 100112D6 push eax 100112D7 call 1001110E 100112DC add esp,4 1001110E jmp 10011238 pfree: 10011238 jmp dword ptr ds:[1001628Ch] 005E1560 db ffh Hope this helps others. It would be great if building postgresql server extensions with VC++ worked out of the box on Windows (in addition of course to using MingW) since I think it would open up a wider audience. Charlie
"Charlie Savage" <cfis@interserv.com> wrote > > I've been able to successfully build server extension using Visual > Studio 2005 for Windows Postgresql 8.1. However, it took a few tweaks > which I thought I should document (maybe these issues could be fixed in > future postgresql versions?): > > Is this one related? http://pgfoundry.org/projects/vcproject/ Regards, Qingqing
Not really. This is for building VC++ extensions that work with Postgresql for Windows, which is built using MinGW. Thus it allows you to build extensions with VC++ that work with Postgresql "out-of-the-box" on the Windows platform. In contrast, the link below is for rebuilding Postgresql itself, but with VC++ instead of MinGW. Charlie Qingqing Zhou wrote: > "Charlie Savage" <cfis@interserv.com> wrote >> I've been able to successfully build server extension using Visual >> Studio 2005 for Windows Postgresql 8.1. However, it took a few tweaks >> which I thought I should document (maybe these issues could be fixed in >> future postgresql versions?): >> >> > > Is this one related? > > http://pgfoundry.org/projects/vcproject/ > > Regards, > Qingqing > >
> > In pg_config.h:405 this is defined: > > > > /* Define to 1 if you have the <strings.h> header file. */ #define > > HAVE_STRINGS_H 1 > > > > However, Visual Studio 2005 does not include this file. For a > > workaround I simply added it but that's a bit of hack. > > Ah, so even though you are using MSVC, you are using the > config file from MinGW. Easy fix, included in the patch > below, and applied for 8.2. Um, is WIN32_CLIENT_ONLY really defined when you're building a backend extension? That seems wrong. The whole problem seems to be the using of the pg_config.h from mingw when building with msvc. Why not just use the correct pg_config.h in the first place? What happens then? IIRC, there were other problems building extensions with MSVC - such as struct alignment and stuff. Not sure if that has been taken care of? If not, there really isn't much point in fixing the headers alone :-) //Magnus
Magnus Hagander wrote: > > > In pg_config.h:405 this is defined: > > > > > > /* Define to 1 if you have the <strings.h> header file. */ #define > > > HAVE_STRINGS_H 1 > > > > > > However, Visual Studio 2005 does not include this file. For a > > > workaround I simply added it but that's a bit of hack. > > > > Ah, so even though you are using MSVC, you are using the > > config file from MinGW. Easy fix, included in the patch > > below, and applied for 8.2. > > Um, is WIN32_CLIENT_ONLY really defined when you're building a backend > extension? That seems wrong. Well, it is defined: #if defined(_MSC_VER) || defined(__BORLANDC__)#define WIN32_CLIENT_ONLY#endif > The whole problem seems to be the using of the pg_config.h from mingw > when building with msvc. Why not just use the correct pg_config.h in the > first place? What happens then? We can't generate a pg_config.h for WIN32_CLIENT_ONLY because there is no shell build environment. > IIRC, there were other problems building extensions with MSVC - such as > struct alignment and stuff. Not sure if that has been taken care of? If > not, there really isn't much point in fixing the headers alone :-) Well, I figure alignment would be based on the CPU, not the compiler, but I might be wrong. I added the capability, and why it was added, so if we decide the idea is a failure, we know what to remove. -- Bruce Momjian http://candle.pha.pa.us SRA OSS, Inc. http://www.sraoss.com + If your life is a hard drive, Christ can be your backup. +
> > Um, is WIN32_CLIENT_ONLY really defined when you're > building a backend > > extension? That seems wrong. > > Well, it is defined: > > #if defined(_MSC_VER) || defined(__BORLANDC__) > #define WIN32_CLIENT_ONLY > #endif Oops. I obviously forgot all about that :-) > > The whole problem seems to be the using of the pg_config.h > from mingw > > when building with msvc. Why not just use the correct > pg_config.h in > > the first place? What happens then? > > We can't generate a pg_config.h for WIN32_CLIENT_ONLY because > there is no shell build environment. Right. But we have include/pg_config.h.win32 which is supposed to be used in this case. The win32 native build of the backend will copy that file into pg_config.h. The problem here is that the backend is built with mingw but the extension with msvc. If you'd start from fresh, there would be no pg_config.h to include. The pg_config.h.win32 file is supposed to be preset for the MSVC environment (which is the same as borland on win32, but different from mingw). > > IIRC, there were other problems building extensions with > MSVC - such > > as struct alignment and stuff. Not sure if that has been taken care > > of? If not, there really isn't much point in fixing the > headers alone > > :-) > > Well, I figure alignment would be based on the CPU, not the > compiler, but I might be wrong. Um. struct packing? I don't remember exactly what it was, but IIRC it was something. > I added the capability, and why it was added, so if we decide > the idea is a failure, we know what to remove. Right. I've never tried using msvc for a server extension, and the general answer has always been "don't do it". But it may well have changed now. //Magnus
"Magnus Hagander" <mha@sollentuna.net> writes:
> The problem here is that the backend is built with mingw but the
> extension with msvc.
I don't think that it's very reasonable to expect that to work.
The two compilers very likely have different rules for struct
packing, to take just the first gotcha. Maybe they are compatible
enough that it will work, but if it does not we cannot fix it.
regards, tom lane
Tom Lane <tgl@sss.pgh.pa.us> writes: > "Magnus Hagander" <mha@sollentuna.net> writes: > > The problem here is that the backend is built with mingw but the > > extension with msvc. > > I don't think that it's very reasonable to expect that to work. > The two compilers very likely have different rules for struct > packing, to take just the first gotcha. Maybe they are compatible > enough that it will work, but if it does not we cannot fix it. Well that sort of stuff is supposed to be covered by the ABI. Consider that if it didn't then you wouldn't be able to use any of the standard libraries without recompiling them for each compiler since a number of standard library APIs depend on structs like timeval or stat. I'm not saying there aren't risks, but in theory it's supposed to work. -- greg
Greg Stark <gsstark@mit.edu> writes:
> Well that sort of stuff is supposed to be covered by the ABI. Consider that if
> it didn't then you wouldn't be able to use any of the standard libraries
> without recompiling them for each compiler since a number of standard library
> APIs depend on structs like timeval or stat.
Maybe. My experience with these sorts of problems is fairly old (hard
lessons from libjpeg development in the mid-90s), but what I learned at
the time is that all the standard Unix library interfaces are carefully
designed to be fairly impervious to struct padding rules (eg, all the
fields are "int", or at least all the same size), and so you can get
apparent interoperability between compilers that don't agree on these
rules. Meanwhile, Microsoft, who never met a feature they didn't like,
invented a foot-gun called #pragma pack, and then had to fix all their
headers to nail down the pack setting. That in turn masks compatibility
issues on that platform, but only if you're willing to #pragma-ify all
of your own headers too.
In short, I don't trust anybody on this issue. I have the scars to
remind me why not.
regards, tom lane
Hi everyone,<br /><br /> Thanks for the feedback. The reason for building extensions with MSVC on windows is to get
accessto the Micrsoft debugging tools since as far as I can see MingW/GDB cannot debug dynamically loaded dlls on the
Windowsplatform (or at least I haven't succeeded at doing it).<br /><br /> Anyway, with the few minor tweaks mentioned
thisseems to work fine. I guess I would be surprised if it didn't since MinGW is commonly used for both creating and
usingDLLs on the Windows platform that integrate with programs and dlls built with VC++. Other examples that work fine
arebuilding extensions for Python and Ruby using MingW, when both runtimes are built with VC++ (so the opposite case).
Ofcourse, this compatibility might just be limited to the MinGW / VC++ combination, but that's all you need for this to
work.<br/><br /> Charlie<br /><br /> Tom Lane wrote: <blockquote cite="mid21532.1141490933@sss.pgh.pa.us"
type="cite"><prewrap="">"Magnus Hagander" <a class="moz-txt-link-rfc2396E"
href="mailto:mha@sollentuna.net"><mha@sollentuna.net></a>writes: </pre><blockquote type="cite"><pre wrap="">The
problemhere is that the backend is built with mingw but the
extension with msvc. </pre></blockquote><pre wrap="">
I don't think that it's very reasonable to expect that to work.
The two compilers very likely have different rules for struct
packing, to take just the first gotcha. Maybe they are compatible
enough that it will work, but if it does not we cannot fix it.
regards, tom lane
</pre></blockquote>
Charlie Savage wrote: > Hi everyone, > > Thanks for the feedback. The reason for building extensions with MSVC > on windows is to get access to the Micrsoft debugging tools since as far > as I can see MingW/GDB cannot debug dynamically loaded dlls on the > Windows platform (or at least I haven't succeeded at doing it). > > Anyway, with the few minor tweaks mentioned this seems to work fine. I > guess I would be surprised if it didn't since MinGW is commonly used for > both creating and using DLLs on the Windows platform that integrate with > programs and dlls built with VC++. Other examples that work fine are > building extensions for Python and Ruby using MingW, when both runtimes > are built with VC++ (so the opposite case). Of course, this > compatibility might just be limited to the MinGW / VC++ combination, but > that's all you need for this to work. OK, I improved the STRINGS_H macro check and added DLLIMPORT in case Win32 users don't export everything. Added comments. This is all 8.2 stuff. -- Bruce Momjian http://candle.pha.pa.us SRA OSS, Inc. http://www.sraoss.com + If your life is a hard drive, Christ can be your backup. + Index: src/include/c.h =================================================================== RCS file: /cvsroot/pgsql/src/include/c.h,v retrieving revision 1.197 diff -c -c -r1.197 c.h *** src/include/c.h 3 Mar 2006 21:35:46 -0000 1.197 --- src/include/c.h 5 Mar 2006 04:39:27 -0000 *************** *** 57,62 **** --- 57,64 ---- #else #if defined(_MSC_VER) || defined(__BORLANDC__) #define WIN32_CLIENT_ONLY + /* Some use MinGW-generated pg_config.h but MSVC for extensions. */ + #undef HAVE_STRINGS_H #endif #endif #include "postgres_ext.h" *************** *** 66,73 **** #include <string.h> #include <stddef.h> #include <stdarg.h> ! /* Some use MinGW-generated pg_config.h but MSVC for extensions. */ ! #if defined(HAVE_STRINGS_H) && !defined(WIN32_CLIENT_ONLY) #include <strings.h> #endif #include <sys/types.h> --- 68,74 ---- #include <string.h> #include <stddef.h> #include <stdarg.h> ! #ifdef HAVE_STRINGS_H #include <strings.h> #endif #include <sys/types.h> Index: src/include/fmgr.h =================================================================== RCS file: /cvsroot/pgsql/src/include/fmgr.h,v retrieving revision 1.40 diff -c -c -r1.40 fmgr.h *** src/include/fmgr.h 15 Oct 2005 02:49:41 -0000 1.40 --- src/include/fmgr.h 5 Mar 2006 04:39:28 -0000 *************** *** 293,302 **** /* Expected signature of an info function */ typedef Pg_finfo_record *(*PGFInfoFunction) (void); ! /* Macro to build an info function associated with the given function name */ ! #define PG_FUNCTION_INFO_V1(funcname) \ ! extern Pg_finfo_record * CppConcat(pg_finfo_,funcname) (void); \ Pg_finfo_record * \ CppConcat(pg_finfo_,funcname) (void) \ { \ --- 293,305 ---- /* Expected signature of an info function */ typedef Pg_finfo_record *(*PGFInfoFunction) (void); ! /* ! * Macro to build an info function associated with the given function name. ! * Win32 loadable functions usually link with 'dlltool --export-all', but it ! * doesn't hurt to add DLLIMPORT in case they don't. ! */ #define PG_FUNCTION_INFO_V1(funcname) \ ! extern DLLIMPORT Pg_finfo_record * CppConcat(pg_finfo_,funcname) (void); \ Pg_finfo_record * \ CppConcat(pg_finfo_,funcname) (void) \ { \