Thread: Compiling extensions on Windows

Hi all

Out of personal interest (in pain and suffering) I was recently looking
into how to compile extensions out-of-tree on Windows using Visual
Studio (i.e. no PGXS).

It looks like the conventional answer to this is "Do a source build of
PG, compile your ext in-tree in contrib/, and hope the result is binary
compatible with release PostgreSQL builds for Windows". Certainly that's
how I've been doing it to date.

How about everyone else here? Does anyone actually build and distribute
extensions out of tree at all?

I'm interested in making the Windows installer distributions a bit more
extension dev friendly. In particular, I'd really like to see EDB's
Windows installers include the libintl.h for the included libintl, since
its omission, combined with Pg being built with ENABLE_NLS, tends to
break things horribly. Users can always undefine ENABLE_NLS, but it's an
unnecessary roadblock.

Are there any objections from -hackers to including 3rd party headers
for libs we expose in our public headers in the binary distribution?

Other than bundling 3rd party headers, any ideas/suggestions for how we
might make ext building saner on Windows?

-- Craig Ringer                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services

On 01/05/2014 10:32 PM, Craig Ringer wrote:
> Hi all
>
> Out of personal interest (in pain and suffering) I was recently looking
> into how to compile extensions out-of-tree on Windows using Visual
> Studio (i.e. no PGXS).
>
> It looks like the conventional answer to this is "Do a source build of
> PG, compile your ext in-tree in contrib/, and hope the result is binary
> compatible with release PostgreSQL builds for Windows". Certainly that's
> how I've been doing it to date.
>
> How about everyone else here? Does anyone actually build and distribute
> extensions out of tree at all?
>
> I'm interested in making the Windows installer distributions a bit more
> extension dev friendly. In particular, I'd really like to see EDB's
> Windows installers include the libintl.h for the included libintl, since
> its omission, combined with Pg being built with ENABLE_NLS, tends to
> break things horribly. Users can always undefine ENABLE_NLS, but it's an
> unnecessary roadblock.
>
> Are there any objections from -hackers to including 3rd party headers
> for libs we expose in our public headers in the binary distribution?
>
> Other than bundling 3rd party headers, any ideas/suggestions for how we
> might make ext building saner on Windows?
>

If you're bundling a DLL then I don't see why the corresponding header 
file shouldn't be included also.

cheers

andrew

On Mon, Jan 6, 2014 at 3:32 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
> Hi all
>
> Out of personal interest (in pain and suffering) I was recently looking
> into how to compile extensions out-of-tree on Windows using Visual
> Studio (i.e. no PGXS).
>
> It looks like the conventional answer to this is "Do a source build of
> PG, compile your ext in-tree in contrib/, and hope the result is binary
> compatible with release PostgreSQL builds for Windows". Certainly that's
> how I've been doing it to date.
>
> How about everyone else here? Does anyone actually build and distribute
> extensions out of tree at all?
>
> I'm interested in making the Windows installer distributions a bit more
> extension dev friendly. In particular, I'd really like to see EDB's
> Windows installers include the libintl.h for the included libintl, since
> its omission, combined with Pg being built with ENABLE_NLS, tends to
> break things horribly. Users can always undefine ENABLE_NLS, but it's an
> unnecessary roadblock.

Sandeep, can you work on fixing this please?

Thanks.

> Are there any objections from -hackers to including 3rd party headers
> for libs we expose in our public headers in the binary distribution?
>
> Other than bundling 3rd party headers, any ideas/suggestions for how we
> might make ext building saner on Windows?
>
> --
>  Craig Ringer                   http://www.2ndQuadrant.com/
>  PostgreSQL Development, 24x7 Support, Training & Services
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers



-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Jan 6, 2014 at 10:57 AM, Sandeep Thakkar
<sandeep.thakkar@enterprisedb.com> wrote:
> Sure. I'll make the changes so that the next available Windows installers
> include lbintl.h in $Installdir/include. How about the changes with respect
> to NLS?

No, there's nothing to change there. Craig was suggesting that users
could disable NLS in their extension to avoid issues with the missing
header, but that's not a good solution.

> On Mon, Jan 6, 2014 at 2:44 PM, Dave Page <dpage@pgadmin.org> wrote:
>>
>> On Mon, Jan 6, 2014 at 3:32 AM, Craig Ringer <craig@2ndquadrant.com>
>> wrote:
>> > Hi all
>> >
>> > Out of personal interest (in pain and suffering) I was recently looking
>> > into how to compile extensions out-of-tree on Windows using Visual
>> > Studio (i.e. no PGXS).
>> >
>> > It looks like the conventional answer to this is "Do a source build of
>> > PG, compile your ext in-tree in contrib/, and hope the result is binary
>> > compatible with release PostgreSQL builds for Windows". Certainly that's
>> > how I've been doing it to date.
>> >
>> > How about everyone else here? Does anyone actually build and distribute
>> > extensions out of tree at all?
>> >
>> > I'm interested in making the Windows installer distributions a bit more
>> > extension dev friendly. In particular, I'd really like to see EDB's
>> > Windows installers include the libintl.h for the included libintl, since
>> > its omission, combined with Pg being built with ENABLE_NLS, tends to
>> > break things horribly. Users can always undefine ENABLE_NLS, but it's an
>> > unnecessary roadblock.
>>
>> Sandeep, can you work on fixing this please?
>>
>> Thanks.
>>
>> > Are there any objections from -hackers to including 3rd party headers
>> > for libs we expose in our public headers in the binary distribution?
>> >
>> > Other than bundling 3rd party headers, any ideas/suggestions for how we
>> > might make ext building saner on Windows?
>> >
>> > --
>> >  Craig Ringer                   http://www.2ndQuadrant.com/
>> >  PostgreSQL Development, 24x7 Support, Training & Services
>> >
>> >
>> > --
>> > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
>> > To make changes to your subscription:
>> > http://www.postgresql.org/mailpref/pgsql-hackers
>>
>>
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>
>
>
>
> --
> Sandeep Thakkar
>



-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

<p dir="ltr">If libintl.h and any headers it in turn includes are bundled, there is no longer an issue with NLS. <p
dir="ltr">Thatwas just a workaround for building exts when Pg's headers tried to refer to nonexistent headers when NLS
wasenabled.<div class="quote">On 6 Jan 2014 18:57, Sandeep Thakkar <sandeep.thakkar@enterprisedb.com> wrote:<br
type="attribution"/><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex"><divdir="ltr">Sure. I'll make the changes so that the next available Windows installers include
lbintl.hin $Installdir/include. How about the changes with respect to NLS?<br /><div class="gmail_extra"><br /><br
/><divclass="gmail_quote"> On Mon, Jan 6, 2014 at 2:44 PM, Dave Page <span dir="ltr"><<a
href="mailto:dpage@pgadmin.org"target="_blank">dpage@pgadmin.org</a>></span> wrote:<br /><blockquote
class="gmail_quote"style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">On Mon, Jan 6,
2014at 3:32 AM, Craig Ringer <<a href="mailto:craig@2ndquadrant.com">craig@2ndquadrant.com</a>> wrote:<br /> >
Hiall<br /> ><br /> > Out of personal interest (in pain and suffering) I was recently looking<br /> > into how
tocompile extensions out-of-tree on Windows using Visual<br /> > Studio (i.e. no PGXS).<br /> ><br /> > It
lookslike the conventional answer to this is "Do a source build of<br /> > PG, compile your ext in-tree in contrib/,
andhope the result is binary<br /> > compatible with release PostgreSQL builds for Windows". Certainly that's<br />
>how I've been doing it to date.<br /> ><br /> > How about everyone else here? Does anyone actually build and
distribute<br/> > extensions out of tree at all?<br /> ><br /> > I'm interested in making the Windows
installerdistributions a bit more<br /> > extension dev friendly. In particular, I'd really like to see EDB's<br />
>Windows installers include the libintl.h for the included libintl, since<br /> > its omission, combined with Pg
beingbuilt with ENABLE_NLS, tends to<br /> > break things horribly. Users can always undefine ENABLE_NLS, but it's
an<br/> > unnecessary roadblock.<br /><br /></div>Sandeep, can you work on fixing this please?<br /><br />
Thanks.<br/><div class="HOEnZb"><div class="h5"><br /> > Are there any objections from -hackers to including 3rd
partyheaders<br /> > for libs we expose in our public headers in the binary distribution?<br /> ><br /> >
Otherthan bundling 3rd party headers, any ideas/suggestions for how we<br /> > might make ext building saner on
Windows?<br/> ><br /> > --<br /> >  Craig Ringer                   <a href="http://www.2ndQuadrant.com/"
target="_blank">http://www.2ndQuadrant.com/</a><br/> >  PostgreSQL Development, 24x7 Support, Training &
Services<br/> ><br /> ><br /></div></div><span class="HOEnZb"><font color="#888888">> --<br /> > Sent via
pgsql-hackersmailing list (<a href="mailto:pgsql-hackers@postgresql.org">pgsql-hackers@postgresql.org</a>)<br /> >
Tomake changes to your subscription:<br /> > <a href="http://www.postgresql.org/mailpref/pgsql-hackers"
target="_blank">http://www.postgresql.org/mailpref/pgsql-hackers</a><br/><br /><br /><br /> --<br /> Dave Page<br />
Blog:<a href="http://pgsnake.blogspot.com" target="_blank">http://pgsnake.blogspot.com</a><br /> Twitter: @pgsnake<br
/><br/> EnterpriseDB UK: <a href="http://www.enterprisedb.com" target="_blank">http://www.enterprisedb.com</a><br />
TheEnterprise PostgreSQL Company<br /></font></span></blockquote></div><br /><br clear="all" /><div><br /></div>-- <br
/><divdir="ltr">Sandeep Thakkar<div><br /></div></div></div></div></blockquote></div> 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/05/2014 07:32 PM, Craig Ringer wrote:
> Out of personal interest (in pain and suffering) I was recently
> looking into how to compile extensions out-of-tree on Windows using
> Visual Studio (i.e. no PGXS).
> 
> It looks like the conventional answer to this is "Do a source build
> of PG, compile your ext in-tree in contrib/, and hope the result is
> binary compatible with release PostgreSQL builds for Windows".
> Certainly that's how I've been doing it to date.

Yes, this pretty much exactly describes how I build PL/R for Windows.
I had to match my build system SDK with the one EDB uses to get a
compatible binary. It would be nice if we had something equivalent to
PGXS on Windows, or maybe even a community build system where authors
could get Windows binaries built.

Joe

- -- 
Joe Conway
credativ LLC: http://www.credativ.us
Linux, PostgreSQL, and general Open Source
Training, Service, Consulting, & 24x7 Support
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJSytzOAAoJEDfy90M199hlgdUP/3wHL7Akq4ONkYlFXq8d3LGJ
h5LNAWvILfJVOI3dmpHHzw8FCmWSsrsnDKuN/GUvJvhioPL0QsWgU1YGLwKNb7Mt
Q3famk8hJCe8PikrL5KvItk1jQtxem+M4wGcKZZoM2cb4soeRDxM5FtTKzZMHGi0
9GA3Tx/TpgGfIP35Xg9ckL/LejyOZIndrRHuJHREGZlIP27AW0SEscZMDq25Q5yy
jPcWki7hAIABohwkRPkFWVZArhSrCe1dA1Yy0Gad5cB/JdVB4xAjkmwLSa2suBod
nQ65G/8Hz88GIRxY1FlzPn+06IDnSdnZmrhxfZLn8Vl/mnMoW9h6pKmBNyWTQMoP
25Ex5/tYIQ6iYyUO3Ic/B/23OMVHu9bWXeyk1hKEpqCFR/1BctzafQI/vA4dRs0u
KRN3hua9GYnX+guw+d9mIkujPAeXphbjaMlgY6ckkENmiAg1HXfzv8+tKsQT4Pwx
IqcSNzIsnzTRSag1IKklwUW6DuTSGyFHyXsbWRA+kkxL2/ucHL7f2mCmbZ3Qg8WW
zthp6dN+9dLC/iH92qiS/nkFFxkkikyBpG2wb+Xcc0Ko1u26xp3e7ZFnCUZQ0Bse
DTiOIywW89ICk7pokI8vMEwJIN5d42dZ01GL6XdLT9iPTnGXSuCQsE2GSMBxUuHs
KbY+hsZrrvWH0QaVkrq5
=V4H5
-----END PGP SIGNATURE-----

On 01/06/2014 07:44 PM, Sandeep Thakkar wrote:
> Okay.
> 
> BTW, I just checked that Windows 32bit installer ships the libintl.h.
> Did you try if it works for you? Or it requires more headers? Somehow,
> Windows 64bit installer installer missed it. I'll fix the build script. 

That appears to be the only header missing for simple compilation. I did
some testing to verify that it works.

http://blog.2ndquadrant.com/compiling-postgresql-extensions-visual-studio-windows/


-- Craig Ringer                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services

On 01/06/2014 07:44 PM, Sandeep Thakkar wrote:
> Okay.
> 
> BTW, I just checked that Windows 32bit installer ships the libintl.h.
> Did you try if it works for you? Or it requires more headers? Somehow,
> Windows 64bit installer installer missed it. I'll fix the build script. 

Actually, on second thoughts there were two other issues. One I reported
separately (double-inclusion of pg_config_os.h due to _WIN32 vs WIN32).
The other is worth looking at here.

We don't set __declspec(dllexport) on extension functions automatically
when building stand-alone on Windows. So it's necessary to explicitly
specify PGDLLEXPORT for each function. We seem to work around this in
the Perl build toolchain by forcing export of everything not explicitly
static (which is, btw, a pretty crappy thing we should revisit in favour
of using PGDLLEXPORT and -fvisibility=hidden on Linux).

Instead we should perhaps be adding this automatically via a prototype
generated by PG_FUNCTION_INFO_V1, or adding PGDLLEXPORT to all our
example documentation. I think the latter is preferable because if we
start generating a prototype for the base function in PG_FUNCTION_INFO
when we didn't before it could break existing code.

Comments?

-- Craig Ringer                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services

Craig Ringer <craig@2ndquadrant.com> writes:
> We don't set __declspec(dllexport) on extension functions automatically
> when building stand-alone on Windows. So it's necessary to explicitly
> specify PGDLLEXPORT for each function.

I'm not sure I believe this.  I don't see any PGDLLEXPORT symbols in any
of the standard contrib modules; how is it that they work?

> Instead we should perhaps be adding this automatically via a prototype
> generated by PG_FUNCTION_INFO_V1, or adding PGDLLEXPORT to all our
> example documentation. I think the latter is preferable because if we
> start generating a prototype for the base function in PG_FUNCTION_INFO
> when we didn't before it could break existing code.

> Comments?

One of the things I've always found particularly vile about Microsoft
is the way that they seem to think it's fine to make people sprinkle
Windows-only droppings throughout code that's supposed to be portable.
I'm not in favor of asking people to write out PGDLLEXPORT manually
on every function unless it's *absolutely* necessary, and the available
evidence suggests to me that it isn't.

So if it's really necessary to change anything here, I'd rather see us
take the approach of hiding it in PG_FUNCTION_INFO_V1.  What happens
if we do that and there's also a manually-written prototype?
        regards, tom lane

Magnus Hagander <magnus@hagander.net> writes:
> On Sat, Jan 11, 2014 at 5:00 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I'm not sure I believe this.  I don't see any PGDLLEXPORT symbols in any
>> of the standard contrib modules; how is it that they work?

> They are built through our perl toolkit, which enables exporting of *all*
> symbols, regardless of flags in the code.

That seems like a perfectly reasonable solution, given the way we use
loadable modules.  Excess symbols in the module shouldn't really do
any harm.  Can't we just document the flags to use for this, if you're
building in some other way?
        regards, tom lane

On 01/11/2014 01:55 PM, Magnus Hagander wrote:
> On Sat, Jan 11, 2014 at 7:05 PM, Tom Lane <tgl@sss.pgh.pa.us 
> <mailto:tgl@sss.pgh.pa.us>> wrote:
>
>     Magnus Hagander <magnus@hagander.net <mailto:magnus@hagander.net>>
>     writes:
>     > On Sat, Jan 11, 2014 at 5:00 PM, Tom Lane <tgl@sss.pgh.pa.us
>     <mailto:tgl@sss.pgh.pa.us>> wrote:
>     >> I'm not sure I believe this.  I don't see any PGDLLEXPORT
>     symbols in any
>     >> of the standard contrib modules; how is it that they work?
>
>     > They are built through our perl toolkit, which enables exporting
>     of *all*
>     > symbols, regardless of flags in the code.
>
>     That seems like a perfectly reasonable solution, given the way we use
>     loadable modules.  Excess symbols in the module shouldn't really do
>     any harm.  Can't we just document the flags to use for this, if you're
>     building in some other way?
>
>
> It's not a build flag, and that's the main problem. It's the 
> src/tools/msvc/gendef.pl <http://gendef.pl> script that builds the 
> export list. And what Craig is after here is being able to build 
> extensions using standard tools without needing our full build 
> infrastructure.
>
>


What I'd like is something that would use or mimic our msvc build tools 
but for extensions. (And no, I don't have time to build it.)

cheers

andrew

On 01/12/2014 12:00 AM, Tom Lane wrote:
> Craig Ringer <craig@2ndquadrant.com> writes:
>> > We don't set __declspec(dllexport) on extension functions automatically
>> > when building stand-alone on Windows. So it's necessary to explicitly
>> > specify PGDLLEXPORT for each function.
> I'm not sure I believe this.  I don't see any PGDLLEXPORT symbols in any
> of the standard contrib modules; how is it that they work?
> 

We force all symbols to be exported using the project file generation
tools in the Pg build system, so PGDLLEXPORT is not required there.

That doesn't help stand alone builds. We don't have anything like PGXS
for Windows, so the only real option for building and distributing
reliably compatible extensions is your own Visual Studio project.

"Just build in contrib/ in a source tree" is not an acceptable answer,
because extensions need binary compatibility with public installer
distributions like the one EDB maintains. Unless you're sure your build
is set up just the same way I'm not at all convinced you're going to get
that, though you may get something that works *most* of the time. This
hasn't historically been a project keen on "mostly" solutions though.

> So if it's really necessary to change anything here, I'd rather see us
> take the approach of hiding it in PG_FUNCTION_INFO_V1.  What happens
> if we do that and there's also a manually-written prototype?

I've run out of weekend and have to go full-speed on upd. sb. views,
BDR, and event triggers tomorrow, so I don't think I'll have a chance to
test it immediately. I'll check after the CF deadline. That's certainly
one option, anyway, and one that'll solve the immediate issue with
extensions, and would be the most practical short term solution if it works.

Otherwise we can just add DLLEXPORT macros on public functions in
contrib/ and the docs. They're harmless on other platforms - and in fact
can be quite handy.



Those "windows droppings" can actually be useful: They annotate sources
with a useful piece of information we don't currently use on *nix, but
probably should. They specify a shared library or executable's public
interface.

Imagine that the macro was called "PGAPI" or "PG_PUBLIC_API", not
"PGDLLEXPORT". That's what it actually *means*, really.

On *nix we treat the executable and shared lib APIs as the union of the
exported symbols of all compilation units in the object. That's rather
crude, as it means that something very internal to Pg must be public if
it's to be visible across more than one compilation unit, and that if we
wish to hide it we have to squash things together into bigger
compilation units than we might otherwise want.

That's a pain for backward compat, as theoretically all of PostgreSQL's
exported symbols are public API. We don't really offer a good way to
tell what's public and what's not anyway.

Instead, we can and (IMO) should be marking our public API explicitly.
If it's supposed to be usable from a contrib / extension / fdw / etc,
mark it PGDLLEXPORT. Or rename that PG_PUBLIC_API and label it that.
Whatever. On Windows, these expand to __declspec(dllexport) and cause
symbol export. On gcc, we switch to building with -fvisiblity=hidden and
have these macros expand to __attribute__ ((dllexport)) . Access to
non-exported symbols becomes a link error. On ancient GCCs and on other
toolchains we continue to export all symbols by default, as now.

This'd reduce symbol table sizes a bit - though it's really only a big
win for C++ code with monster symbol tables. More importantly, it'd make
it easier to tell what's supposed to be reasonably stable across
versions as public API, and what's totally internal.

(It might be that we have enough exts delving deep in to Pg's guts
already that this is impractical. It'd be fun to see.)

Related references:
 http://gcc.gnu.org/wiki/Visibility http://people.redhat.com/drepper/dsohowto.pdf

-- Craig Ringer                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services

On 01/12/2014 04:54 PM, Craig Ringer wrote:
> On 01/12/2014 12:00 AM, Tom Lane wrote:
>> So if it's really necessary to change anything here, I'd rather see us
>> take the approach of hiding it in PG_FUNCTION_INFO_V1.  What happens
>> if we do that and there's also a manually-written prototype?
> 
> That's certainly
> one option, anyway, and one that'll solve the immediate issue with
> extensions, and would be the most practical short term solution if it works.

... which it kind-of does.

Turned out to be trivial to test. If the prototype with PGDLLEXPORT
appears *first*, then all is well. If the prototype with PGDLLEXPORT
appears AFTER a user-provided prototype it fails with:

1>DemoExtension.c(16): error C2375: 'add_one' : redefinition; different
linkage
1>          DemoExtension.c(14) : see declaration of 'add_one'

Two copies of the prototype, both with PGDLLEXPORT, work fine.

So really the question is: Do we care? The current usage of extensions
built outside the Pg tree on Windows is likely to be nearly zero, and is
already fiddly. I'm not too fussed if we make people fix up their
prototypes.

I think we can just emit a prototype for the function from
PG_FUNCTION_INFO_V1 . The only things it's going to break is the odd bit
of weirdly-built, not really supported extensions on Windows.

Are there any platforms that object to prototype redefinition? If not,
we can just emit a prototype on all platforms, not just Windows.

-- Craig Ringer                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services

Craig Ringer <craig@2ndquadrant.com> writes:
> Turned out to be trivial to test. If the prototype with PGDLLEXPORT
> appears *first*, then all is well. If the prototype with PGDLLEXPORT
> appears AFTER a user-provided prototype it fails with:

That's sort of what I thought would happen.  It's problematic because
putting the extern before the PG_FUNCTION_INFO_V1 is standard practice,
especially if you have the extern in a header file.

> I think we can just emit a prototype for the function from
> PG_FUNCTION_INFO_V1.

Doesn't sound like it; we'll still be forced to modify or remove
manually-written externs in basically all contrib and third-party code,
if we want it to build on Windows.  Which, as I said before, does not
seem to me to be a workable solution path at all.  It would take
years to track down all the third-party code and get it fixed, if
the authors don't themselves build/test on Windows.

And I continue to maintain that it's not acceptable for the Windows port
to require this anyway.  If any other platform came to us and said "hey
guys, you need to plaster this non-ANSI-C decoration on every exported
function", what response do you think they'd get?

One last point is that "automatically export every external symbol" is
exactly what happens for these modules on Unix-ish platforms.  If it
doesn't work the same way on Windows, we're just opening ourselves up to
subtle portability issues.

This needs to be fixed in the Windows build system, not the source code.
        regards, tom lane

On 01/13/2014 02:04 AM, Tom Lane wrote:
> Craig Ringer <craig@2ndquadrant.com> writes:
> 
>> I think we can just emit a prototype for the function from
>> PG_FUNCTION_INFO_V1.
> 
> Doesn't sound like it.

On second thought, agreed. The externs tending to appear in headers
kills that.

In that case - after the rush for the CF close, expect a patch from me
against post-9.4 that adds PGDLLEXPORT to the documentation examples,
and another that adds them for contribs (to help people using them as
examples). Nothing else needed, and it doesn't have to affect the *nix
build process or server/ code in any way.

I'll also add a note in the docs explaining what's wrong if you get an
error about an obviously-present function being missing in your
extension on Windows when it works on other platforms.

> And I continue to maintain that it's not acceptable for the Windows port
> to require this anyway.  If any other platform came to us and said "hey
> guys, you need to plaster this non-ANSI-C decoration on every exported
> function", what response do you think they'd get?
> 
> One last point is that "automatically export every external symbol" is
> exactly what happens for these modules on Unix-ish platforms.  If it
> doesn't work the same way on Windows, we're just opening ourselves up to
> subtle portability issues.

The portability issues are easily dealt with by using gcc's symbol
visibility features on *nix, which will produce slightly smaller
binaries with faster linkage too. I'll demo that once I've got the
current work off my plate. Platforms w/o gcc visibility don't need to
care, nothing changes, they just don't get the benefits. Everyone
develops on platforms that have gcc and visibility features so they'll
spot any issues introduced.

As described earlier, doing this helps differentiate "internal stuff"
from "public API" if we choose to, as well.

If we don't want to attempt to treat anything as "internal,
non-public-use, non-stable API", then there's no point using visibility
- everything not "static" is public API and should be exported to the
world. That's how things are now. In that case I'd stick with just
improving the docs to cover PGDLLEXPORT, and maybe the contribs.

I do think we should think about actually hiding private internal API
post-9.4, though.

-- Craig Ringer                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/07/2014 12:41 AM, Joe Conway wrote:
> Yes, this pretty much exactly describes how I build PL/R for
> Windows. I had to match my build system SDK with the one EDB uses
> to get a compatible binary. It would be nice if we had something
> equivalent to PGXS on Windows, or maybe even a community build
> system where authors could get Windows binaries built.

I'd be interested in hearing if you have any luck doing this with a
standlone MSVC build: .dll build type, no manifest, compile as C code,
appropriate include and lib paths, define WIN32, link to postgres.lib,
add PGDLLEXPORT to all declarations of functions with a
PG_FUNCTION_INFO_V1, build.

- -- Craig Ringer                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.15 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJS01XNAAoJELBXNkqjr+S2S4QH/icetcqZDmsHFHjZnm3hY+LP
M0+rU8+E3on5V/JmWR3bBESmlYXRYM53LmgwapwCfQ45rsrMSiKACyxtl/XkWsEh
38NTsagjlZtsyZIiUoe9d0szSNQerS86ZDBwXAvJnBSNLQy1AnDQ5tzsFbdeZph4
veL6MnoKNIacfbkEBoCjM0KyYdjnYnt4nRlmbGKQNg/h4Y9KqgJsFFpk0r8dfz+v
KNPPyOmdHIcMyCgJS9hIdAdzc+CPPjYBZC3oVVQAuIOsqccmTykPr4Nh3MVcSfCy
JJTRZhzgU6TdRZIi4adY8Ys39O+TJM2T5Wr1xJ6+Eapnd6L1AiY/08jbZIrsFVc=
=5/Y6
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/12/2014 06:56 PM, Craig Ringer wrote:
> On 01/07/2014 12:41 AM, Joe Conway wrote:
>> Yes, this pretty much exactly describes how I build PL/R for 
>> Windows. I had to match my build system SDK with the one EDB
>> uses to get a compatible binary. It would be nice if we had
>> something equivalent to PGXS on Windows, or maybe even a
>> community build system where authors could get Windows binaries
>> built.
> 
> I'd be interested in hearing if you have any luck doing this with
> a standlone MSVC build: .dll build type, no manifest, compile as C
> code, appropriate include and lib paths, define WIN32, link to
> postgres.lib, add PGDLLEXPORT to all declarations of functions with
> a PG_FUNCTION_INFO_V1, build.

Yes, that's more-or-less how I do it. I checked with EDB to be sure I
had the same SDK (looks like last time I did it it was SDK 7.1), then I:

1) download postgres source
2) copy plr source into contrib directory
3) modify src/tools/msvc/* to make plr look like any other contrib
4) build

As long as the SDK is matched, the resulting plr.dll works fine with
the EDB one-click installer version of postgres (or at least so far).

Joe

- -- 
Joe Conway
credativ LLC: http://www.credativ.us
Linux, PostgreSQL, and general Open Source
Training, Service, Consulting, & 24x7 Support
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJS01m8AAoJEDfy90M199hlkPEP/A1iCDJeRchfBM4OQB73k0VN
+4o9aVxo3n4jjoOiKI3zctpP6bHQI9/wyI9OYAKnRr3PutttrEyDn1MO3wBffTVN
k84rXyKKROgZOnGpQrE33QG82J1TRHmxZRyeQirQplekWEsB2RpiGu+jZJDjUsLn
svmgPZOka5tc+KqC4OJ3qXprv5OW4h6xMfZmUljn6jdzsuEubE6p4X/XqqGUJmVP
t6caIJ3yKvIcedQ7QZmxWEJXYGGoKlr6qv2Vo99SfqK9lGX6A7DFwAG3DaLmWqa/
rULapnfnET5jUtfncu3zkz6Kok848BsoaSfFZpHyyzvU2RlfAAPWCv0ZnNrSYTsl
s7blVBzanBoX63gy4MwMOcGdHmjPyYjeJ78m53GE28cvv62JEGubISkWoCbmOPYu
+UmxaCPR1l03n/QAv3Qo0fCEtU09OXzYci8NDEKqbdkWt9K/M4bNlv207J0CywiX
pEwQGVciaR5ylKYFZpKpCpgf6OElBVFaUk8uE41swgxM3McXWoPFuKocnhTWd0u6
9Fyb7W8W2kAJeOkiNWcpsX2DNHKMkZ8siTfMufyqHDxVfaJLie/0oTZejnGJf4Lh
vOBpqxdowN6xVhBR796vtzxjLg50vDj8NzOKvn6evtJMnWJkUiRBsY8JrR6g3fII
lb+3NTRa5x0ptApbsUKJ
=wFy+
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/13/2014 11:13 AM, Joe Conway wrote:

> Yes, that's more-or-less how I do it. I checked with EDB to be sure
> I had the same SDK (looks like last time I did it it was SDK 7.1),
> then I:
> 
> 1) download postgres source 2) copy plr source into contrib
> directory

What I mean is that you should not need a full Pg build tree to
compile extensions. Just as we use PGXS on *nix, so it is possible to
just use Visual Studio to build the extension on Windows as a VS
project like any other.

> As long as the SDK is matched, the resulting plr.dll works fine
> with the EDB one-click installer version of postgres (or at least
> so far).

The SDK shouldn't even need to match. Windows software is expected to
cope with mismatched, mix-and-match C runtimes. Icky, but occasionally
useful.

So long as you don't pass a FILE* across DLL boundaries, or free()
memory that was malloc()'d in another DLL, mixing runtimes should work
fine. I'd be interested in knowing what problems you encountered when
the runtime didn't match.

Maybe it's specific to how we do build within a Pg source tree? I was
able to use three different SDK builds against EDB's latest 9.3
packages - SDK 7.1 (x86 and x64), toolchain v90 (x86), and toolchain
v100 (x86) - to compile and test a moderately trivial extension.

- -- Craig Ringer                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.15 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJS01vxAAoJELBXNkqjr+S2zcoIAI75MetuO+osAlSOMdMDAgKh
7rqKu8VPyF2bjFVAiYdZOS9Yvlg3yZDzEG2JMbUSJOz3KtGIirZkIub8hgI4q4/W
6PBg5UZAiwK30AXILTqGBXio/Z+olbCPOKvcVv05OF4WExYLMek5Hc2SKy3UIudj
HHvC10LkSeVvJvNj+rK7SUjQIpwa4a2+cc0gx87z7kd8ElVwJ3D/c1Eb3DM8mdsg
KqIlmWkGolwnk1fb/JoabkO9XVvRjPgpj/aR9Ak6mUH7QTXGMqafdpPeCV9BhGRK
d6MDDT3ncoEzHZd7GsKeqVWBFz9eZ7hLXiQR6rZ7bidvNVc4V7NnM2dr50uhoec=
=pgpf
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/12/2014 07:22 PM, Craig Ringer wrote:
> On 01/13/2014 11:13 AM, Joe Conway wrote: What I mean is that you
> should not need a full Pg build tree to compile extensions. Just as
> we use PGXS on *nix, so it is possible to just use Visual Studio to
> build the extension on Windows as a VS project like any other.

I have never gotten that to work.

>> As long as the SDK is matched, the resulting plr.dll works fine 
>> with the EDB one-click installer version of postgres (or at
>> least so far).
> 
> The SDK shouldn't even need to match. Windows software is expected
> to cope with mismatched, mix-and-match C runtimes. Icky, but
> occasionally useful.

Unfortunately my experience tells me otherwise. Mismatched SDK =
plr.dll that does not work correctly with EDB builds of postgres.

> So long as you don't pass a FILE* across DLL boundaries, or free() 
> memory that was malloc()'d in another DLL, mixing runtimes should
> work fine. I'd be interested in knowing what problems you
> encountered when the runtime didn't match.

plr would not load. I'm not good enough with MSVC and Windows
debuggers to figure out why, and don't have the time or interest to
become so, I just know matching the SDK to the postgres build resulted
in it owrking for me and no more complaints of this type from the
field. It would work fine with the postgres build I made, but not with
the EDB build which of course is what virtually everyone running pg on
Windows uses.

> Maybe it's specific to how we do build within a Pg source tree? I
> was able to use three different SDK builds against EDB's latest
> 9.3 packages - SDK 7.1 (x86 and x64), toolchain v90 (x86), and
> toolchain v100 (x86) - to compile and test a moderately trivial
> extension.

Trivial is probably the key thing. With PL/R it also has to link
against the R.dll which is built with yet another toolchain on Windows.

Joe

- -- 
Joe Conway
credativ LLC: http://www.credativ.us
Linux, PostgreSQL, and general Open Source
Training, Service, Consulting, & 24x7 Support
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJS018UAAoJEDfy90M199hl/mAP/2OhXbvrlKSzP6fWi8g9Tez0
PnWaOFXXPIeOi5lJ/o5rC7HrQdiCCBljSBAffq0PKl8SVn2cDwmA5E2n4T3JfZQF
PYGZiYuqoiZ6i+svW7x6XocDnIhJgMJvTvys7ToApjmkD3VEgj7RO8MHQyYmVDsh
A9WIPpyb96mTmzlTLnHZDkfL7MgEof4kTHHC2jOa6i3wMq+zATc6lBTXOcrrGzS8
qd/iIap0kNdwKgLEX/jXip0YOB8SMfxOeHVLV+790JUwnWmBJnbn3XDqFEmj39kK
dGEP8vaxjPppyEmMvkGZd5Hxw6WIGFASTjyn6kXH1VfVqsLNZYO+rwTXnSVtyqH0
aFCKLT7awMBjFSh3plFQcqxeeqElZZaCRNVO5xooQ28xoUoUl/wVYqI0yoF9hKKm
NlJ8jJGB6aEImFlQ7QUg2eZRfMpyYc9J06uaX1+/L3g71O4O2Xzgc6gPVWvYCIQP
BvcNBtUlxA38H5wjiMSlyyz4Si95cIIbqDfUliKZ1Ab0W24en0vnvISxk6v/2GKc
vE9X7GRFUjmUJNoIvkRu0hnzp5S955sO0X6Q6pDmgM2esGRADMTntY0Bcxp8R2qg
qiZkVs1vfuewLmzz4LqixItW9BhMHK3zdGcv07xntNt+EAaz8g3cU5tBZP8CP6y8
JTa/fyFth7hL+ttkH5hc
=9+nY
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/13/2014 11:35 AM, Joe Conway wrote:
> On 01/12/2014 07:22 PM, Craig Ringer wrote:
>> On 01/13/2014 11:13 AM, Joe Conway wrote: What I mean is that
>> you should not need a full Pg build tree to compile extensions.
>> Just as we use PGXS on *nix, so it is possible to just use Visual
>> Studio to build the extension on Windows as a VS project like any
>> other.
> 
> I have never gotten that to work.
> 
>>> As long as the SDK is matched, the resulting plr.dll works fine
>>>  with the EDB one-click installer version of postgres (or at 
>>> least so far).
> 
>> The SDK shouldn't even need to match. Windows software is
>> expected to cope with mismatched, mix-and-match C runtimes. Icky,
>> but occasionally useful.
> 
> Unfortunately my experience tells me otherwise. Mismatched SDK = 
> plr.dll that does not work correctly with EDB builds of postgres.

Something for me to play with once this CF is over, then.

- -- Craig Ringer                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.15 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJS01/mAAoJELBXNkqjr+S2ZQcH/0rHjCW+6pvg8727dJHSmYfM
fY18VBlgYzjfzgPaFLYgp4OqT7VHl2h1d7AapVo8wRblVD4z1hEFtW6/j56Mx1wm
UTw7rWhs7utA9G+gzWcHJz9VDxED0ROFH+IwurSM85P57ztPKaRhvp6YT3fibhYb
kqw51FrEfvlnhCMi3Art3DGmHtzDRLwGTI03YOr/GRfWsccHPwpRLkpQkyuMsyOX
UBhPenz0OhtkVxEfFSmyVWmu6NzlOQyxgFgl8zW7R9pq4fTBgOfo198RDkGKnCno
9KYq3H8VqLxHgpyR2KP3netrqDvDBzk0xVmgoaJbQyT6HyuQDWH6lIHE+5RTyyM=
=nreR
-----END PGP SIGNATURE-----

Craig Ringer wrote:
> Out of personal interest (in pain and suffering) I was recently looking
> into how to compile extensions out-of-tree on Windows using Visual
> Studio (i.e. no PGXS).
> 
> It looks like the conventional answer to this is "Do a source build of
> PG, compile your ext in-tree in contrib/, and hope the result is binary
> compatible with release PostgreSQL builds for Windows". Certainly that's
> how I've been doing it to date.
> 
> How about everyone else here? Does anyone actually build and distribute
> extensions out of tree at all?

I build binaries for oracle_fdw for EDB's binaries "by hand",
i.e. I look at what PGXS does on Linux and try to do something
comparable with "cl".  It's rather painful, and something like
PGXS for MSVC would be very welcome.

Yours,
Laurenz Albe