Thread: C-procedure crashed in Postgres 8.3.3 when using 'text' variable (WinXP)
Hello! I'd like to write C-function returning text for using in PG 8.3.3. (WinXP SP2) For compilation I use MinGW-5.1.4 (gcc 3.4.5),MSYS-1.0.10. The code looks like this (in reduced variant): #include "postgres.h" #include "fmgr.h" #include "executor/executor.h" #include "utils/timestamp.h" #include "utils/builtins.h" #include "utils/formatting.h" #ifdef PG_MODULE_MAGIC PG_MODULE_MAGIC; #endif #define PG_CSTR_GET_TEXT(cstrp) \ DatumGetTextP(DirectFunctionCall1(textin, CStringGetDatum(cstrp))) PG_FUNCTION_INFO_V1(SomeFunction); Datum SomeFunction(PG_FUNCTION_ARGS) { PG_RETURN_TEXT_P(PG_CSTR_GET_TEXT("my_string")); } Then I create stored procedure in PG: CREATE OR REPLACE FUNCTION "service"."some_function" () RETURNS text AS '$libdir/some_dll', 'SomeFunction' LANGUAGE C STRICT; ----------------------------------------------------- When I'm trying to use "service"."some_function" PG fails. I tried to use the example from documentation: PG_FUNCTION_INFO_V1(concat_text); Datum concat_text(PG_FUNCTION_ARGS) { text *arg1 = PG_GETARG_TEXT_P(0); text *arg2 = PG_GETARG_TEXT_P(1); int32 new_text_size = VARSIZE(arg1) + VARSIZE(arg2) - VARHDRSZ; text *new_text = (text *) palloc(new_text_size); SET_VARSIZE(new_text, new_text_size); memcpy(VARDATA(new_text), VARDATA(arg1), VARSIZE(arg1) - VARHDRSZ); memcpy(VARDATA(new_text) + (VARSIZE(arg1) - VARHDRSZ), VARDATA(arg2), VARSIZE(arg2) - VARHDRSZ); PG_RETURN_TEXT_P(new_text); } CREATE FUNCTION concat_text(text, text) RETURNS text AS '$libdir/some_dll', 'concat_text' LANGUAGE C STRICT; And I got the same result. When I'm trying to return integer, f.e., evetything is fine. Also all this examples worked fine with PostgreSQL 8.2. I saw some similar problems - but I didn't find the solution ( http://archives.postgresql.org/pgsql-general/2008-05/msg00060.php http://archives.postgresql.org/pgsql-general/2008-05/msg00097.php Have somebody any ideas how to resolve this problem? Thanks in advance, Marina
el dorado <do_ra_do@mail.ru> writes: > I'd like to write C-function returning text for using in PG 8.3.3. (WinXP SP2) > For compilation I use MinGW-5.1.4 (gcc 3.4.5),MSYS-1.0.10. Was the Postgres server you're using built the same way? I seem to recall some incompatibilities between MinGW and MSVC builds. regards, tom lane
Re: C-procedure crashed in Postgres 8.3.3 when using 'text' variable (WinXP)
From
Craig Ringer
Date:
Tom Lane wrote: > el dorado <do_ra_do@mail.ru> writes: >> I'd like to write C-function returning text for using in PG 8.3.3. (WinXP SP2) >> For compilation I use MinGW-5.1.4 (gcc 3.4.5),MSYS-1.0.10. > > Was the Postgres server you're using built the same way? I seem to > recall some incompatibilities between MinGW and MSVC builds. I'd be surprised if there weren't some given how tricky it is to mix different compilers on Windows, where that usually means mixing C runtimes as well. On any platform it's best to stick to the same C runtime for all libraries to be linked into the same process*. That said, PostgreSQL does appear to use its own allocator for memory passed across library boundaries, and if it doesn't pass any FILE pointers across library boundaries either then that's the worst two problems that arise with a C runtime mismatch taken care of. As far as I know, anyway, and I've had the misfortune to have to dig into it a bit. It seems like a good idea to use the same version of the same compiler that your version of PostgreSQL was built with anyway. If nothing else, I've only taken a cursory look at the code and have little knowledge about it, so there could well be code paths/uses where memory is malloc()'d in one DLL and freed()'d in another, etc. I'm sure there all sorts of other possible wrinkles, and it'd just be easier to stick with the same compiler. You can download Visual Studio 2005 Express Edition for free from MS. * Thankfully UNIXes tend to be sane enough to use the same C library across all C compilers installed (and generally stick to one compiler anyway), so most of us are able to avoid this exciting little complication. -- Craig Ringer
Re: C-procedure crashed in Postgres 8.3.3 when using 'text' variable (WinXP)
From
Alvaro Herrera
Date:
Craig Ringer wrote: > That said, PostgreSQL does appear to use its own allocator for memory > passed across library boundaries, and if it doesn't pass any FILE > pointers across library boundaries either then that's the worst two > problems that arise with a C runtime mismatch taken care of. As far as I > know, anyway, and I've had the misfortune to have to dig into it a bit. Note that our allocator uses malloc() underneath. And I would think that we do pass FILE pointers around too; AllocateFile works with FILE, so any library that's using that method to get to files would be in trouble (which they should, because it's the way to ensure we don't leak files on transaction abort and also to ensure that the system continues to work on the face of lots of open files). -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Re: C-procedure crashed in Postgres 8.3.3 when using 'text' variable (WinXP)
From
Craig Ringer
Date:
Alvaro Herrera wrote: > Craig Ringer wrote: > >> That said, PostgreSQL does appear to use its own allocator for memory >> passed across library boundaries, and if it doesn't pass any FILE >> pointers across library boundaries either then that's the worst two >> problems that arise with a C runtime mismatch taken care of. As far as I >> know, anyway, and I've had the misfortune to have to dig into it a bit. > > Note that our allocator uses malloc() underneath. Sure. That's fine, so long as the wrapper functions palloc and pfree are real functions that reside in the same DLL on windows. You can pass pointers around freely, otherwise you couldn't do much at all ... the problem is with a call path like this: - DLL2 calls a function in DLL1 - DLL1 malloc()'s some memory and returns a pointer to it - DLL2 free()'s the pointer My understanding of the reason that this breaks if DLL1 and DLL2 are linked to different C runtimes is that it's essentially a double free plus a memory leak. The allocator in DLL2 has never allocated the memory so the free call is freeing unallocated memory. It's never properly freed by the allocator in DLL2, so it's leaked as far as that allocator is concerned. It's quite possible that my understanding of the problem isn't correct, though. In any case, the safe thing to do is: - DLL2 calls a function in DLL1 - DLL1 malloc()'s some memory and returns a pointer to it - DLL2 calls a free() wrapper function defined in DLL1 - The free() wrapper in DLL1 calls the real free() to release the memory If palloc() and pfree() are real functions defined in the same DLL, then it should be just fine as use of them will ensure that everything uses the same underlying memory allocator. A problem might arise if Pg ever passes memory not allocated with palloc() around, though. It looks like palloc is a macro for the real function MemoryContextAlloc, and pfree is just a real function - so it should be fine. They don't expose references to malloc() and free() directly. This actually applies to any situation in which multiple malloc()/free() implementations are used - but most people and platforms are sane enough to avoid that. > And I would think > that we do pass FILE pointers around too; AllocateFile works with FILE, > so any library that's using that method to get to files would be in > trouble (which they should, because it's the way to ensure we don't leak > files on transaction abort and also to ensure that the system continues > to work on the face of lots of open files). Yep, I'm pretty sure that'll cause issues when using a mingw-compiled object in a msvc++-compiled copy of Pg. -- Craig Ringer