Thread: shared_preload_libraries support on Win32?
(working on the PL debugger...) <br /><br /> It appears that the libraries listed in shared_preload_libraries will *not*be inherited by spawned backends on Win32 platforms.<br /><br /> Do we have to do something special to make that work?<br/><br /> Using ProcessExplorer (from sysinternals.com), I can see that my plugins are loaded into the postmaster,but not into the individual backends.<br /><br /> If I set local_preload_libraries equal to shared_preload_libraries,the plugins are loaded into the backends as expected (so it seems unlikely that I have a pathnameor permissions problem).<br /><br /> Should we just call process_shared_preload_libraries() after calling read_nondefault_variables()(in SubPostmasterMain())?<br /><br /> -- Korry<br /><table cellpadding="0" cellspacing="0"width="100%"><tr><td><br /><br /> --<br /> Korry Douglas <a href="mailto:korryd@enterprisedb.com">korryd@enterprisedb.com</a><br/> EnterpriseDB <a href="http://www.enterprisedb.com">http://www.enterprisedb.com</a></td></tr></table>
<korryd@enterprisedb.com> writes:
> It appears that the libraries listed in shared_preload_libraries will
> *not* be inherited by spawned backends on Win32 platforms.
Well, yeah, because it's a fork/exec on that platform.
> Should we just call process_shared_preload_libraries() after calling
> read_nondefault_variables() (in SubPostmasterMain())?
I don't entirely see the point. The value of shared_preload_libraries
is to avoid paying per-process overhead to load the libraries, and that
benefit is already lost in a fork/exec world. Might as well just let
the libraries be loaded when and if needed.
I think we've failed to document that shared_preload_libraries doesn't
work on Windows, which is an oversight.
regards, tom lane
Tom Lane wrote: > <korryd@enterprisedb.com> writes: > >> It appears that the libraries listed in shared_preload_libraries will >> *not* be inherited by spawned backends on Win32 platforms. >> > > Well, yeah, because it's a fork/exec on that platform. > > >> Should we just call process_shared_preload_libraries() after calling >> read_nondefault_variables() (in SubPostmasterMain())? >> > > I don't entirely see the point. The value of shared_preload_libraries > is to avoid paying per-process overhead to load the libraries, and that > benefit is already lost in a fork/exec world. Might as well just let > the libraries be loaded when and if needed. > > The only benefit I can see is that it would assist in having a common configuration. > I think we've failed to document that shared_preload_libraries doesn't > work on Windows, which is an oversight. > > Maybe postmaster should also log a warning if it detects this. cheers andrew
Andrew Dunstan <andrew@dunslane.net> writes:
> Tom Lane wrote:
>> I don't entirely see the point. The value of shared_preload_libraries
>> is to avoid paying per-process overhead to load the libraries, and that
>> benefit is already lost in a fork/exec world. Might as well just let
>> the libraries be loaded when and if needed.
> The only benefit I can see is that it would assist in having a common
> configuration.
Actually ... I take that back. I was thinking of the original purpose
of preload_libraries, which was strictly performance optimization.
But in the new world of plugins there may be functional reasons for
wanting libraries to be loaded into backends --- and
shared_preload_libraries is not isomorphic to local_preload_libraries.
The permissions situation is different.
Korry's right, we should force re-loading of shared_preload_libraries
in the EXEC_BACKEND case. The needed documentation change is to point
out that on Windows this is not a performance win, but it might still
be wanted for instrumentation or debugging plugins.
regards, tom lane
<blockquote type="CITE"><pre> <font color="#000000">Actually ... I take that back. I was thinking of the original purpose</font> <font color="#000000">of preload_libraries, which was strictly performance optimization.</font> <font color="#000000">But in the new world of plugins there may be functional reasons for</font> <font color="#000000">wanting libraries to be loaded into backends --- and</font> <font color="#000000">shared_preload_libraries is not isomorphic to local_preload_libraries.</font> <font color="#000000">The permissions situation is different.</font> </pre></blockquote><br /> And, shared_preload_libraries is processed (in the postmaster) before the shared-memory segmentis created, so a shared_preload_library can call RequestAddinShmemSpace() and RequestAddinLWLocks(), but a local_preload_librarycannot.<br /><br /><br /> -- Korry
<korryd@enterprisedb.com> writes:
>> But in the new world of plugins there may be functional reasons for
>> wanting libraries to be loaded into backends --- and
>> shared_preload_libraries is not isomorphic to local_preload_libraries.
>> The permissions situation is different.
> And, shared_preload_libraries is processed (in the postmaster) before
> the shared-memory segment is created, so a shared_preload_library can
> call RequestAddinShmemSpace() and RequestAddinLWLocks(), but a
> local_preload_library cannot.
That doesn't seem like an issue though, since the copy in the postmaster
will have done that anyway.
regards, tom lane
<blockquote type="CITE"><pre> <font color="#000000">> And, shared_preload_libraries is processed (in the postmaster) before</font> <font color="#000000">> the shared-memory segment is created, so a shared_preload_library can</font> <font color="#000000">> call RequestAddinShmemSpace() and RequestAddinLWLocks(), but a</font> <font color="#000000">> local_preload_library cannot.</font> <font color="#000000">That doesn't seem like an issue though, since the copy in the postmaster</font> <font color="#000000">will have done that anyway.</font> </pre></blockquote><br /> You're right - we need the copy in the postmaster (to setup shared memory and LW locks), and weneed them in the backends too. I just want to avoid having to set both shared_preload_libraries and local_preload_libraries(to the same thing). Adding a call to process_shared_preload_libraries() in SubPostmasterMain() seemsto fix the problem for me.<br /><br /> Thanks for your input. I'll submit a patch.<br /><br /><br /> --Korry
korryd@enterprisedb.com wrote: > > > And, shared_preload_libraries is processed (in the postmaster) before > > > the shared-memory segment is created, so a shared_preload_library can > > > call RequestAddinShmemSpace() and RequestAddinLWLocks(), but a > > > local_preload_library cannot. > > > > That doesn't seem like an issue though, since the copy in the postmaster > > will have done that anyway. > > > You're right - we need the copy in the postmaster (to setup shared > memory and LW locks), and we need them in the backends too. I just want > to avoid having to set both shared_preload_libraries and > local_preload_libraries (to the same thing). Adding a call to > process_shared_preload_libraries() in SubPostmasterMain() seems to fix > the problem for me. Just make sure you don't load the libraries in bgwriter et al ... -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Alvaro Herrera <alvherre@commandprompt.com> writes:
> korryd@enterprisedb.com wrote:
>> You're right - we need the copy in the postmaster (to setup shared
>> memory and LW locks), and we need them in the backends too.
> Just make sure you don't load the libraries in bgwriter et al ...
I see that Korry's patch doesn't do that, but I'm wondering why exactly.
In a Unix environment such libraries *would* be propagated into bgwriter
and every other postmaster child; is there a reason for the setup on
Windows to be different? In particular, what about autovacuum, which
ISTM should be as close to a standard backend as possible?
Either way we do it, authors of plugins used this way will have to test
both cases (I'm glad I insisted on EXEC_BACKEND mode being testable under
Unix ...)
regards, tom lane
<blockquote type="CITE"><pre> <font color="#000000">>> You're right - we need the copy in the postmaster (to setup shared</font> <font color="#000000">>> memory and LW locks), and we need them in the backends too.</font> <font color="#000000">> Just make sure you don't load the libraries in bgwriter et al ...</font> <font color="#000000">I see that Korry's patch doesn't do that, but I'm wondering why exactly.</font> <font color="#000000">In a Unix environment such libraries *would* be propagated into bgwriter</font> <font color="#000000">and every other postmaster child; is there a reason for the setup on</font> <font color="#000000">Windows to be different? In particular, what about autovacuum, which</font> <font color="#000000">ISTM should be as close to a standard backend as possible?</font> </pre></blockquote> I thought about that too... does autovacuum ever need to re-index? If so, it seems that it would needaccess to any index functions (for function-based indexes) that might reside in a shared_preload_library.<br /><br /><blockquotetype="CITE"><pre> <font color="#000000">Either way we do it, authors of plugins used this way will have to test</font> <font color="#000000">both cases (I'm glad I insisted on EXEC_BACKEND mode being testable under</font> <font color="#000000">Unix ...)</font> </pre></blockquote><br /> And I'm glad that RequestAddinShmemSpace() and RequestAddinLWLocks() don't complain if called afterpostmaster start :-)<br /><br /> -- Korry<br /><table cellpadding="0" cellspacing="0" width="100%"><tr><td><br/><br /> --<br /> Korry Douglas <a href="mailto:korryd@enterprisedb.com">korryd@enterprisedb.com</a><br/> EnterpriseDB <a href="http://www.enterprisedb.com">http://www.enterprisedb.com</a></td></tr></table>
<korryd@enterprisedb.com> writes:
>> I see that Korry's patch doesn't do that, but I'm wondering why exactly.
>> In a Unix environment such libraries *would* be propagated into bgwriter
>> and every other postmaster child; is there a reason for the setup on
>> Windows to be different? In particular, what about autovacuum, which
>> ISTM should be as close to a standard backend as possible?
> I thought about that too... does autovacuum ever need to re-index? If
> so, it seems that it would need access to any index functions (for
> function-based indexes) that might reside in a shared_preload_library.
Any ordinary C-language function is not an issue, because its library
will get autoloaded upon use. AFAICS what we have to think about here
is instrumentation or debugging plugins that someone might wish to have
running in the postmaster's special children. Maybe there's no such
animal; I'm not sure. But in the Unix environment they'd be active in
those processes.
regards, tom lane