Thread: Detailed documentation for external calls (threading, shared resources etc)
Greetings, This is actually a request for documentation guidance. I intend to develop an extension to postgresql. Basically I'd like to place calls to network using ZeroMQ, and I need to have detailed information about a lot of things, especially threading issues. I need to have some global resources which will be presumably used by multiple threads. I can see that there is a lot of documentation, but I'd really appreciate pointers towards the books, or key documents that'd help me move forward faster (docs/books about inner workings of key functionality) I'll be using C (most likely the best option) to develop code, so which books/documents would you recommend? Cheers Seref
Re: Detailed documentation for external calls (threading, shared resources etc)
From
Florian Pflug
Date:
On Jun12, 2011, at 19:26 , Seref Arikan wrote: > This is actually a request for documentation guidance. I intend to > develop an extension to postgresql. Basically I'd like to place calls > to network using ZeroMQ, and I need to have detailed information about > a lot of things, especially threading issues. I need to have some > global resources which will be presumably used by multiple threads. > I can see that there is a lot of documentation, but I'd really > appreciate pointers towards the books, or key documents that'd help me > move forward faster (docs/books about inner workings of key > functionality) I'll be using C (most likely the best option) to > develop code, so which books/documents would you recommend? There are no threading issues in postgres, because postgres doesn't use threads. Each client connection is serviced by one backend process, launched by the postmaster when a new client connects. Communication between backend processes takes places via a shared memory segment and at times also via signals. The documentation contains extensive information about how to interface 3rd-party code with postgres. To see how to interface C functions with the SQL layer, read http://www.postgresql.org/docs/9.0/interactive/xfunc-c.html. If you need to also access the database from your C-language functions, also read http://www.postgresql.org/docs/9.0/interactive/spi.html. More exhaustive documentation is spread around the source tree in the form of README files. I suggest you read the ones concerned with the postgres subsystems you're dealing with. At the very least, you should read .//src/backend/utils/mmgr/README which explains how postgres manages memory. The various contrib modules, found in contrib/ in the source tree, are also a good reference. best regards, Florian Pflug
Re: Detailed documentation for external calls (threading, shared resources etc)
From
Greg Stark
Date:
<p>Well process or thread, the issues are the same. <p>Check out the existing modules and how they use spinlocks and lwlocksto protect shared memory data structures.<p>One thing to beware of is that there's no shared memory manager so allshared data structures need to be fixed size allocated on startup. Anything where that isn't good enough usually endsup as an on disk data structure.<div class="gmail_quote">On Jun 12, 2011 8:37 PM, "Florian Pflug" <<a href="mailto:fgp@phlo.org">fgp@phlo.org</a>>wrote:<br type="attribution" />> On Jun12, 2011, at 19:26 , Seref Arikanwrote:<br />>> This is actually a request for documentation guidance. I intend to<br /> >> develop an extensionto postgresql. Basically I'd like to place calls<br />>> to network using ZeroMQ, and I need to have detailedinformation about<br />>> a lot of things, especially threading issues. I need to have some<br /> >>global resources which will be presumably used by multiple threads.<br />>> I can see that there is a lot ofdocumentation, but I'd really<br />>> appreciate pointers towards the books, or key documents that'd help me<br />>> move forward faster (docs/books about inner workings of key<br />>> functionality) I'll be using C (mostlikely the best option) to<br />>> develop code, so which books/documents would you recommend?<br /> > <br/>> <br />> There are no threading issues in postgres, because postgres doesn't<br />> use threads. Each clientconnection is serviced by one backend process,<br />> launched by the postmaster when a new client connects. Communication<br/> > between backend processes takes places via a shared memory segment<br />> and at times also viasignals.<br />> <br />> The documentation contains extensive information about how to interface<br />> 3rd-partycode with postgres. To see how to interface C functions with<br /> > the SQL layer, read <a href="http://www.postgresql.org/docs/9.0/interactive/xfunc-c.html">http://www.postgresql.org/docs/9.0/interactive/xfunc-c.html</a>.<br />>If you need to also access the database from your C-language functions,<br /> > also read <a href="http://www.postgresql.org/docs/9.0/interactive/spi.html">http://www.postgresql.org/docs/9.0/interactive/spi.html</a>.<br />><br />> More exhaustive documentation is spread around the source tree in<br /> > the form of README files. Isuggest you read the ones concerned with<br />> the postgres subsystems you're dealing with. At the very least, you<br/>> should read .//src/backend/utils/mmgr/README which explains how<br /> > postgres manages memory.<br />><br />> The various contrib modules, found in contrib/ in the source tree,<br />> are also a good reference.<br/>> <br />> best regards,<br />> Florian Pflug<br />> <br />> <br /> > <br />> -- <br />>Sent via pgsql-hackers mailing list (<a href="mailto:pgsql-hackers@postgresql.org">pgsql-hackers@postgresql.org</a>)<br/>> To make changes to your subscription:<br/>> <a href="http://www.postgresql.org/mailpref/pgsql-hackers">http://www.postgresql.org/mailpref/pgsql-hackers</a><br/></div>
Re: Detailed documentation for external calls (threading, shared resources etc)
From
Dimitri Fontaine
Date:
Seref Arikan <serefarikan@kurumsalteknoloji.com> writes: > This is actually a request for documentation guidance. I intend to > develop an extension to postgresql. Basically I'd like to place calls > to network using ZeroMQ, and I need to have detailed information about You didn't tell us about what you want to achieve, only how. It could be that your problem could be solved with a custom PGQ consumer, that leaves well outside the database server. PGQ is the generic queue layer on which the replication system Londiste is built. http://wiki.postgresql.org/wiki/Skytools http://wiki.postgresql.org/wiki/PGQ_Tutorial http://wiki.postgresql.org/wiki/Londiste_Tutorial Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support