Re: Threaded PosgreSQL server - Mailing list pgsql-hackers

Justin Clift wrote:
> 
> Haroldo Stenger wrote:
> <snip>
> >
> > I agree with cross-polinization among open source projects. BTW, this practice
> > should be encouraged, and not called "stealing", not even as a joke, as I've
> > seen it called for example for the TCP/IP Linux stack code (99% sure this was
> > the one module), which came from the *BSD projects, in its very first version.
> > Also mentioning that BSD -> GPL was possible, but not the other way round; I
> > don't mean to start a war or anything, just exposing facts.
> >
> > > The point is that nobody is even implying that this is a "for v7.3"
> > > project ... there have been several projects that have been initiated over
> > > the years that have straddled releases, and we have alot of very good
> > > developers, and testers, that will make sure that any changes are "for the
> > > good" ...
> >
> > Yes, I agree. If starting to think & code thread safe *now* proves *not* to be a
> > waste of everybody's time, that's the path to follow. This very point is the one
> > under technical examination, right?
> 
> So, with this thought in mind of "starting to think & code thread safe",
> we should start putting together a set of reference guidlines,
> especially drawing on the experience of people whom have good, solid
> experience with threaded, multi-process, cross-platform coding.  It
> should take into account the people who are reading it, may not be as
> experienced in this um... specialised area of coding too.
> 
> We've identified "global variables" needing to be done in a better and
> more consistent way.
> 
> So, what else do coders need to do when "thinking and coding thread
> safe", that we can make into a guidline for forthcoming PostgreSQL
> coding?

Going from a "process model" to a "threaded model" is a HUGE
undertaking. In the process model, all data is assumed to be private,
and shared data must be explicitly shared.  In a threaded model all data
is implicitly shared and private data must be explicitly made private.
Do not under estimate what this means or how hard it is to convert one
to the other.

Also:

Think of file handles. In a threaded version of postgreSQL, all
connections will be competing for file handles. I think the limit in
Linux is 1024.

All threads will be competing for memory mapping. As systems get more
and more RAM, on the x86 and other 32 bit machines, process space is
limited to 2 to 3 gig. If you have 8 gig in your system, PostgreSQL
won't be able to use it.

As I have said before, multithreading queries within a connection
process would be pretty cool, on a low load server, this could make a
big performance increase, but it may be easier to create a couple I/O
threads per connection process and devise some queuing mechanism for
disk reads/write. In essence provide an asynchronous I/O system. This
would give you the some of the performance of multithreading a query,
while not requiring a complete thread-safe implementation.

I think threading connections is a VERY bad idea. I am dubious that the
amount of work will result in a decent return on investment.