Thread: libpq is not thread safe
When postgreSQL is compiled with --thread-safe that libpq should be thread safe. But it is not true when somebody call fork(). The problem is that fork() forks only active threads and some mutex can stay locked by another thread. We use ssl_config mutex which is global. We need implement atfork handlers to fix this. See http://www.opengroup.org/onlinepubs/009695399/functions/pthread_atfork.html We should add pthread_atfork into _ini libpq section. Another problem with fork is that new process inherit connections and so on. Which is not also good, but it is happened also on single threaded application and developer can fix it in own code. Maybe some notice in documentation should help what application should do after fork. Comments? Zdenek
Zdenek Kotala wrote: > When postgreSQL is compiled with --thread-safe that libpq should be > thread safe. But it is not true when somebody call fork(). The problem > is that fork() forks only active threads and some mutex can stay locked > by another thread. We use ssl_config mutex which is global. > > We need implement atfork handlers to fix this. See > http://www.opengroup.org/onlinepubs/009695399/functions/pthread_atfork.html > > We should add pthread_atfork into _ini libpq section. > > Another problem with fork is that new process inherit connections and so > on. Which is not also good, but it is happened also on single threaded > application and developer can fix it in own code. Maybe some notice in > documentation should help what application should do after fork. Yep, added to TODO list: Make libpq thread-safe in programs that use fork() This requires the use of pthread_atfork() to release global locks held in libpq * http://archives.postgresql.org/pgsql-hackers/2009-04/msg00747.php -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Zdenek Kotala <Zdenek.Kotala@Sun.COM> writes:
> When postgreSQL is compiled with --thread-safe that libpq should be
> thread safe. But it is not true when somebody call fork(). The problem
> is that fork() forks only active threads and some mutex can stay locked
> by another thread. We use ssl_config mutex which is global.
fork() without exec() when there are open libpq connections is
unbelievably dangerous anyway --- you will have multiple processes
that all think they own the same database connection. I think writing
code to deal with this for the ssl_config mutex is entirely a waste
of time.
regards, tom lane
Tom Lane píše v ne 03. 05. 2009 v 16:39 -0400: > Zdenek Kotala <Zdenek.Kotala@Sun.COM> writes: > > When postgreSQL is compiled with --thread-safe that libpq should be > > thread safe. But it is not true when somebody call fork(). The problem > > is that fork() forks only active threads and some mutex can stay locked > > by another thread. We use ssl_config mutex which is global. > > fork() without exec() when there are open libpq connections is > unbelievably dangerous anyway --- you will have multiple processes > that all think they own the same database connection. The difference is that developer can close connection, but he is not able to unlock a lock after fork. OK libpq does not offer any PQFinish variant which frees only resources and close connection without terminate message, but he can do it with dirty hacking. Another advantage of atfork handler is that you can close all open connection and clean resource (similar to what pkcs11 library does). But at this moment libpq does not keep list of open connections and atfork handler works only with pthreads. > I think writing code to deal with this for the ssl_config mutex is entirely a waste > of time. yeah, I prefer to document it together how to deal with fork() and sessions. Zdenek
Zdenek Kotala wrote: > > Tom Lane p??e v ne 03. 05. 2009 v 16:39 -0400: > > Zdenek Kotala <Zdenek.Kotala@Sun.COM> writes: > > > When postgreSQL is compiled with --thread-safe that libpq should be > > > thread safe. But it is not true when somebody call fork(). The problem > > > is that fork() forks only active threads and some mutex can stay locked > > > by another thread. We use ssl_config mutex which is global. > > > > fork() without exec() when there are open libpq connections is > > unbelievably dangerous anyway --- you will have multiple processes > > that all think they own the same database connection. > > The difference is that developer can close connection, but he is not > able to unlock a lock after fork. OK libpq does not offer any PQFinish > variant which frees only resources and close connection without > terminate message, but he can do it with dirty hacking. > > Another advantage of atfork handler is that you can close all open > connection and clean resource (similar to what pkcs11 library does). But > at this moment libpq does not keep list of open connections and atfork > handler works only with pthreads. > > > I think writing code to deal with this for the ssl_config mutex is entirely a waste > > of time. > > yeah, I prefer to document it together how to deal with fork() and > sessions. Done, patch attached and applied. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + Index: doc/src/sgml/libpq.sgml =================================================================== RCS file: /cvsroot/pgsql/doc/src/sgml/libpq.sgml,v retrieving revision 1.288 diff -c -c -r1.288 libpq.sgml *** doc/src/sgml/libpq.sgml 27 Apr 2009 16:27:36 -0000 1.288 --- doc/src/sgml/libpq.sgml 28 May 2009 20:01:37 -0000 *************** *** 64,69 **** --- 64,79 ---- whether a connection was successfully made before queries are sent via the connection object. + <warning> + <para> + On Unix, forking a process with open libpq connections can lead to + unpredictable results because the parent and child processes share + the same sockets and operating system resources. For this reason, + such usage is not recommended, though doing an <function>exec</> from + the child process to load a new executable is safe. + </para> + </warning> + <note> <para> On Windows, there is a way to improve performance if a single
Bruce Momjian wrote: > > Another advantage of atfork handler is that you can close all open > > connection and clean resource (similar to what pkcs11 library does). But > > at this moment libpq does not keep list of open connections and atfork > > handler works only with pthreads. > > > > > I think writing code to deal with this for the ssl_config mutex is entirely a waste > > > of time. > > > > yeah, I prefer to document it together how to deal with fork() and > > sessions. > > Done, patch attached and applied. I went with a <warning> because it seemed most appropriate, but it looks very large: http://developer.postgresql.org/pgdocs/postgres/libpq-connect.html Should it be a notice? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian píše v čt 28. 05. 2009 v 17:20 -0400: > > > > Done, patch attached and applied. > > I went with a <warning> because it seemed most appropriate, but it looks > very large: > > http://developer.postgresql.org/pgdocs/postgres/libpq-connect.html > > Should it be a notice? I prefer warning. It is important message and beginners usually don't know it. Zdenek
Zdenek Kotala wrote: > > Bruce Momjian p??e v ?t 28. 05. 2009 v 17:20 -0400: > > > > > > > Done, patch attached and applied. > > > > I went with a <warning> because it seemed most appropriate, but it looks > > very large: > > > > http://developer.postgresql.org/pgdocs/postgres/libpq-connect.html > > > > Should it be a notice? > > I prefer warning. It is important message and beginners usually don't > know it. OK. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +