On Fri, Mar 16, 2012 at 8:34 PM, Greg Stark
<stark@mit.edu> wrote:
On Fri, Mar 16, 2012 at 11:29 PM, Jeff Davis <
pgsql@j-davis.com> wrote:
> There is a lot of difference between those two. In particular, it looks
> like the problem you are seeing is coming from the background writer,
> which is not running during initdb.
library opens any connections prior to forking and then uses them
after forking that would work at first but it would get confused
quickly once more than one backend tries to use the same connection.
The data being sent would all be mixed together and they would see
responses to requests other processes sent.
The opened connections thing was the first thing I thought of.. the HDFS documentation claims the C interface is thread-safe..
However, since I noticed that different processes (having different process IDs) are calling the mdwrite, then it might be a possibility.
I tried reconnecting to HDFS on every mdwrite request, but it didn't work out.
You need to ensure that any network connections are opened up *after*
the new processes are forked.
There are other things going on that could cause problems. initdb
probably doesn't deal with many errors so it might not be casing any
longjmps that happen when Postgres handles errors. I suspect it
doesn't create any temporary files, etc.
There's also a long history of threads not playing well with Postgres.
I think that's overblown and I believe they should work fine. Most of
it was caused by a bug in an old version of libc. But you do have to
ensure that the other threads don't call any Postgres functions
because the Postgres code is very much not thread-aware.
--
greg
I'm novice in PG, and if I got this one running then I'll have achieved what I wanted to do without further digging in PG. So I was thinking if there was a configuration option (or something similar) that will eliminate (or reduce) the role the background writer process.. I believe it can be one of the WAL options but I'm not sure.. Any suggestions?
