Thread: slow performance problems: Setting processes per-user limits for daemons (such as postgres, interchange, etc.)

How do I set maximum number of concurrent processes for users that run
daemons, such as postgreSQL (under postgres user), or interchange, or mysql,
or anything for that matter.  limits.conf edits by themselves don't work.

I found in performance testing that my system would lock up under a light
load (400 concurrent HTTP GET's or so) because "Could not Fork" errors.  And
`top` showed I never got much more than 510 processes running.

I've been successful in creating a user (test), then setting it's
performance limits (/etc/security/limits.conf), and ALSO setting
/etc/pam.d/login with a "session    required
/lib/security/pam_limits.so".  But when I perform the same steps for
postgres, (I even tried making a /etc/pam.d/postgresql file) it doesn't have
any effect.

I've tried all the docs that I know, can anyone give me the magic
combination to increase my 256 processes-per-user dilemma that applies to
daemons?

I know that Oracle avoids the problem all together and just has you run it's
osh program (which is suid root) and that sets ulimits.

I'm using rh62 on a dual Zeon, raid0/1 on 6 ultra160 cheetahs, and 1gb ram.

Thanks for any help,

Dan Browning
Network & Database Administrator
Cyclone Computer Systems


On Wed, 13 Sep 2000, Dan Browning wrote:

> How do I set maximum number of concurrent processes for users that run
> daemons, such as postgreSQL (under postgres user), or interchange, or mysql,
> or anything for that matter.  limits.conf edits by themselves don't work.

Kurt Seifried recently wrote a little blurb about PAM
in SysAdmin, but it looks like you've already explored
that area.  It might help to keep plugging away at that, because
it sure looks like a finer grained way to impose limits
on processes.  Of course, PAM can only control processes
which are linked against libpam.  Does postgres even
call libpam anywhere?  If it doesn't, maybe you can
find a way to call it somewhere.

The 256 processes per user sounds suspiciously like a 8 bit
counter somewhere in kernel space.  There has been effort
to increase a whole bunch of Linux limits to make it more
enterprise-ready, some of these things are available
as patches against the 2.2 kernel series, but you might
need to wait for the 2.4 series, unless you want to try
a 2.3 kernel.

Does it help to try and keep your connections to postgres
"persistent"?

Gord

Matter Realisations     http://www.materialisations.com/
Gordon Haverland, B.Sc. M.Eng. President
101  9504 182 St. NW    Edmonton, AB, CA  T5T 3A7
780/481-8019            ghaverla @ freenet.edmonton.ab.ca