Thread: Capping CPU usage?

Folks:

I had a new question from a client: is it possible to "cap" CPU usage
 for PostgreSQL running on Linux? They don't care if the procedure
 degrades Postgres performance, but they can't afford to have Postgres
 take up more than 30% of processor for more than 400 milliseconds
(they
 are running some real-time operations).

I can't imagine that postmaster could do this, but I thought it there
 might be some kind of Linux Kernel CPU quota option I haven't heard
of.
  Can anybody point me in the right direction?

-Josh Berkus

On Wednesday, December 11, 2002, at 12:12 PM, Josh Berkus wrote:
> I can't imagine that postmaster could do this, but I thought it there
>  might be some kind of Linux Kernel CPU quota option I haven't heard
> of.
>   Can anybody point me in the right direction?
>

You can always use nice(1) to lower it's priority.  This would allow
other processes to get the CPU more often, effectively limiting it in
the face of more demanding processes.

ulimit has a CPU time option, but it's probably not what you want.  I
don't believe there is a kernel option for such a thing.  I don't
recall seeing this type of accounting anywhere, but there are likely
some patches.

Cory 'G' Watson

On Wed, 2002-12-11 at 13:12, Josh Berkus wrote:
> Folks:
>
> I had a new question from a client: is it possible to "cap" CPU usage
>  for PostgreSQL running on Linux? They don't care if the procedure
>  degrades Postgres performance, but they can't afford to have Postgres
>  take up more than 30% of processor for more than 400 milliseconds
> (they
>  are running some real-time operations).
>
> I can't imagine that postmaster could do this, but I thought it there
>  might be some kind of Linux Kernel CPU quota option I haven't heard
> of.
>   Can anybody point me in the right direction?

Don't know about Linux, but BSD cannot do that.  CPU limits are hard --
once you hit it it'll dump the process.

Anyway, would it be sufficient to simply reduce the priority of the
process?

--
Rod Taylor <rbt@rbt.ca>

PGP Key: http://www.rbt.ca/rbtpub.asc

> > I can't imagine that postmaster could do this, but I thought it there
> >  might be some kind of Linux Kernel CPU quota option I haven't heard
> > of.
> >   Can anybody point me in the right direction?


I was reading an interview last night (found from /.) on the O(1) scheduler. One thing that was mentioned was batch
taskswhich get only cpu that's not being used for other things, in blocks of 3 seconds. It has some harder enforcement
ofnice levels (i.e. batch @ 10 can completely prevent a batch @ 15 from running untill it completes, but is completely
interruptableby ordinary processes).  Since all the parameters are tweakable, many while running, this may be a place
tolook.   

eric

Rod Taylor <rbt@rbt.ca> writes:
>> I had a new question from a client: is it possible to "cap" CPU usage
>> for PostgreSQL running on Linux?

> Anyway, would it be sufficient to simply reduce the priority of the
> process?

If the issue is to prevent Postgres *as a whole* from hogging CPU usage,
I would think that nice-ing the postmaster at launch would work
beautifully.  Requests like "I want Postgres to use no more than 30%
of CPU" make no sense to me: if the CPU is otherwise idle, why should
you insist on reserving 70% of it for the idle loop?

But what we commonly see is "I want to cap the resource usage of this
particular query", and that is a whole lot harder.  You cannot win by
nice-ing one single backend, because of priority-inversion concerns.
(The queries you would like to be high-priority might be blocked waiting
for locks held by low-priority backends.)

            regards, tom lane

Tom,

> If the issue is to prevent Postgres *as a whole* from hogging CPU
> usage,
> I would think that nice-ing the postmaster at launch would work
> beautifully.  Requests like "I want Postgres to use no more than 30%
> of CPU" make no sense to me: if the CPU is otherwise idle, why should
> you insist on reserving 70% of it for the idle loop?

<grin>  That's what I asked the person who asked me.  Apparently, they
want to do real-time operations without forking out for a real-time OS.
  My response was "you can nice the postmaster, and simplify your
queries, but that's about it".

Thank you, everybody, for confirming this.

-Josh Berkus

On Thu, 2002-12-12 at 11:43, Josh Berkus wrote:
> Tom,
>
> > If the issue is to prevent Postgres *as a whole* from hogging CPU
> > usage,
> > I would think that nice-ing the postmaster at launch would work
> > beautifully.  Requests like "I want Postgres to use no more than 30%
> > of CPU" make no sense to me: if the CPU is otherwise idle, why should
> > you insist on reserving 70% of it for the idle loop?
>
> <grin>  That's what I asked the person who asked me.  Apparently, they
> want to do real-time operations without forking out for a real-time OS.
>   My response was "you can nice the postmaster, and simplify your
> queries, but that's about it".

Maybe, even with processes niced down low, the current Linux scheduler
"drop down" a currently schueduled/executing process.

Maybe that would change with the low-latency patches, or with the
O(1) scheduler in kernel 2.6.

--
+---------------------------------------------------------------+
| Ron Johnson, Jr.        mailto:ron.l.johnson@cox.net          |
| Jefferson, LA  USA      http://members.cox.net/ron.l.johnson  |
|                                                               |
| "My advice to you is to get married: If you find a good wife, |
| you will be happy; if not, you will become a philosopher."    |
|    Socrates                                                   |
+---------------------------------------------------------------+