Thread: high %system time

high %system time

From
Jacob Coby
Date:
We recently upgraded from php 4.3.10 to 5.1.2, and the %system time has
skyrocketed:

Cpu(s): 42.8% us, 43.6% sy,  0.0% ni, 11.3% id,  2.2% wa,  0.2% hi,  0.0% si
Mem:   8312844k total,  7566168k used,   746676k free,    22356k buffers
Swap:  2040244k total,      520k used,  2039724k free,  6920384k cached

it used to be around 5% to 10%.  The server is a quad-xeon dell pe 6650
running CentOS 4.2 with 8G of RAM running pg 8.1.1.

How can I determine what is causing such high system load?  it seems to
have immediately jumped with the php upgrade.

thanks,
--
-Jacob

Re: high %system time

From
Bill Moran
Date:
On Tue, 06 Jun 2006 09:19:10 -0400
Jacob Coby <jcoby@listingbook.com> wrote:

> We recently upgraded from php 4.3.10 to 5.1.2, and the %system time has
> skyrocketed:
>
> Cpu(s): 42.8% us, 43.6% sy,  0.0% ni, 11.3% id,  2.2% wa,  0.2% hi,  0.0% si
> Mem:   8312844k total,  7566168k used,   746676k free,    22356k buffers
> Swap:  2040244k total,      520k used,  2039724k free,  6920384k cached
>
> it used to be around 5% to 10%.  The server is a quad-xeon dell pe 6650
> running CentOS 4.2 with 8G of RAM running pg 8.1.1.
>
> How can I determine what is causing such high system load?  it seems to
> have immediately jumped with the php upgrade.

This sounds more like a PHP question than a PostgreSQL question.

However, we had a similar problem recently, and I ran ktrace (on FreeBSD)
to track down the system calls.  2 things jumped out:
1) pg_connect() creates a LOT of system time compared to pg_pconnect()
2) require_once() and include_once() are truly evil.

HTH.

--
Bill Moran
Collaborative Fusion Inc.

Re: high %system time

From
Jacob Coby
Date:
Bill Moran wrote:
> On Tue, 06 Jun 2006 09:19:10 -0400
> Jacob Coby <jcoby@listingbook.com> wrote:
>
>> We recently upgraded from php 4.3.10 to 5.1.2, and the %system time has
>> skyrocketed:
>>
>> Cpu(s): 42.8% us, 43.6% sy,  0.0% ni, 11.3% id,  2.2% wa,  0.2% hi,  0.0% si
>> Mem:   8312844k total,  7566168k used,   746676k free,    22356k buffers
>> Swap:  2040244k total,      520k used,  2039724k free,  6920384k cached
>>
>> it used to be around 5% to 10%.  The server is a quad-xeon dell pe 6650
>> running CentOS 4.2 with 8G of RAM running pg 8.1.1.
>>
>> How can I determine what is causing such high system load?  it seems to
>> have immediately jumped with the php upgrade.
>
> This sounds more like a PHP question than a PostgreSQL question.

I know.  I'm looking for advice on what would affect system time.  Its
actually not related so much to the php upgrade as a major code change
we did.  What baffles me is that the db abstraction layer didn't change.

>
> However, we had a similar problem recently, and I ran ktrace (on FreeBSD)
> to track down the system calls.  2 things jumped out:
> 1) pg_connect() creates a LOT of system time compared to pg_pconnect()

We use pg_pconnect().  We also reuse connections to avoid the overhead
of RESET ALL (php issues a RESET ALL on persistent connections, which
can take up to 0.1s)

> 2) require_once() and include_once() are truly evil.

Can you elaborate a bit on this?  Privately if you want since it's off
topic for this list.

Thanks.
--
Jacob Coby


Re: high %system time

From
Bill Moran
Date:
On Tue, 06 Jun 2006 09:37:16 -0400
Jacob Coby <jcoby@listingbook.com> wrote:

> Bill Moran wrote:
>
> > 2) require_once() and include_once() are truly evil.
>
> Can you elaborate a bit on this?  Privately if you want since it's off
> topic for this list.

They're evil.  They are a bad idea gone horribly awry.

Here are some specific reason why they should never be used and should
be removed from the language:
1) They encourage sloppy coding.  If hackers are using *_once() it means
   they don't know their inclusion hierarchy.
2) They incur lots of system time by stat()ing lots of directories and
   files for everything you include.
3) If you have the same file included multiple times, it still runs the
   stat()s _every_time_, even if the file has already been included and
   doesn't need included again.  I believe this is necessary for security
   purposes, but it's a LOT of overhead.

After determining that require_once() was requiring significant amounts
of CPU for our application, we tasked a single developer to organize and
replace them all with require().  It took him an hour or two.  The result
was about a 2x performance boost, with significant reduction in system
time.  It was a big enough jump that the bottleneck relocated to another
location and I haven't been back to optimizing PHP since.

In conclusion, (require|include)_once() are evil.  The only time they
should be used is when the application is so small that it doesn't need
them.

--
Bill Moran
Collaborative Fusion Inc.