Thread: Pg+Linux swap use
Hi, Old: Post 7.3.2, P4 1.8, 1 MB RAM, 2 x IDE SW RAID 1, RedHat 8 New: Post 7.3.4, Xeon 2.4, 1 MB RAM, 2 x SCSI 15k SW RAID 1, RedHat 9 Both use: Only postgresql on server. Buffers = 8192, effective cache = 100000 In old plataform the free and vmstat reports no use of swap. In new, the swap is in constant use (40-100 MB), with a low but constant swap out and swap in. The cache memory ~ 830000 and buffers ~ 43000 I try to reduce the buffers to 1024 with no effects. Thanks, Alexandre
On Thu, 30 Oct 2003, alexandre :: aldeia digital wrote: > Hi, > > Old: Post 7.3.2, P4 1.8, 1 MB RAM, 2 x IDE SW RAID 1, RedHat 8 > New: Post 7.3.4, Xeon 2.4, 1 MB RAM, 2 x SCSI 15k SW RAID 1, RedHat 9 > > Both use: Only postgresql on server. Buffers = 8192, effective cache = 100000 > > In old plataform the free and vmstat reports no use of swap. > In new, the swap is in constant use (40-100 MB), with a low but constant > swap out and swap in. The cache memory ~ 830000 and buffers ~ 43000 Do both machines have the same amount of swap, and what kernels are the two running? Also, do the xeons have hyperthreading turned on. I doubt hyperthreading has anything to do with this problem, but I've found that turning off hyperthreading in the BIOS gets me a faster machine under Postgresql so I just thought I'd throw that out there.
On Thu, 30 Oct 2003 17:49:08 -0200 (BRST) "alexandre :: aldeia digital" <alepaes@aldeiadigital.com.br> wrote: > Both use: Only postgresql on server. Buffers = 8192, effective cache = > 100000 > Well, I'm assuming you meant 1GB of ram, not 1MB :) Check a ps auxw to see what is running. Perhaps X is running gobbling up your precious mem. But still.. with 1GB there should be virtually no swap activity. How busy is the DB? How many connections? and is sort_mem set high? -- Jeff Trout <jeff@jefftrout.com> http://www.jefftrout.com/ http://www.stuarthamm.net/
Jeff wrote: > On Thu, 30 Oct 2003 17:49:08 -0200 (BRST) > "alexandre :: aldeia digital" <alepaes@aldeiadigital.com.br> wrote: > > >>Both use: Only postgresql on server. Buffers = 8192, effective cache = >>100000 > Well, I'm assuming you meant 1GB of ram, not 1MB :) > > Check a ps auxw to see what is running. Perhaps X is running gobbling up > your precious mem. But still.. with 1GB there should be virtually no > swap activity. > > How busy is the DB? How many connections? > > and is sort_mem set high? Also are two kernels exactly same? In my experience linux kernel behaves slightly different from version to version w.r.t swap aggressiveness... Shridhar
Scott, Jeff and Shridhar: 1 GB RAM :) The stock kernels are not the same, HyperThreading enabled. 80 simultaneous connections. sort_mem = 4096 I will compile my own kernel on this weekend, and I will report to the list after. Thank's all Alexandre > Also are two kernels exactly same? In my experience linux kernel behaves > slightly different from version to version w.r.t swap aggressiveness... > > Shridhar > > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster >
On Fri, Oct 31, 2003 at 12:03:59PM -0200, alexandre :: aldeia digital wrote:
> Scott, Jeff and Shridhar:
>
> 1 GB RAM :)
>
> The stock kernels are not the same, HyperThreading enabled. 80
Some people have reported that things actually slow down with HT
enabled. Have you tried turning it off?
A
--
----
Andrew Sullivan 204-4141 Yonge Street
Afilias Canada Toronto, Ontario Canada
<andrew@libertyrms.info> M2P 2A8
+1 416 646 3304 x110
Not being one to hijack threads, but I haven't heard of this performance hit
when using HT, I have what should all rights be a pretty fast server, dual
2.4 Xeons with HT 205gb raid 5 array, 1 gig of memory. And it is only 50% as
fast as my old server which was a dual AMD MP 1400's with a 45gb raid 5
array and 1gb of ram. I have read everything I could find on Pg performance
tweaked all the variables that were suggested and nothing. Which is why I
subscribed to this list, just been lurking so far but this caught my eye.
Rob
-----Original Message-----
From: pgsql-performance-owner@postgresql.org
[mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Andrew Sullivan
Sent: Friday, October 31, 2003 8:36 AM
To: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] Pg+Linux swap use
On Fri, Oct 31, 2003 at 12:03:59PM -0200, alexandre :: aldeia digital wrote:
> Scott, Jeff and Shridhar:
>
> 1 GB RAM :)
>
> The stock kernels are not the same, HyperThreading enabled. 80
Some people have reported that things actually slow down with HT
enabled. Have you tried turning it off?
A
--
----
Andrew Sullivan 204-4141 Yonge Street
Afilias Canada Toronto, Ontario Canada
<andrew@libertyrms.info> M2P 2A8
+1 416 646 3304 x110
---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend
On Fri, 31 Oct 2003 09:31:19 -0600 "Rob Sell" <lists@facnd.com> wrote: > Not being one to hijack threads, but I haven't heard of this > performance hit when using HT, I have what should all rights be a > pretty fast server, dual 2.4 Xeons with HT 205gb raid 5 array, 1 gig > of memory. And it is only 50% as fast as my old server which was a > dual AMD MP 1400's with a 45gb raid 5 array and 1gb of ram. I have > read everything I could find on Pg performance tweaked all the > variables that were suggested and nothing. Which is why I subscribed > to this list, just been lurking so far but this caught my eye. > > Rob There's benchmarks around that show in _some_ cases HT is not all it is cracked up to be, somtimes running slower. But the real point of this thread isn't all this stuff about hyperthreading, the problem is the guy is seeing swapping.. I'm guessing RH is running some useless stuff in the BG.. or maybe he's running a retarded kernel... or.. maybe.. just.. maybe.. little elves are doing it. -- Jeff Trout <jeff@jefftrout.com> http://www.jefftrout.com/ http://www.stuarthamm.net/
Jeff wrote: > On Fri, 31 Oct 2003 09:31:19 -0600 > "Rob Sell" <lists@facnd.com> wrote: > > >>Not being one to hijack threads, but I haven't heard of this >>performance hit when using HT, I have what should all rights be a >>pretty fast server, dual 2.4 Xeons with HT 205gb raid 5 array, 1 gig >>of memory. And it is only 50% as fast as my old server which was a >>dual AMD MP 1400's with a 45gb raid 5 array and 1gb of ram. I have >>read everything I could find on Pg performance tweaked all the >>variables that were suggested and nothing. Which is why I subscribed >>to this list, just been lurking so far but this caught my eye. >> >>Rob > There's benchmarks around that show in _some_ cases HT is not all it is > cracked up to be, somtimes running slower. To use HT effectively on needs. 1. A kernel that understands HT. 2. A task scheduler that understands HT 3. A CPU intensive load. So if you are running a stock RedHat and production postgresql database, turn it off. It won't hurt certainly(Almost certainly) > I'm guessing RH is running some useless stuff in the BG.. or maybe he's > running a retarded kernel... or.. maybe.. just.. maybe.. little elves > are doing it. Too much..:-) I guess Alexandre can tune bdflush to be little less agressive. Comparing bdflush values on two machines might turn up something. His idea of compiling kernel is also good one. He can also try tuning some values in /proc/sys/vm but I don't find any documentation offhand. I usually run slackware and a handcompiled 2.6-test4. None of them use any swap unless true memory starts falling low. This touch-swap-even-if-oodles-of-ram-is-free is something I have't experienced on my desktop for quite a while. Shridhar
Just for an additional viewpoint. I'm finishing up a project based on FreeBSD and PostgreSQL. The target server is a Dual 2.4G Intel machine. I have tested the application with hyperthreading enabled and disabled. To all appearances, enabling hyperthreading makes the box act like a quad, with the expected increase in processing capability - _for_this_application_. I have also heard the claims and seen the tests that show hyperthreading occasionally decreasing performance. I think in the end, you just have to test your particular application to see how it reacts. Rob Sell wrote: > Not being one to hijack threads, but I haven't heard of this performance hit > when using HT, I have what should all rights be a pretty fast server, dual > 2.4 Xeons with HT 205gb raid 5 array, 1 gig of memory. And it is only 50% as > fast as my old server which was a dual AMD MP 1400's with a 45gb raid 5 > array and 1gb of ram. I have read everything I could find on Pg performance > tweaked all the variables that were suggested and nothing. Which is why I > subscribed to this list, just been lurking so far but this caught my eye. > > Rob > > -----Original Message----- > From: pgsql-performance-owner@postgresql.org > [mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Andrew Sullivan > Sent: Friday, October 31, 2003 8:36 AM > To: pgsql-performance@postgresql.org > Subject: Re: [PERFORM] Pg+Linux swap use > > On Fri, Oct 31, 2003 at 12:03:59PM -0200, alexandre :: aldeia digital wrote: > >>Scott, Jeff and Shridhar: >> >>1 GB RAM :) >> >>The stock kernels are not the same, HyperThreading enabled. 80 > > > Some people have reported that things actually slow down with HT > enabled. Have you tried turning it off? > > A > -- Bill Moran Potential Technologies http://www.potentialtech.com
Bill Moran <wmoran@potentialtech.com> writes: > Just for an additional viewpoint. I'm finishing up a project based on FreeBSD > and PostgreSQL. The target server is a Dual 2.4G Intel machine. I have tested > the application with hyperthreading enabled and disabled. To all appearances, > enabling hyperthreading makes the box act like a quad, with the expected increase > in processing capability - _for_this_application_. > > I have also heard the claims and seen the tests that show hyperthreading > occasionally decreasing performance. I think in the end, you just have to > test your particular application to see how it reacts. My understanding is that the case where HT hurts is precisely your case. When you have two real processors with HT the kernel will sometimes schedule two jobs on the two virtual processors on the same real processor leaving the two virtual processors on the other real processor idle. As far as I know a single processor machine with HT does not benefit from disabling HT. -- greg
For the record I am running on SuSE with a pretty much stock kernel. Not to sound naïve, but is turning of HT something done in the bios? Rob -----Original Message----- From: pgsql-performance-owner@postgresql.org [mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Bill Moran Sent: Friday, October 31, 2003 9:56 AM To: pgsql-performance@postgresql.org Subject: Re: [PERFORM] Pg+Linux swap use Just for an additional viewpoint. I'm finishing up a project based on FreeBSD and PostgreSQL. The target server is a Dual 2.4G Intel machine. I have tested the application with hyperthreading enabled and disabled. To all appearances, enabling hyperthreading makes the box act like a quad, with the expected increase in processing capability - _for_this_application_. I have also heard the claims and seen the tests that show hyperthreading occasionally decreasing performance. I think in the end, you just have to test your particular application to see how it reacts. Rob Sell wrote: > Not being one to hijack threads, but I haven't heard of this performance hit > when using HT, I have what should all rights be a pretty fast server, dual > 2.4 Xeons with HT 205gb raid 5 array, 1 gig of memory. And it is only 50% as > fast as my old server which was a dual AMD MP 1400's with a 45gb raid 5 > array and 1gb of ram. I have read everything I could find on Pg performance > tweaked all the variables that were suggested and nothing. Which is why I > subscribed to this list, just been lurking so far but this caught my eye. > > Rob > > -----Original Message----- > From: pgsql-performance-owner@postgresql.org > [mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Andrew Sullivan > Sent: Friday, October 31, 2003 8:36 AM > To: pgsql-performance@postgresql.org > Subject: Re: [PERFORM] Pg+Linux swap use > > On Fri, Oct 31, 2003 at 12:03:59PM -0200, alexandre :: aldeia digital wrote: > >>Scott, Jeff and Shridhar: >> >>1 GB RAM :) >> >>The stock kernels are not the same, HyperThreading enabled. 80 > > > Some people have reported that things actually slow down with HT > enabled. Have you tried turning it off? > > A > -- Bill Moran Potential Technologies http://www.potentialtech.com ---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Greg Stark wrote: > Bill Moran <wmoran@potentialtech.com> writes: > >>Just for an additional viewpoint. I'm finishing up a project based on FreeBSD >>and PostgreSQL. The target server is a Dual 2.4G Intel machine. I have tested >>the application with hyperthreading enabled and disabled. To all appearances, >>enabling hyperthreading makes the box act like a quad, with the expected increase >>in processing capability - _for_this_application_. >> >>I have also heard the claims and seen the tests that show hyperthreading >>occasionally decreasing performance. I think in the end, you just have to >>test your particular application to see how it reacts. > > My understanding is that the case where HT hurts is precisely your case. When > you have two real processors with HT the kernel will sometimes schedule two > jobs on the two virtual processors on the same real processor leaving the two > virtual processors on the other real processor idle. Yup, that's why I tested it. While more testing is probably in order, I could find no disadvantages to running with HTT enabled. And when I ran many background jobs (which is likely to happen during batch processing on this sytem) the system seemed to perform as if it really had 4 processors. Perhaps this is an indication of the quality of the FreeBSD scheduler (maybe it's HTT aware and makes decisions accordingly?), but I'm not involved enough in that level of development to do any more than speculate. Again, this is pure speculation ... can anyone with a more technical insight comment on whether my guess is correct, or whether I'm not testing enough? -- Bill Moran Potential Technologies http://www.potentialtech.com
On Fri, Oct 31, 2003 at 10:42:06AM -0600, Rob Sell wrote:
> For the record I am running on SuSE with a pretty much stock kernel. Not to
> sound na?ve, but is turning of HT something done in the bios?
As far as I know, yes.
A
--
----
Andrew Sullivan 204-4141 Yonge Street
Afilias Canada Toronto, Ontario Canada
<andrew@libertyrms.info> M2P 2A8
+1 416 646 3304 x110
On Fri, 2003-10-31 at 11:37, Greg Stark wrote: > My understanding is that the case where HT hurts is precisely your case. When > you have two real processors with HT the kernel will sometimes schedule two > jobs on the two virtual processors on the same real processor leaving the two > virtual processors on the other real processor idle. If you're seeing this behavior, it's sounds like a bug/deficiency in your kernel's scheduler: if it is HT-aware, it should go to some lengths to avoid this kind of processor allocation. -Neil
We had a problem at work that when a windows box would connect to a samba share with a lot of files in it, the kswapd was going nuts, even though we weren't low on memory at all. Updating to the 2.4.18 or so of the later redhats fixed that issue. It might be related. I think the kflush daemon can get fooled into thinking it needs to get hoppin right now in older 2.4.x kernels. On Fri, 31 Oct 2003, alexandre :: aldeia digital wrote: > Scott, Jeff and Shridhar: > > 1 GB RAM :) > > The stock kernels are not the same, HyperThreading enabled. 80 > simultaneous connections. sort_mem = 4096 > > I will compile my own kernel on this weekend, and I will report > to the list after. > > Thank's all > > Alexandre > > > > Also are two kernels exactly same? In my experience linux kernel behaves > > slightly different from version to version w.r.t swap aggressiveness... > > > > Shridhar > > > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 4: Don't 'kill -9' the postmaster > > > > > ---------------------------(end of broadcast)--------------------------- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/docs/faqs/FAQ.html > >
Rob Sell wrote: > Not being one to hijack threads, but I haven't heard of this performance hit > when using HT, I have what should all rights be a pretty fast server, dual > 2.4 Xeons with HT 205gb raid 5 array, 1 gig of memory. And it is only 50% as > fast as my old server which was a dual AMD MP 1400's with a 45gb raid 5 > array and 1gb of ram. I have read everything I could find on Pg performance > tweaked all the variables that were suggested and nothing. Which is why I > subscribed to this list, just been lurking so far but this caught my eye. Not to get into a big Intel vs AMD argument but 50% sounds about right. Let's first assume that the QS rating for the MP1400 is relatively accurate and convert that to a 1.4GHz Xeon. 2.4/1.4 = +71%. Since processor performance does not increase linearly with clockspeed, 50% is in line with expectations. Then you throw in the fact that (1) QS ratings for slower AMD chips are understated (but overstated for the fastest chips), (2) AMD uses a point-to-point CPU/memory interface (much better for SMP) versus the P4/Xeon's shared bus, (3) Athlon architecture is more suited for DB work compared to the P4, I'd say you're lucky to see 50% more performance from a Xeon 2.4. As for HT, I've seen quite a few benchmarks where HT hurts performance. The problem is it's not only app and workload specific but also system and usage specific. As it involves the internal rescheduling of processes, adding more simultaneous processes could help to a point and then start hurting or vice-versa.
William Yu <wyu@talisys.com> writes: > Rob Sell wrote: > > > Not being one to hijack threads, but I haven't heard of this performance hit > > when using HT, I have what should all rights be a pretty fast server, dual > > 2.4 Xeons with HT 205gb raid 5 array, 1 gig of memory. And it is only 50% as > > fast as my old server which was a dual AMD MP 1400's with a 45gb raid 5 > > array and 1gb of ram. > > Not to get into a big Intel vs AMD argument but 50% sounds about right. Let's > first assume that the QS rating for the MP1400 is relatively accurate and > convert that to a 1.4GHz Xeon. 2.4/1.4 = +71%. Since processor performance > does not increase linearly with clockspeed, 50% is in line with expectations. Hm. You've read "50% as fast" as "50% faster". I wonder which the original poster intended. -- greg
Greg Stark wrote: > > William Yu <wyu@talisys.com> writes: > > > Rob Sell wrote: > > > > > Not being one to hijack threads, but I haven't heard of this performance hit > > > when using HT, I have what should all rights be a pretty fast server, dual > > > 2.4 Xeons with HT 205gb raid 5 array, 1 gig of memory. And it is only 50% as > > > fast as my old server which was a dual AMD MP 1400's with a 45gb raid 5 > > > array and 1gb of ram. > > > > Not to get into a big Intel vs AMD argument but 50% sounds about right. Let's > > first assume that the QS rating for the MP1400 is relatively accurate and > > convert that to a 1.4GHz Xeon. 2.4/1.4 = +71%. Since processor performance > > does not increase linearly with clockspeed, 50% is in line with expectations. > > Hm. You've read "50% as fast" as "50% faster". > I wonder which the original poster intended. Hyper-threading makes 2 cpus be 4 cpu's, but the 4 cpu's are each only 70% as fast, so HT is taking 2x cpus and making it 4x0.70 cpu's, which gives 2.80 cpu's, and you get that only if you are hammering all four cpu's with a full load. Imagine ifd get two cpu-bound processes on the first die (first 2 cpu's of 4) and the other CPU die is idle, and you can see that HT isn't all that useful unless you are sure to keep all 4 cpu's busy. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073