Thread: pgfoundry moved ...
Should be a noticeable improvement ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
> Should be a noticeable improvement ... Much better. A lot faster. *but*. The initial load time of each page is still pretty bad. Meaning it takes a long time for it to start sending the data back to the browser, but once it starts sending it's fast. (Though initial time is also a *lot* faster than before, that's the part that's still noticeable) //Magnus
Ya, Joshua mentioned this ... I still have a bunch of work to do now that the new server is in place, and I suspect the 'load times' you are seeing now is database related, which is one aspect I'm going to be working on over the next few days to improve ... On Thu, 28 Apr 2005, Magnus Hagander wrote: >> Should be a noticeable improvement ... > > Much better. A lot faster. *but*. The initial load time of each page is > still pretty bad. Meaning it takes a long time for it to start sending > the data back to the browser, but once it starts sending it's fast. > > (Though initial time is also a *lot* faster than before, that's the part > that's still noticeable) > > //Magnus > ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Marc, > Ya, Joshua mentioned this ... I still have a bunch of work to do now that > the new server is in place, and I suspect the 'load times' you are seeing > now is database related, which is one aspect I'm going to be working on > over the next few days to improve ... Hopefully we'll be moving pgFoundry off to the new server very soon. So don't devote a bunch of time towards it that could be used for fixing performance on www.postgresql.org. -- Josh Berkus Aglio Database Solutions San Francisco
If there are performance issues on www.postgresql.org it seems silly not to use borg. The machine is way under utilized and you have more than enough donated bandwidth to play with there. Gavin Josh Berkus wrote: >Marc, > > > >>Ya, Joshua mentioned this ... I still have a bunch of work to do now that >>the new server is in place, and I suspect the 'load times' you are seeing >>now is database related, which is one aspect I'm going to be working on >>over the next few days to improve ... >> >> > >Hopefully we'll be moving pgFoundry off to the new server very soon. So >don't devote a bunch of time towards it that could be used for fixing >performance on www.postgresql.org. > > >
> If there are performance issues on www.postgresql.org it > seems silly not to use borg. The machine is way under > utilized and you have more than enough donated bandwidth to > play with there. The only performance issues related to the base www site right now I think is the fact that I/O is horribly slow in the jail for svr2 (which runs wwwmaster) and that one is already on borg IIRC. The mirror script runs a *lot* faster on a heavily loaded linux server that I've tested it on. Not sure if it's the I/O bw on borg that's over the edge, or if it's the jails themselves causing it. I know Dave talked about comparing I/O on the main host vs the jails, not sure if he ever got through to doing it. //Magnus
Hmm, that is interesting. I have no experience in troubleshooting jails, but that server was a pgsql db server (on Gentoo Linux) prior to being borg, and has been very quick under heavy loads without any I/O issues. Gavin Magnus Hagander wrote: >>If there are performance issues on www.postgresql.org it >>seems silly not to use borg. The machine is way under >>utilized and you have more than enough donated bandwidth to >>play with there. >> >> > >The only performance issues related to the base www site right now I >think is the fact that I/O is horribly slow in the jail for svr2 (which >runs wwwmaster) and that one is already on borg IIRC. The mirror script >runs a *lot* faster on a heavily loaded linux server that I've tested it >on. Not sure if it's the I/O bw on borg that's over the edge, or if it's >the jails themselves causing it. I know Dave talked about comparing I/O >on the main host vs the jails, not sure if he ever got through to doing >it. > > >//Magnus > >---------------------------(end of broadcast)--------------------------- >TIP 9: the planner will ignore your desire to choose an index scan if your > joining column's datatypes do not match > >
On Thu, 28 Apr 2005, Josh Berkus wrote: > Marc, > >> Ya, Joshua mentioned this ... I still have a bunch of work to do now that >> the new server is in place, and I suspect the 'load times' you are seeing >> now is database related, which is one aspect I'm going to be working on >> over the next few days to improve ... > > Hopefully we'll be moving pgFoundry off to the new server very soon. So > don't devote a bunch of time towards it that could be used for fixing > performance on www.postgresql.org. Talk to Magnus about that ... he's the one maintaining/running that server ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
It is running on borg: svr2# uname -a FreeBSD svr2.postgresql.org 4.11-PRERELEASE FreeBSD 4.11-PRERELEASE #0: Thu Dec 16 10:52:33 PST 2004 root@borg:/usr/obj/usr/src/sys/GENERIC i386 On Thu, 28 Apr 2005, Gavin M. Roy wrote: > If there are performance issues on www.postgresql.org it seems silly not to > use borg. The machine is way under utilized and you have more than enough > donated bandwidth to play with there. > > Gavin > > Josh Berkus wrote: > >> Marc, >> >> >>> Ya, Joshua mentioned this ... I still have a bunch of work to do now that >>> the new server is in place, and I suspect the 'load times' you are seeing >>> now is database related, which is one aspect I'm going to be working on >>> over the next few days to improve ... >>> >> >> Hopefully we'll be moving pgFoundry off to the new server very soon. So >> don't devote a bunch of time towards it that could be used for fixing >> performance on www.postgresql.org. >> >> > > ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
On Thu, 28 Apr 2005, Gavin M. Roy wrote: > Hmm, that is interesting. I have no experience in troubleshooting jails, but > that server was a pgsql db server (on Gentoo Linux) prior to being borg, and > has been very quick under heavy loads without any I/O issues. I just removed svr2 as being a relay point for outgoing list emails ... looking at iostat 5 on that machine, the CPU was wrong close to 0 idle, whereas now seems to be closer to 100 ... might have been throwing too much into the works ;( Does that make a difference for anyone? > > Gavin > > Magnus Hagander wrote: > >>> If there are performance issues on www.postgresql.org it seems silly not >>> to use borg. The machine is way under utilized and you have more than >>> enough donated bandwidth to play with there. >>> >> >> The only performance issues related to the base www site right now I >> think is the fact that I/O is horribly slow in the jail for svr2 (which >> runs wwwmaster) and that one is already on borg IIRC. The mirror script >> runs a *lot* faster on a heavily loaded linux server that I've tested it >> on. Not sure if it's the I/O bw on borg that's over the edge, or if it's >> the jails themselves causing it. I know Dave talked about comparing I/O >> on the main host vs the jails, not sure if he ever got through to doing >> it. >> >> >> //Magnus >> >> ---------------------------(end of broadcast)--------------------------- >> TIP 9: the planner will ignore your desire to choose an index scan if your >> joining column's datatypes do not match >> > > ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
>> Hmm, that is interesting. I have no experience in >troubleshooting jails, but >> that server was a pgsql db server (on Gentoo Linux) prior to >being borg, and >> has been very quick under heavy loads without any I/O issues. > >I just removed svr2 as being a relay point for outgoing list >emails ... >looking at iostat 5 on that machine, the CPU was wrong close >to 0 idle, >whereas now seems to be closer to 100 ... might have been throwing too >much into the works ;( > >Does that make a difference for anyone? It *feels* a bit faster, but I don't have the time to run an actual mirror test script to check right now (will try to do that later). And mail relaying in general tends to load up the I/O quite a bit (lots of small syncs in different areas of the disk) I'm not sure what kind of hardware it's on. I did the following totally unscientific benchmark to get a quick figure. The comparison box is a three years old HP Proliant DL360 with 512Mb RAM and a RAID-1 disk set, that's loaded with a whole bunch of I/O intensive stuff (like mail relaying for about 20,000 users, several web hosts, some db etc). it runs linux 2.4.30 with an ext3 filesystem (totally untuned). Reference box: real 0m21.318s user 0m6.160s sys 0m17.270s borg: real 0m25.795s user 0m2.318s sys 0m20.453s svr2: real 0m33.068s user 0m2.226s sys 0m21.130s From this test it seems the jail has some overhead, around 20-25%. Not sure how to read the difference witht he reference box considering I don't know what hw is in Borg :) But as I said, it does *seem* a lot snappier now. //Magnus
On Thu, 28 Apr 2005, Magnus Hagander wrote: >>> Hmm, that is interesting. I have no experience in >> troubleshooting jails, but >>> that server was a pgsql db server (on Gentoo Linux) prior to >> being borg, and >>> has been very quick under heavy loads without any I/O issues. >> >> I just removed svr2 as being a relay point for outgoing list >> emails ... >> looking at iostat 5 on that machine, the CPU was wrong close >> to 0 idle, >> whereas now seems to be closer to 100 ... might have been throwing too >> much into the works ;( >> >> Does that make a difference for anyone? > > It *feels* a bit faster, but I don't have the time to run an actual > mirror test script to check right now (will try to do that later). And > mail relaying in general tends to load up the I/O quite a bit (lots of > small syncs in different areas of the disk) > > I'm not sure what kind of hardware it's on. I did the following totally > unscientific benchmark to get a quick figure. The comparison box is a > three years old HP Proliant DL360 with 512Mb RAM and a RAID-1 disk set, > that's loaded with a whole bunch of I/O intensive stuff (like mail > relaying for about 20,000 users, several web hosts, some db etc). it > runs linux 2.4.30 with an ext3 filesystem (totally untuned). > > > Reference box: > real 0m21.318s > user 0m6.160s > sys 0m17.270s > > borg: > real 0m25.795s > user 0m2.318s > sys 0m20.453s > > svr2: > real 0m33.068s > user 0m2.226s > sys 0m21.130s > > > From this test it seems the jail has some overhead, around 20-25%. Not > sure how to read the difference witht he reference box considering I > don't know what hw is in Borg :) But as I said, it does *seem* a lot > snappier now. The overhead that is being experienced is most likely the unionfs itself, and not the jail ... its one of the reasons I'm watching the VFS rewrite work being done by DragonFlyBSD closely, since they plan on fixing the bugs in unionfs, including improving its overall performance ... :( ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
> -----Original Message----- > From: pgsql-www-owner@postgresql.org > [mailto:pgsql-www-owner@postgresql.org] On Behalf Of Marc G. Fournier > Sent: 28 April 2005 19:43 > To: Gavin M. Roy > Cc: Magnus Hagander; Josh Berkus; Marc G. Fournier; > pgsql-www@postgresql.org > Subject: Re: [pgsql-www] pgfoundry moved ... > > I just removed svr2 as being a relay point for outgoing list > emails ... > looking at iostat 5 on that machine, the CPU was wrong close > to 0 idle, > whereas now seems to be closer to 100 ... might have been > throwing too > much into the works ;( > > Does that make a difference for anyone? Haven't checked that yet, but if you need a replacment relay, use ferengi - theres a dual 3GHz Xeon box doing nothing more than serving 1/5th of the static content for www.postgresql.org. Borg is fairly loaded in comparison. It's a dual P3 running one of the 5 static servers, as well as the master dynamic server and it's PostgreSQL backend. Regards, Dave.
> -----Original Message----- > From: pgsql-www-owner@postgresql.org > [mailto:pgsql-www-owner@postgresql.org] On Behalf Of Magnus Hagander > Sent: 28 April 2005 18:13 > To: Gavin M. Roy; Josh Berkus > Cc: Marc G. Fournier; pgsql-www@postgresql.org > Subject: Re: [pgsql-www] pgfoundry moved ... > > I know Dave talked about > comparing I/O > on the main host vs the jails, not sure if he ever got > through to doing > it. I ran Bonnie++ a few times and saw very variable performance. Both the host and svr2 peaked about the same as I recall though. As soon as we have another front end webserver, I do want to remove both svr2 and borg from the rotation. Both are very overworked compared to the other boxes. Also, Magnus was say that the bt processes on svr4 are very sluggish. Perhaps we should move them to ferengi as well. Regards, Dave.
On Thu, 28 Apr 2005, Dave Page wrote: > Haven't checked that yet, but if you need a replacment relay, use > ferengi - theres a dual 3GHz Xeon box doing nothing more than serving > 1/5th of the static content for www.postgresql.org. That would be cool ... we've got two relays right now, but spreading the load does help by keeping the queues shortened :) Can you configure it to accept connections from svr1.postgresql.org (200.46.204.71)? ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
> -----Original Message----- > From: Marc G. Fournier [mailto:scrappy@postgresql.org] > Sent: 28 April 2005 20:46 > To: Dave Page > Cc: Marc G. Fournier; Gavin M. Roy; Magnus Hagander; Josh > Berkus; pgsql-www@postgresql.org > Subject: RE: [pgsql-www] pgfoundry moved ... > > On Thu, 28 Apr 2005, Dave Page wrote: > > > Haven't checked that yet, but if you need a replacment relay, use > > ferengi - theres a dual 3GHz Xeon box doing nothing more > than serving > > 1/5th of the static content for www.postgresql.org. > > That would be cool ... we've got two relays right now, but > spreading the > load does help by keeping the queues shortened :) > > Can you configure it to accept connections from svr1.postgresql.org > (200.46.204.71)? OK, well having beaten postfix and suse firewall into shape that's ready to go now. Regards, Dave.
Is't possible to understand what's an actual problem, database or web part ? Is't possible to see timings for typical longest queries ? Probably there is some profiling support which show timings for each component used. If gbord would be Mason based applications it could be done very easy. Oleg On Thu, 28 Apr 2005, Dave Page wrote: > > >> -----Original Message----- >> From: pgsql-www-owner@postgresql.org >> [mailto:pgsql-www-owner@postgresql.org] On Behalf Of Marc G. Fournier >> Sent: 28 April 2005 19:43 >> To: Gavin M. Roy >> Cc: Magnus Hagander; Josh Berkus; Marc G. Fournier; >> pgsql-www@postgresql.org >> Subject: Re: [pgsql-www] pgfoundry moved ... >> >> I just removed svr2 as being a relay point for outgoing list >> emails ... >> looking at iostat 5 on that machine, the CPU was wrong close >> to 0 idle, >> whereas now seems to be closer to 100 ... might have been >> throwing too >> much into the works ;( >> >> Does that make a difference for anyone? > > Haven't checked that yet, but if you need a replacment relay, use > ferengi - theres a dual 3GHz Xeon box doing nothing more than serving > 1/5th of the static content for www.postgresql.org. > > Borg is fairly loaded in comparison. It's a dual P3 running one of the 5 > static servers, as well as the master dynamic server and it's PostgreSQL > backend. > > Regards, Dave. > > ---------------------------(end of broadcast)--------------------------- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/docs/faq > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
> -----Original Message----- > From: Oleg Bartunov [mailto:oleg@sai.msu.su] > Sent: 29 April 2005 05:42 > To: Dave Page > Cc: Marc G. Fournier; Gavin M. Roy; Magnus Hagander; Josh > Berkus; pgsql-www@postgresql.org > Subject: Re: [pgsql-www] pgfoundry moved ... > > Is't possible to understand what's an actual problem, database or > web part ? Is't possible to see timings for typical longest queries ? > Probably there is some profiling support which show timings for > each component used. If gbord would be Mason based > applications it could be > done very easy. We've spent time on that in the past, and nothing obvious is apparent, other than disk IO being slow in general. The same problem was seen when svr2 was on one of Marc's boxes. I'm fairly convinced it's a unionfs issue. Regards, Dave.
On Fri, 29 Apr 2005, Dave Page wrote: > > >> -----Original Message----- >> From: Oleg Bartunov [mailto:oleg@sai.msu.su] >> Sent: 29 April 2005 05:42 >> To: Dave Page >> Cc: Marc G. Fournier; Gavin M. Roy; Magnus Hagander; Josh >> Berkus; pgsql-www@postgresql.org >> Subject: Re: [pgsql-www] pgfoundry moved ... >> >> Is't possible to understand what's an actual problem, database or >> web part ? Is't possible to see timings for typical longest queries ? >> Probably there is some profiling support which show timings for >> each component used. If gbord would be Mason based >> applications it could be >> done very easy. > > We've spent time on that in the past, and nothing obvious is apparent, > other than disk IO being slow in general. The same problem was seen when > svr2 was on one of Marc's boxes. I'm fairly convinced it's a unionfs > issue. The only thing that bothers me is that it shouldn't be as apparent on borg as it was on ours, since there is only *one* VM running there, vs >30 that were running on our servers ... so unionfs shouldn't be hampering performance nearly as much ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
>> Is't possible to understand what's an actual problem, database or >> web part ? Is't possible to see timings for typical longest queries ? >> Probably there is some profiling support which show timings for >> each component used. If gbord would be Mason based >> applications it could be >> done very easy. > >We've spent time on that in the past, and nothing obvious is apparent, >other than disk IO being slow in general. The same problem was >seen when >svr2 was on one of Marc's boxes. I'm fairly convinced it's a unionfs >issue. In my experience, it's at least definitly not the db. Pages that have nothing to do with the db has been equally slow. So I'm willing to buy in with Daves guess. //Magnus
> -----Original Message----- > From: Marc G. Fournier [mailto:scrappy@postgresql.org] > Sent: 29 April 2005 15:35 > To: Dave Page > Cc: Oleg Bartunov; Marc G. Fournier; Gavin M. Roy; Magnus > Hagander; Josh Berkus; pgsql-www@postgresql.org > Subject: RE: [pgsql-www] pgfoundry moved ... > > The only thing that bothers me is that it shouldn't be as > apparent on borg > as it was on ours, since there is only *one* VM running > there, vs >30 that > were running on our servers ... so unionfs shouldn't be hampering > performance nearly as much ... Agreed, however, that's a dual PIII 1GHz box with multi-gigabytes of RAM and RAID. I can say with confidence that I've seen better responsiveness and seemingly higher performance on Linux boxes running at half that speed with a quarter of the memory. The main site build takes ages on there, much as it did on svr2 - running it here on what is effectively a P4 desktop takes about 20 minutes on a bad day. Regards, Dave
On Fri, 29 Apr 2005, Magnus Hagander wrote:
>>> Is't possible to understand what's an actual problem, database or
>>> web part ? Is't possible to see timings for typical longest queries ?
>>> Probably there is some profiling support which show timings for
>>> each component used. If gbord would be Mason based
>>> applications it could be
>>> done very easy.
>>
>> We've spent time on that in the past, and nothing obvious is apparent,
>> other than disk IO being slow in general. The same problem was
>> seen when
>> svr2 was on one of Marc's boxes. I'm fairly convinced it's a unionfs
>> issue.
>
> In my experience, it's at least definitly not the db. Pages that have
> nothing to do with the db has been equally slow. So I'm willing to buy
> in with Daves guess.
Ok, I think it's bad architecture. I already told Marc about that.
It's very easy to separate processing binary static files like images from
dynamic content. Just setup thttpd. Next step to setup frontend web server
which should be very light with cacheing capability - we use mod_accel module
for apache. It's frontentd which communicate with browser (probably slow link).
Backend, with PHP, mod_auth_pgsql should be use for page generation, it
should communicate only with frontend. The main idea is that you need much
less backends (plus postgres connection for each backend), so much lesser
resources will be used. What's the problem to setup this ?
>
> //Magnus
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postgresql.org so that your
> message can get through to the mailing list cleanly
>
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
>>>> Is't possible to understand what's an actual problem, database or >>>> web part ? Is't possible to see timings for typical >longest queries ? >>>> Probably there is some profiling support which show timings for >>>> each component used. If gbord would be Mason based >>>> applications it could be >>>> done very easy. >>> >>> We've spent time on that in the past, and nothing obvious >is apparent, >>> other than disk IO being slow in general. The same problem was >>> seen when >>> svr2 was on one of Marc's boxes. I'm fairly convinced it's a unionfs >>> issue. >> >> In my experience, it's at least definitly not the db. Pages that have >> nothing to do with the db has been equally slow. So I'm >willing to buy >> in with Daves guess. > >Ok, I think it's bad architecture. I already told Marc about that. >It's very easy to separate processing binary static files like >images from >dynamic content. Just setup thttpd. Next step to setup >frontend web server >which should be very light with cacheing capability - we use >mod_accel module >for apache. It's frontentd which communicate with browser >(probably slow link). Um. You really aren't up to speed on how things are, are you? ;-) We *do* use static frontends. Five of them actually, globally distributed. This is not where the performance problem is. >Backend, with PHP, mod_auth_pgsql should be use for page >generation, it >should communicate only with frontend. The main idea is that >you need much >less backends (plus postgres connection for each backend), so >much lesser >resources will be used. What's the problem to setup this ? Nothing - though a slightly different method is used where the frontends get their static content with rsync. But the main idea is the same. Except the backend is slow *anyway* - even with just the regen-for-frontend stuff loading it down. It doesn't show up for the end user, but it does make the site rebuild slower, whjich means it has to run less frequently (once/hour for the most often updated stuff, less often for docs and ftp tree). //Magnus
On Fri, 29 Apr 2005, Magnus Hagander wrote: >>>>> Is't possible to understand what's an actual problem, database or >>>>> web part ? Is't possible to see timings for typical >> longest queries ? >>>>> Probably there is some profiling support which show timings for >>>>> each component used. If gbord would be Mason based >>>>> applications it could be >>>>> done very easy. >>>> >>>> We've spent time on that in the past, and nothing obvious >> is apparent, >>>> other than disk IO being slow in general. The same problem was >>>> seen when >>>> svr2 was on one of Marc's boxes. I'm fairly convinced it's a unionfs >>>> issue. >>> >>> In my experience, it's at least definitly not the db. Pages that have >>> nothing to do with the db has been equally slow. So I'm >> willing to buy >>> in with Daves guess. >> >> Ok, I think it's bad architecture. I already told Marc about that. >> It's very easy to separate processing binary static files like >> images from >> dynamic content. Just setup thttpd. Next step to setup >> frontend web server >> which should be very light with cacheing capability - we use >> mod_accel module >> for apache. It's frontentd which communicate with browser >> (probably slow link). > > Um. You really aren't up to speed on how things are, are you? ;-) We > *do* use static frontends. Five of them actually, globally distributed. > This is not where the performance problem is. I see that your static frontend has PHP compiled and other stuff. curl -I http://pgfoundry.org/ HTTP/1.1 200 OK Date: Fri, 29 Apr 2005 19:04:14 GMT Server: Apache/1.3.33 (Unix) mod_auth_pgsql/0.9.12 PHP/4.3.11 X-Powered-By: PHP/4.3.11 Content-Type: text/html; charset=UTF-8 it's silly. All that stuff eat resources. Even old pentium would serve a lot of *static* pages without problem, you don't need "globally distributed" frontends. > >> Backend, with PHP, mod_auth_pgsql should be use for page >> generation, it >> should communicate only with frontend. The main idea is that >> you need much >> less backends (plus postgres connection for each backend), so >> much lesser >> resources will be used. What's the problem to setup this ? > > Nothing - though a slightly different method is used where the frontends > get their static content with rsync. But the main idea is the same. > > Except the backend is slow *anyway* - even with just the > regen-for-frontend stuff loading it down. It doesn't show up for the end > user, but it does make the site rebuild slower, whjich means it has to > run less frequently (once/hour for the most often updated stuff, less > often for docs and ftp tree). Hmm, not very nice. I don't think pgfoundry is very busy site. Is there some web stat available ? How many pages generated dynamically ? Could you run simple ab benchmark ? Technology I described is simple, commonly adopted and proven in rather busy sites (millions visitors per day). Anyway, I just tried to help. Probably, you know more information why services are so slow and has clever idea how to improve situation. > > //Magnus > > ---------------------------(end of broadcast)--------------------------- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/docs/faq > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
On Sat, 30 Apr 2005, Oleg Bartunov wrote: > Hmm, not very nice. I don't think pgfoundry is very busy site. I thought we were talking about the main web server? How did pgfoundry come into this? > Is there some web stat available ? How many pages generated dynamically ? all of pgfoundry is generated dynamically ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
>> Um. You really aren't up to speed on how things are, are you? ;-) We >> *do* use static frontends. Five of them actually, globally >distributed. >> This is not where the performance problem is. > >I see that your static frontend has PHP compiled and other stuff. > >curl -I http://pgfoundry.org/ >HTTP/1.1 200 OK >Date: Fri, 29 Apr 2005 19:04:14 GMT >Server: Apache/1.3.33 (Unix) mod_auth_pgsql/0.9.12 PHP/4.3.11 >X-Powered-By: PHP/4.3.11 >Content-Type: text/html; charset=UTF-8 I was talking about www.postgresql.org, not pgfoundry.org. I realise the subject still says pgfoundry, but the discussion was more along the performance of www.postgresql.org at this stage.. >it's silly. All that stuff eat resources. Even old pentium would >serve a lot of *static* pages without problem, you don't need >"globally distributed" >frontends. As long as it's not used, it hardly makes a difference. Anyway, we need the distribution mainly for redundancy, not performance. >> Except the backend is slow *anyway* - even with just the >> regen-for-frontend stuff loading it down. It doesn't show up >for the end >> user, but it does make the site rebuild slower, whjich means >it has to >> run less frequently (once/hour for the most often updated stuff, less >> often for docs and ftp tree). > >Hmm, not very nice. I don't think pgfoundry is very busy site. >Is there some web stat available ? How many pages generated >dynamically ? >Could you run simple ab benchmark ? Technology I described is >simple, commonly >adopted and proven in rather busy sites (millions visitors per day). >Anyway, I just tried to help. Probably, you know more information why >services are so slow and has clever idea how to improve situation. Again, we're discussing differen things. As for pgfoundry, yes, it seems to put a very high load on the systems, but I can't talk abuot any details there, since I don't know them. I'm sure there are many different ways to solve those issues. //Magnus
On Fri, 29 Apr 2005, Marc G. Fournier wrote: > On Sat, 30 Apr 2005, Oleg Bartunov wrote: > >> Hmm, not very nice. I don't think pgfoundry is very busy site. > > I thought we were talking about the main web server? How did pgfoundry come > into this? see subject :) > >> Is there some web stat available ? How many pages generated dynamically ? > > all of pgfoundry is generated dynamically ... > > ---- > Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) > Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664 > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
ok, guys. we really talked about different things :) The main web site is indeed fully static, so such slowdown usually inidcates limiting resources (sockets, open files, ...). Oleg On Fri, 29 Apr 2005, Magnus Hagander wrote: >>> Um. You really aren't up to speed on how things are, are you? ;-) We >>> *do* use static frontends. Five of them actually, globally >> distributed. >>> This is not where the performance problem is. >> >> I see that your static frontend has PHP compiled and other stuff. >> >> curl -I http://pgfoundry.org/ >> HTTP/1.1 200 OK >> Date: Fri, 29 Apr 2005 19:04:14 GMT >> Server: Apache/1.3.33 (Unix) mod_auth_pgsql/0.9.12 PHP/4.3.11 >> X-Powered-By: PHP/4.3.11 >> Content-Type: text/html; charset=UTF-8 > > I was talking about www.postgresql.org, not pgfoundry.org. I realise the > subject still says pgfoundry, but the discussion was more along the > performance of www.postgresql.org at this stage.. > > >> it's silly. All that stuff eat resources. Even old pentium would >> serve a lot of *static* pages without problem, you don't need >> "globally distributed" >> frontends. > > As long as it's not used, it hardly makes a difference. Anyway, we need > the distribution mainly for redundancy, not performance. > > >>> Except the backend is slow *anyway* - even with just the >>> regen-for-frontend stuff loading it down. It doesn't show up >> for the end >>> user, but it does make the site rebuild slower, whjich means >> it has to >>> run less frequently (once/hour for the most often updated stuff, less >>> often for docs and ftp tree). >> >> Hmm, not very nice. I don't think pgfoundry is very busy site. >> Is there some web stat available ? How many pages generated >> dynamically ? >> Could you run simple ab benchmark ? Technology I described is >> simple, commonly >> adopted and proven in rather busy sites (millions visitors per day). >> Anyway, I just tried to help. Probably, you know more information why >> services are so slow and has clever idea how to improve situation. > > Again, we're discussing differen things. > As for pgfoundry, yes, it seems to put a very high load on the systems, > but I can't talk abuot any details there, since I don't know them. I'm > sure there are many different ways to solve those issues. > > //Magnus > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
On Sat, 30 Apr 2005, Oleg Bartunov wrote: > see subject :) And it was meant to have been a simple "things should be better" note, and it tangented off to the main web server :) ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
-----Original Message----- From: Oleg Bartunov [mailto:oleg@sai.msu.su] Sent: Fri 4/29/2005 8:11 PM To: Magnus Hagander Cc: Dave Page; Marc G. Fournier; Gavin M. Roy; Josh Berkus; pgsql-www@postgresql.org Subject: Re: [pgsql-www] pgfoundry moved ... > Ok, I think it's bad architecture. I already told Marc about that. > It's very easy to separate processing binary static files like images from > dynamic content. Just setup thttpd. Next step to setup frontend web server > which should be very light with cacheing capability - we use mod_accel module > for apache. It's frontentd which communicate with browser (probably slow link). > Backend, with PHP, mod_auth_pgsql should be use for page generation, it > should communicate only with frontend. The main idea is that you need much > less backends (plus postgres connection for each backend), so much lesser > resources will be used. What's the problem to setup this ? This is irrelevant to svr2. Regards, Dave