Re: Inadequate hosting for www.postgresql.org - Mailing list pgsql-www

Its interesting to see how webperf.org sees different postgresql.org
pages. For example,

http://www.webperf.org/breakdown.html?URL=www.alexey.beta.postgresql.org
Connect     DNS     FirstByte     End     Size(Bytes)     URL     Status
0.033     0.074     3.226     4.225     17654     http://www.alexey.beta.postgresql.org     200
3.258     3.258     3.854     3.933     43     http://www.alexey.beta.postgresql.org/layout/images/0.gif     200
3.379     3.379     3.850     4.356     8190     http://www.alexey.beta.postgresql.org/layout/images/logo.png     200
3.519     3.520     3.930     4.070     2101     http://www.alexey.beta.postgresql.org/layout/css/new.css     200
3.710     3.729     4.067     4.363     11470     http://ads.postgresql.org/display_image.php?lvl=sponsor     200
3.935     3.936     8.678     8.801     2644     http://www.alexey.beta.postgresql.org/news.rss     200
4.072     4.072     7.736     7.739     864     http://www.alexey.beta.postgresql.org/events.rss     200
4.228     4.228     4.496     4.499     207     http://www.alexey.beta.postgresql.org/layout/images/mark_menu.png
200

It shows, that possible problem could be news.rss, events.rss.
www.postgresql.org behaves much better
http://www.webperf.org/breakdown.html?URL=http%3A%2F%2Fwww.postgresql.org
Connect     DNS     FirstByte     End     Size(Bytes)     URL     Status
0.033     0.074     0.403     1.118     21429     http://www.postgresql.org     200
0.417     0.417     1.082     1.099     929     http://www.postgresql.org/postgresql.css     200
0.581     0.581     1.220     1.354     4447     http://www.postgresql.org/images/banner-left.jpg     200
0.700     0.700     1.211     1.336     2929     http://www.postgresql.org/images/banner-right.jpg     200
0.823     0.841     1.259     1.522     14397     http://ads.postgresql.org/display_image.php?lvl=sponsor     200
0.955     0.955     1.252     1.498     14397     http://ads.postgresql.org/display_image.php?lvl=free     200

But, I again want to  stress that current architecture of postgresql.org sites
are very-very bad and you will never get nice performance and scalability
especially for dynamic sites without:

1.  Redirecting different traffic to different servers.

     a)images and other binary things should be served by small and fast
     server like thttpd or lighttpd. Currently, images are served by
     apache+php server which is wasting of resources ! These servers
     could easily serve several thousands req/sec, so one images.postgresql.org
     could serve all such kind of traffic.

     b)requests for dynamic pages should be forwarded to backend servers
       with php compiled and keeping persistent connection to db.
       These backends are fully hidden and accessable only by localhost,
       so they will not stay busy while serving could-be-slow client
       (slow connectivity), That means resources (memory)  will be not wasted
       and there is no demand to have many servers online, because interaction
       with clients will be server only by frontends, see below

     c)requests for static pages are served by plain httpd server like apache -
       frontend. These servers should have cacheing capability to cache
       answers from dynamic servers, so many-many requests for dynamic pages
       will not actualy forwarded to heavy backends ! Our pages are not actually
       very dynamic.

     This is a classical 3 servers setup and it described in many books and
     success stories.


     Oleg
On Tue, 2 Nov 2004, Marc G. Fournier wrote:

> On Tue, 2 Nov 2004, Marc G. Fournier wrote:
>
>> On Tue, 2 Nov 2004, Alexey Borzov wrote:
>>
>>> Hi,
>>>
>>> Marc G. Fournier wrote:
>>>> You still haven't answered any of the questions that I asked in a private
>>>> email ... Is this a time of day issue (from your two samples, it looks
>>>> like it is)?  If you set it up to run hourly, what do the #s look like
>>>> for each run throughout the day?  What does the loadavg look like on the
>>>> server when you are running the script?  Or is the database being slow?
>>>
>>> I tried running mirror later today:
>>>
>>> Nov 02 12:42:47 mirror [info] Mirroring finished. 423 page(s) saved, 2354
>>> second(s) spent
>>>
>>> I also am running it now, getting the same ~5 sec per page response time
>>> and uptime command states:
>>>
>>> 7:49PM  up 10 days,  4:36, 4 users, load averages: 4.00, 3.22, 3.04
>>>
>>> The database itself does not look slow, nothing like these 5 seconds to
>>> connect / send the standard query.
>>>
>>>> Considering that Dave states above that the current script takes minutes
>>>> to generate >7000 pages, what are you doing differently that makes it so
>>>> much slower?  And, why exactly are we changing from the current method if
>>>> the new method is going to require a dedicated server to run it?
>>>
>>> Yes, that's one way to ask these questions. The other way is: why are we
>>> having response times of ~5 seconds for not-too-complex pages?
>>
>> Your guess is as good as mine ... using my browser from here, I'm not seen
>> 5sec delays, and I'm two countries away from it ... using lynx on the VM
>> itself, goign through the web site, I'm seeing instantaneous responses ...
>> I'm definitely not seeing 5sec delays ...
>>
>> Can you extend your script so that it outputs URL spider'd and time taken
>> in msec for it?  The pages that I'm testing definitely aren't giving me
>> 5sec lag times from the server itself ... so if you can give me a 'slow
>> URL' that I can look at, that would help ...
>
> 'k, I just looked at your mirror.php script .. and realized that its not the
> main web site thats the problem, its your new site ... why is your site so
> much slower to load up with lynx then the site that is there now?
> http://www.postgresql.org comes up instantly,
> http://www.alexey.beta.postgresql.org/index.html.en takes forever to load ...
> why?
>
> I can actually get coffee waiting for your site to load in my browser here,
> where there is almost zero delay loading the current site ...
>
> ----
> Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
> Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664
>
> ---------------------------(end of broadcast)---------------------------
> TIP 7: don't forget to increase your free space map settings
>

     Regards,
         Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83