Thread: Archives too slow

Archives too slow

From
"Greg Sabino Mullane"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


I am not happy with how slow the archives site is. It reflects
bad on PostgreSQL and supports the stereotype of PG being "slow".
For example, this page:

http://archives.postgresql.org/pgsql-hackers-win32/2004-07/msg00003.php

took 27 seconds to load completely. Some days, the times are better, but
it always takes a notieeable delay for the page to load. It also seems
to load in three sections: the top banner ad / search box, the title
and header section, and then the rest of the page. The first two load
today in about 10 seconds, the third about 16 seconds after that.
Subsequent hits tend to be much faster (disk caching one presumes), so
if you test, change the above URL to a random month, day, and message.
I don;t think it is on my end: I'm on a very fast connection, and (for
comparison) pgsql.ru loads very, very fast, and runs it searches of
the entire archives faster than arhives.postgresql.org can display
a single message.

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 200408281951
-----BEGIN PGP SIGNATURE-----

iD8DBQFBMRs3vJuQZxSWSsgRAgTqAJ4rjb9HpR+glJS4y4fGYrTLnahIxwCgjrUv
WteQbp0fd3QaZPbVb5wZM+4=
=puf4
-----END PGP SIGNATURE-----



Re: Archives too slow

From
Bruce Momjian
Date:
I can confirm the performance currently is terrible.

---------------------------------------------------------------------------

Greg Sabino Mullane wrote:
[ There is text before PGP section. ]
>
[ PGP not available, raw data follows ]
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> I am not happy with how slow the archives site is. It reflects
> bad on PostgreSQL and supports the stereotype of PG being "slow".
> For example, this page:
>
> http://archives.postgresql.org/pgsql-hackers-win32/2004-07/msg00003.php
>
> took 27 seconds to load completely. Some days, the times are better, but
> it always takes a notieeable delay for the page to load. It also seems
> to load in three sections: the top banner ad / search box, the title
> and header section, and then the rest of the page. The first two load
> today in about 10 seconds, the third about 16 seconds after that.
> Subsequent hits tend to be much faster (disk caching one presumes), so
> if you test, change the above URL to a random month, day, and message.
> I don;t think it is on my end: I'm on a very fast connection, and (for
> comparison) pgsql.ru loads very, very fast, and runs it searches of
> the entire archives faster than arhives.postgresql.org can display
> a single message.
>
> - --
> Greg Sabino Mullane greg@turnstep.com
> PGP Key: 0x14964AC8 200408281951
> -----BEGIN PGP SIGNATURE-----
>
> iD8DBQFBMRs3vJuQZxSWSsgRAgTqAJ4rjb9HpR+glJS4y4fGYrTLnahIxwCgjrUv
> WteQbp0fd3QaZPbVb5wZM+4=
> =puf4
> -----END PGP SIGNATURE-----
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
>
[ Decrypting message... End of raw data. ]

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Archives too slow

From
"Marc G. Fournier"
Date:
I'm going to look at mirroring it onto the same server that runs
ftp.postgresql.org ... archives is the worst site that we run, since its
all a bunch of little flat files, so when it gets indexed by the various
search engines, disk I/O goes through the roof ... we had googlebot index
it once where we had to literally shut down the server for a few minutes
while we waited for load to drop ...

Will work on this this weekend


On Sat, 28 Aug 2004, Greg Sabino Mullane wrote:

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> I am not happy with how slow the archives site is. It reflects
> bad on PostgreSQL and supports the stereotype of PG being "slow".
> For example, this page:
>
> http://archives.postgresql.org/pgsql-hackers-win32/2004-07/msg00003.php
>
> took 27 seconds to load completely. Some days, the times are better, but
> it always takes a notieeable delay for the page to load. It also seems
> to load in three sections: the top banner ad / search box, the title
> and header section, and then the rest of the page. The first two load
> today in about 10 seconds, the third about 16 seconds after that.
> Subsequent hits tend to be much faster (disk caching one presumes), so
> if you test, change the above URL to a random month, day, and message.
> I don;t think it is on my end: I'm on a very fast connection, and (for
> comparison) pgsql.ru loads very, very fast, and runs it searches of
> the entire archives faster than arhives.postgresql.org can display
> a single message.
>
> - --
> Greg Sabino Mullane greg@turnstep.com
> PGP Key: 0x14964AC8 200408281951
> -----BEGIN PGP SIGNATURE-----
>
> iD8DBQFBMRs3vJuQZxSWSsgRAgTqAJ4rjb9HpR+glJS4y4fGYrTLnahIxwCgjrUv
> WteQbp0fd3QaZPbVb5wZM+4=
> =puf4
> -----END PGP SIGNATURE-----
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
>

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664

Re: Archives too slow

From
"Greg Sabino Mullane"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


> I'm going to look at mirroring it onto the same server that runs
> ftp.postgresql.org ... archives is the worst site that we run, since its
> all a bunch of little flat files, so when it gets indexed by the various
> search engines, disk I/O goes through the roof ... we had googlebot index
> it once where we had to literally shut down the server for a few minutes
> while we waited for load to drop ...

On googlebot's page[1], they claim they never go more than once every few
seconds. Surely this should not be a problem as long as these are static
pages. They also have an email address on that page where you can request
that Google go a little gentler on your site.

Also, if the pages are static (or static plus simple cgis), have you considered
using boa? [2] I use it for a large site that has a lot of static pages
and it does great - it's a small, clean, minimal web server written in C.

A final option is an accelerator cache [3]. Not sure if PG is using one
yet, but it probably should be.

[1] http://www.google.com/bot.html

[2] http://www.boa.org/

[3] http://www.squid-cache.org/Doc/FAQ/FAQ-20.html#what-is-httpd-accelerator

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 200408290732
-----BEGIN PGP SIGNATURE-----

iD8DBQFBMcEGvJuQZxSWSsgRAhK9AKCLQ4CIW2JQQDg+BFI12DyhaFiFVgCg1DW5
VkM2ayTI9OK6M1kIscAwgxs=
=1uGd
-----END PGP SIGNATURE-----



Re: Archives too slow

From
Oleg Bartunov
Date:
I always didn't understand why not have static pages for mailing list
archive if they already generated.

Another thing to consider is using
lightweight frontend with ability to cache pages generated by heavy backend.
www.pgsql.ru uses 3-servers setup - frontend (apache+mod_accel),
backend ( apache + modperl ) and thttpd (very light and fast) for serving
binary data (images, for example). Only frontend interacts with user, so slow
clients (bad connectivity) don't bother heavy backend and, consequently,
db server.


    Oleg
On Sun, 29 Aug 2004, Greg Sabino Mullane wrote:

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> > I'm going to look at mirroring it onto the same server that runs
> > ftp.postgresql.org ... archives is the worst site that we run, since its
> > all a bunch of little flat files, so when it gets indexed by the various
> > search engines, disk I/O goes through the roof ... we had googlebot index
> > it once where we had to literally shut down the server for a few minutes
> > while we waited for load to drop ...
>
> On googlebot's page[1], they claim they never go more than once every few
> seconds. Surely this should not be a problem as long as these are static
> pages. They also have an email address on that page where you can request
> that Google go a little gentler on your site.
>
> Also, if the pages are static (or static plus simple cgis), have you considered
> using boa? [2] I use it for a large site that has a lot of static pages
> and it does great - it's a small, clean, minimal web server written in C.
>
> A final option is an accelerator cache [3]. Not sure if PG is using one
> yet, but it probably should be.
>
> [1] http://www.google.com/bot.html
>
> [2] http://www.boa.org/
>
> [3] http://www.squid-cache.org/Doc/FAQ/FAQ-20.html#what-is-httpd-accelerator
>
> - --
> Greg Sabino Mullane greg@turnstep.com
> PGP Key: 0x14964AC8 200408290732
> -----BEGIN PGP SIGNATURE-----
>
> iD8DBQFBMcEGvJuQZxSWSsgRAhK9AKCLQ4CIW2JQQDg+BFI12DyhaFiFVgCg1DW5
> VkM2ayTI9OK6M1kIscAwgxs=
> =1uGd
> -----END PGP SIGNATURE-----
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
>                http://archives.postgresql.org
>

    Regards,
        Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

Re: Archives too slow

From
"Marc G. Fournier"
Date:
The reason why they are dynamic was so that we could 'strip' the garbage
out for the search engines, so that they didn't search all of the extra
verbage, only the message itself ... it also meant that if we changed the
format (ie. added a new list to the left menu), we didn't have to
regenerate all the pages ...

I'm open to moving them back to static myself, if nobody objects *shrug*

In fact, if we get rid of the php code altogether, we could go with what
one other suggested, and use a 'light weight' web server instead of apache
... you mention thttpd below, someone else mentioned one called Boa(?) ...
never having used either, I'm flexible either way ...

Does anyone care if I get rid of the .php code?  Before I do that,
assuming no ... does anyone know a way of 'hiding' sections of HTML code
from search engines?  Right now, we're doing that with the PHP, and I nkow
there is/was a <!-- --> way of doing it, but someone (Oleg?) mentioned
that it isn't very consistent in being honored by search engines ... ?




On Sun, 29 Aug 2004, Oleg Bartunov wrote:

> I always didn't understand why not have static pages for mailing list
> archive if they already generated.
>
> Another thing to consider is using
> lightweight frontend with ability to cache pages generated by heavy backend.
> www.pgsql.ru uses 3-servers setup - frontend (apache+mod_accel),
> backend ( apache + modperl ) and thttpd (very light and fast) for serving
> binary data (images, for example). Only frontend interacts with user, so slow
> clients (bad connectivity) don't bother heavy backend and, consequently,
> db server.
>
>
>     Oleg
> On Sun, 29 Aug 2004, Greg Sabino Mullane wrote:
>
>>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>>
>>> I'm going to look at mirroring it onto the same server that runs
>>> ftp.postgresql.org ... archives is the worst site that we run, since its
>>> all a bunch of little flat files, so when it gets indexed by the various
>>> search engines, disk I/O goes through the roof ... we had googlebot index
>>> it once where we had to literally shut down the server for a few minutes
>>> while we waited for load to drop ...
>>
>> On googlebot's page[1], they claim they never go more than once every few
>> seconds. Surely this should not be a problem as long as these are static
>> pages. They also have an email address on that page where you can request
>> that Google go a little gentler on your site.
>>
>> Also, if the pages are static (or static plus simple cgis), have you considered
>> using boa? [2] I use it for a large site that has a lot of static pages
>> and it does great - it's a small, clean, minimal web server written in C.
>>
>> A final option is an accelerator cache [3]. Not sure if PG is using one
>> yet, but it probably should be.
>>
>> [1] http://www.google.com/bot.html
>>
>> [2] http://www.boa.org/
>>
>> [3] http://www.squid-cache.org/Doc/FAQ/FAQ-20.html#what-is-httpd-accelerator
>>
>> - --
>> Greg Sabino Mullane greg@turnstep.com
>> PGP Key: 0x14964AC8 200408290732
>> -----BEGIN PGP SIGNATURE-----
>>
>> iD8DBQFBMcEGvJuQZxSWSsgRAhK9AKCLQ4CIW2JQQDg+BFI12DyhaFiFVgCg1DW5
>> VkM2ayTI9OK6M1kIscAwgxs=
>> =1uGd
>> -----END PGP SIGNATURE-----
>>
>>
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 6: Have you searched our list archives?
>>
>>                http://archives.postgresql.org
>>
>
>     Regards,
>         Oleg
> _____________________________________________________________
> Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
> Sternberg Astronomical Institute, Moscow University (Russia)
> Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
> phone: +007(095)939-16-83, +007(095)939-23-83
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: the planner will ignore your desire to choose an index scan if your
>      joining column's datatypes do not match
>

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664

Re: Archives too slow

From
Oleg Bartunov
Date:
On Sun, 29 Aug 2004, Marc G. Fournier wrote:

>
> The reason why they are dynamic was so that we could 'strip' the garbage
> out for the search engines, so that they didn't search all of the extra
> verbage, only the message itself ... it also meant that if we changed the
> format (ie. added a new list to the left menu), we didn't have to
> regenerate all the pages ...

You always could use SSI for that.


>
> I'm open to moving them back to static myself, if nobody objects *shrug*
>
> In fact, if we get rid of the php code altogether, we could go with what
> one other suggested, and use a 'light weight' web server instead of apache
> ... you mention thttpd below, someone else mentioned one called Boa(?) ...
> never having used either, I'm flexible either way ...
>

thttpd is nice for simple binary data, but currently we evaluate lighttpd
http://jan.kneschke.de/projects/lighttpd/, which looks realy powerful.
It's faster than thttpd having much more features.



> Does anyone care if I get rid of the .php code?  Before I do that,
> assuming no ... does anyone know a way of 'hiding' sections of HTML code
> from search engines?  Right now, we're doing that with the PHP, and I nkow
> there is/was a <!-- --> way of doing it, but someone (Oleg?) mentioned
> that it isn't very consistent in being honored by search engines ... ?
>

<!-- --> doesn't help, because comment tags also hide content from browser :)
In principle, smart search engines should understand firm elements
like navigation bar and penalize their weight. We do that, at least.



>
>
>
> On Sun, 29 Aug 2004, Oleg Bartunov wrote:
>
> > I always didn't understand why not have static pages for mailing list
> > archive if they already generated.
> >
> > Another thing to consider is using
> > lightweight frontend with ability to cache pages generated by heavy backend.
> > www.pgsql.ru uses 3-servers setup - frontend (apache+mod_accel),
> > backend ( apache + modperl ) and thttpd (very light and fast) for serving
> > binary data (images, for example). Only frontend interacts with user, so slow
> > clients (bad connectivity) don't bother heavy backend and, consequently,
> > db server.
> >
> >
> >     Oleg
> > On Sun, 29 Aug 2004, Greg Sabino Mullane wrote:
> >
> >>
> >> -----BEGIN PGP SIGNED MESSAGE-----
> >> Hash: SHA1
> >>
> >>
> >>> I'm going to look at mirroring it onto the same server that runs
> >>> ftp.postgresql.org ... archives is the worst site that we run, since its
> >>> all a bunch of little flat files, so when it gets indexed by the various
> >>> search engines, disk I/O goes through the roof ... we had googlebot index
> >>> it once where we had to literally shut down the server for a few minutes
> >>> while we waited for load to drop ...
> >>
> >> On googlebot's page[1], they claim they never go more than once every few
> >> seconds. Surely this should not be a problem as long as these are static
> >> pages. They also have an email address on that page where you can request
> >> that Google go a little gentler on your site.
> >>
> >> Also, if the pages are static (or static plus simple cgis), have you considered
> >> using boa? [2] I use it for a large site that has a lot of static pages
> >> and it does great - it's a small, clean, minimal web server written in C.
> >>
> >> A final option is an accelerator cache [3]. Not sure if PG is using one
> >> yet, but it probably should be.
> >>
> >> [1] http://www.google.com/bot.html
> >>
> >> [2] http://www.boa.org/
> >>
> >> [3] http://www.squid-cache.org/Doc/FAQ/FAQ-20.html#what-is-httpd-accelerator
> >>
> >> - --
> >> Greg Sabino Mullane greg@turnstep.com
> >> PGP Key: 0x14964AC8 200408290732
> >> -----BEGIN PGP SIGNATURE-----
> >>
> >> iD8DBQFBMcEGvJuQZxSWSsgRAhK9AKCLQ4CIW2JQQDg+BFI12DyhaFiFVgCg1DW5
> >> VkM2ayTI9OK6M1kIscAwgxs=
> >> =1uGd
> >> -----END PGP SIGNATURE-----
> >>
> >>
> >>
> >> ---------------------------(end of broadcast)---------------------------
> >> TIP 6: Have you searched our list archives?
> >>
> >>                http://archives.postgresql.org
> >>
> >
> >     Regards,
> >         Oleg
> > _____________________________________________________________
> > Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
> > Sternberg Astronomical Institute, Moscow University (Russia)
> > Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
> > phone: +007(095)939-16-83, +007(095)939-23-83
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 9: the planner will ignore your desire to choose an index scan if your
> >      joining column's datatypes do not match
> >
>
> ----
> Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
> Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664
>
> ---------------------------(end of broadcast)---------------------------
> TIP 8: explain analyze is your friend
>

    Regards,
        Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

Re: Archives too slow

From
"John Hansen"
Date:
<!-- --> doesn't help, because comment tags also hide content from browser :)
In principle, smart search engines should understand firm elements
like navigation bar and penalize their weight. We do that, at least.

I think you misunderstood this.
The comment tags do not hide things from the browser, think of this:
 
<html>
...
<!--noindex-->
stuff we want hidden
<!--/noindex-->
stuff we don't want hidden
...
</html>
 
... John

Re: Archives too slow

From
Josh Berkus
Date:
Folks:

Archives are more than slow now.  They're down.   And the rest of
www.postgresql.org is molasses-slow as well.

I have another suggestion:  Command Prompt has offered to host the archives;
they have excess capacity and believe that they can make them run faster.
Why not let them?

--
Josh Berkus
Aglio Database Solutions
San Francisco

Re: Archives too slow

From
"Marc G. Fournier"
Date:
On Mon, 30 Aug 2004, Josh Berkus wrote:

> Folks:
>
> Archives are more than slow now.  They're down.   And the rest of
> www.postgresql.org is molasses-slow as well.
>
> I have another suggestion:  Command Prompt has offered to host the archives;
> they have excess capacity and believe that they can make them run faster.
> Why not let them?

'k, I must have missed the offer, but sure ... I can work with Joshua to
get the scripts and all put into place ...

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664

Re: Archives too slow

From
Bruce Momjian
Date:
Marc G. Fournier wrote:
> On Mon, 30 Aug 2004, Josh Berkus wrote:
>
> > Folks:
> >
> > Archives are more than slow now.  They're down.   And the rest of
> > www.postgresql.org is molasses-slow as well.
> >
> > I have another suggestion:  Command Prompt has offered to host the archives;
> > they have excess capacity and believe that they can make them run faster.
> > Why not let them?
>
> 'k, I must have missed the offer, but sure ... I can work with Joshua to
> get the scripts and all put into place ...

Agreed.  I got the offer from Joshua Drake via private email but was not
sure how to address the issue.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Archives too slow

From
Oleg Bartunov
Date:
On Tue, 31 Aug 2004, John Hansen wrote:

> <!-- --> doesn't help, because comment tags also hide content from browser :)
> In principle, smart search engines should understand firm elements
> like navigation bar and penalize their weight. We do that, at least.
>
> I think you misunderstood this.
> The comment tags do not hide things from the browser, think of this:
>
> <html>
> ...
> <!--noindex-->
> stuff we want hidden
> <!--/noindex-->
> stuff we don't want hidden
> ...
> </html>

Aha, got that. I don't know such syntax and not sure if it's well supported.
I know about <NOINDEX>.....</NOINDEX> by atomz.com, though


>
> ... John
>

    Regards,
        Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

Re: Archives too slow

From
"Dave Page"
Date:

> -----Original Message-----
> From: pgsql-www-owner@postgresql.org
> [mailto:pgsql-www-owner@postgresql.org] On Behalf Of Oleg Bartunov
> Sent: 30 August 2004 20:41
> To: John Hansen
> Cc: Marc G. Fournier; Greg Sabino Mullane; pgsql-www@postgresql.org
> Subject: Re: [pgsql-www] Archives too slow
>
> Aha, got that. I don't know such syntax and not sure if it's
> well supported.
> I know about <NOINDEX>.....</NOINDEX> by atomz.com, though

That's the problem I think - there is no standard, just lot's of
variations :-(

/D

Re: Archives too slow

From
"Marc G. Fournier"
Date:
On Mon, 30 Aug 2004, Bruce Momjian wrote:

> Marc G. Fournier wrote:
>> On Mon, 30 Aug 2004, Josh Berkus wrote:
>>
>>> Folks:
>>>
>>> Archives are more than slow now.  They're down.   And the rest of
>>> www.postgresql.org is molasses-slow as well.
>>>
>>> I have another suggestion:  Command Prompt has offered to host the archives;
>>> they have excess capacity and believe that they can make them run faster.
>>> Why not let them?
>>
>> 'k, I must have missed the offer, but sure ... I can work with Joshua to
>> get the scripts and all put into place ...
>
> Agreed.  I got the offer from Joshua Drake via private email but was not
> sure how to address the issue.

Joshua and I are talking about it ... :)

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664

Re: Archives too slow

From
"Marc G. Fournier"
Date:
On Mon, 30 Aug 2004, Oleg Bartunov wrote:

> On Tue, 31 Aug 2004, John Hansen wrote:
>
>> <!-- --> doesn't help, because comment tags also hide content from browser :)
>> In principle, smart search engines should understand firm elements
>> like navigation bar and penalize their weight. We do that, at least.
>>
>> I think you misunderstood this.
>> The comment tags do not hide things from the browser, think of this:
>>
>> <html>
>> ...
>> <!--noindex-->
>> stuff we want hidden
>> <!--/noindex-->
>> stuff we don't want hidden
>> ...
>> </html>
>
> Aha, got that. I don't know such syntax and not sure if it's well supported.
> I know about <NOINDEX>.....</NOINDEX> by atomz.com, though

Does Google have an equivalent that they honor?


----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664

Re: Archives too slow

From
"John Hansen"
Date:
> >
> > Aha, got that. I don't know such syntax and not sure if
> it's well supported.
> > I know about <NOINDEX>.....</NOINDEX> by atomz.com, though
>
> Does Google have an equivalent that they honor?
>

I've asked google, and their reply was consistent with the information available on their website.
Google does not allow cloaking, and any attempt to do so, will result in a permanent ban.

They define cloaking as presenting different content to search engines as to what regular visitors see.

... John


Re: Archives too slow

From
"Marc G. Fournier"
Date:
On Wed, 1 Sep 2004, John Hansen wrote:

>>>
>>> Aha, got that. I don't know such syntax and not sure if
>> it's well supported.
>>> I know about <NOINDEX>.....</NOINDEX> by atomz.com, though
>>
>> Does Google have an equivalent that they honor?
>>
>
> I've asked google, and their reply was consistent with the information
> available on their website. Google does not allow cloaking, and any
> attempt to do so, will result in a permanent ban.
>
> They define cloaking as presenting different content to search engines
> as to what regular visitors see.

Guess they don't check often then since we've been doing it for months now
:)

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664

Re: Archives too slow

From
"John Hansen"
Date:
> >
> > They define cloaking as presenting different content to
> search engines
> > as to what regular visitors see.
>
> Guess they don't check often then since we've been doing it
> for months now
> :)
>

Yea, I think tho, that it's meant as a way to discourage so-called SEO's from creating pages whose sole purpose is to
skewthe 
ranking.

One could for example just join all these offers of link exchanges one gets, and just not show them to the search
engines.That 
would improve your own ranking in a positive way, hehe.

... John


Btw, marc, could you tar.bz2 up the archives for me,... I'm moving machines and will need to recrawl all of it.