Re: META Tags on Archives - Mailing list pgsql-www

On Mon, 2 Feb 2004, Jeroen Ruigrok/asmodai wrote:

> -On [20040201 23:43], Marc G. Fournier (scrappy@postgresql.org) wrote:
> >     <META NAME="robots" CONTENT="nofollow, index, archive">
>
> According to http://www.robotstxt.org/wc/meta-user.html
> archive|noarchive does not exist.
>
> Where'd you find it?

Actually, that one was in the original .resource file, but a quick search
on google shows:

http://www.bauser.com/websnob/meta/robots.html

and

http://www.google.com/webmasters/faq.html#cached

the funny thing is that this one:

http://www.katpatuka.org/pub/doc/robotexclusion.html

refers to the NOARCHIVE, but puts to:

http://www.w3.org/Search/9605-Indexing-Workshop/ReportOutcomes/Spidering.txt

which doesn't include it ...

I love standards that everyone follows *roll eyes*

I'm gathering its somethign that some use (Google does, apparently), and
some don't ...

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664