Thread: Search points to ancient manuals

Search points to ancient manuals

From
Josh Berkus
Date:
1. go to postgresql.org home page.

2. type "compile postgres" in the search box.

3. the first result will be the installation chapter from the 6.5
version of the manual.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com



Re: Search points to ancient manuals

From
"Kevin Grittner"
Date:
Josh Berkus wrote:

> 1. go to postgresql.org home page.
> 
> 2. type "compile postgres" in the search box.
> 
> 3. the first result will be the installation chapter from the 6.5
> version of the manual.

Of course, if you search for "compile postgresql" the top two entries
are for 9.2. Perhaps we should make those equivalent in searches?

-Kevin



Re: Search points to ancient manuals

From
Josh Berkus
Date:
On 10/30/12 3:53 PM, Kevin Grittner wrote:
> Josh Berkus wrote:
> 
>> 1. go to postgresql.org home page.
>>
>> 2. type "compile postgres" in the search box.
>>
>> 3. the first result will be the installation chapter from the 6.5
>> version of the manual.
> 
> Of course, if you search for "compile postgresql" the top two entries
> are for 9.2. Perhaps we should make those equivalent in searches?

Well, we could really use some way to make docs other than the current
version have a much lower score, automatically.  I don't know how to
implement this, though.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com



Re: Search points to ancient manuals

From
Magnus Hagander
Date:
<p dir="ltr"><br /> On Oct 31, 2012 6:51 PM, "Josh Berkus" <<a
href="mailto:josh@agliodbs.com">josh@agliodbs.com</a>>wrote:<br /> ><br /> > On 10/30/12 3:53 PM, Kevin
Grittnerwrote:<br /> > > Josh Berkus wrote:<br /> > ><br /> > >> 1. go to <a
href="http://postgresql.org">postgresql.org</a>home page.<br /> > >><br /> > >> 2. type "compile
postgres"in the search box.<br /> > >><br /> > >> 3. the first result will be the installation
chapterfrom the 6.5<br /> > >> version of the manual.<br /> > ><br /> > > Of course, if you search
for"compile postgresql" the top two entries<br /> > > are for 9.2. Perhaps we should make those equivalent in
searches?<br/> ><br /> > Well, we could really use some way to make docs other than the current<br /> >
versionhave a much lower score, automatically.  I don't know how to<br /> > implement this, though.<br /><p
dir="ltr">Wealready do this for Google, so it shouldn't be rocket science.<p dir="ltr">At one point we just didn't
indexthe unsupported versions at all, but we turned that back on when we realized the cure was worse than the disease.
However,it seems we never got around to downing the priority of the old docs. <p dir="ltr">It's just so a sql query of
course,so patches welcome ;) <p dir="ltr">/Magnus <br /> 

Re: Search points to ancient manuals

From
Craig Ringer
Date:
<div class="moz-cite-prefix">On 11/01/2012 06:00 PM, Magnus Hagander wrote:<br /></div><blockquote
cite="mid:CABUevEyTXVBJQsqKCNWFz1CA_2QQoU0CsWeVX5wMzQbqc_RuyA@mail.gmail.com"type="cite">We already do this for Google,
soit shouldn't be rocket science.</blockquote> I still routinely see 8.1 / 8.2 / 8.3 results in searches. For example,
myfirst hit for a Google search for EXPLAIN ANALYZE is <br /><br /><a
href="http://www.postgresql.org/docs/8.1/static/sql-explain.html">http://www.postgresql.org/docs/8.1/static/sql-explain.html</a><br
/><br/> I've often wished that the docs would redirect to the /current/interactive version when the referer is google.
Notenough to write the code yet, though.<br /><br /> --<br /> Craig Ringer<br /> 

Re: Search points to ancient manuals

From
Dave Page
Date:
On Thu, Nov 1, 2012 at 11:13 AM, Craig Ringer <ringerc@ringerc.id.au> wrote:
> On 11/01/2012 06:00 PM, Magnus Hagander wrote:
>
> We already do this for Google, so it shouldn't be rocket science.
>
> I still routinely see 8.1 / 8.2 / 8.3 results in searches. For example, my
> first hit for a Google search for EXPLAIN ANALYZE is
>
> http://www.postgresql.org/docs/8.1/static/sql-explain.html
>
> I've often wished that the docs would redirect to the /current/interactive
> version when the referer is google. Not enough to write the code yet,
> though.

Good grief no! I often do things like:
https://www.google.com/search?q=explain+analyze+postgresql+8.3

What would be nice would be for Google to pay more attention to the
ranking hints we already give them in the site map.

-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: Search points to ancient manuals

From
Marti Raudsepp
Date:
On Thu, Nov 1, 2012 at 1:13 PM, Craig Ringer <ringerc@ringerc.id.au> wrote:
> I've often wished that the docs would redirect to the /current/interactive version when the referer is google. Not
enoughto write the code yet, though.
 

That solution is backwards. You click on the Google link that says
"Documentation: 8.1: EXPLAIN", but when you click on it, you magically
end up in PostgreSQL 9.2 docs.

Instead we should encourage or force Googlebot (and other search
engines) to index/prefer the current version of docs. I hear the
sitemaps file already gives higher priority to current, but clearly
that's not having the effect it should. I suspect most of our
referrers link to versioned docs, which boosts their rank, and the
search engine has no way of telling that the individual versions are
mostly equivalent.

We might detect search engines by their User-Agent header and always
serve them redirects to /current/ (if the page exists in current) or
the latest version. But that would have the effect of de-indexing old
versions of pages that have been revised -- which isn't an issue most
of the time, but is not ideal either.

Also it would require us to serve docs pages with "Vary: User-Agent",
which would make caching less efficient (but maybe there are
workarounds?)

Any thoughts?

Regards,
Marti



Re: Search points to ancient manuals

From
Greg Stark
Date:
On Thu, Nov 1, 2012 at 11:35 AM, Marti Raudsepp <marti@juffo.org> wrote:
> On Thu, Nov 1, 2012 at 1:13 PM, Craig Ringer <ringerc@ringerc.id.au> wrote:
>> I've often wished that the docs would redirect to the /current/interactive version when the referer is google. Not
enoughto write the code yet, though. 
>
> That solution is backwards. You click on the Google link that says
> "Documentation: 8.1: EXPLAIN", but when you click on it, you magically
> end up in PostgreSQL 9.2 docs.

You can actually get the search string from the referrer header and
see if 8.1 is in it I suppose. But that's still not very good. The
user might have seen multiple versions in their search results and
chosen a specific version they wanted. Being redirected to a page
different from what the search engine saw and showed the snippet is a
bad idea.

> Instead we should encourage or force Googlebot (and other search
> engines) to index/prefer the current version of docs. I hear the
> sitemaps file already gives higher priority to current, but clearly
> that's not having the effect it should. I suspect most of our

The documentation on sitemap.org isn't very clear on what this priority does:

>> The priority of this URL relative to other URLs on your site. Valid values range from 0.0 to 1.0. This value does
not
>> affect how your pages are compared to pages on other sites—it only lets the search engines know which pages
>> you deem most important for the crawlers.
>>
>> The default priority of a page is 0.5.
>>
>> Please note that the priority you assign to a page is not likely to influence the position of your URLs in a search
>> engine's result pages. Search engines may use this information when selecting between URLs on the same site,
>> so you can use this tag to increase the likelihood that your most important pages are present in a search index.
>>
>> Also, please note that assigning a high priority to all of the URLs on your site is not likely to help you. Since
the
>> priority is relative, it is only used to select between URLs on your site.

There are multiple suggestions here about what the priority might
mean. Of course it's up to the individual search engines how they use
this information.

Keep in mind that Google might have an opinion about which results are
most relevant to the user. When 9.2 is released it might be the case
that most uses are actually still looking for 9.1. It may even be the
case that most users today are currently looking for 8.4 or 9.0 due to
some popular distribution shipping it.


--
greg



Re: Search points to ancient manuals

From
Magnus Hagander
Date:
On Thu, Nov 1, 2012 at 12:35 PM, Marti Raudsepp <marti@juffo.org> wrote:
On Thu, Nov 1, 2012 at 1:13 PM, Craig Ringer <ringerc@ringerc.id.au> wrote:
> I've often wished that the docs would redirect to the /current/interactive version when the referer is google. Not enough to write the code yet, though.

That solution is backwards. You click on the Google link that says
"Documentation: 8.1: EXPLAIN", but when you click on it, you magically
end up in PostgreSQL 9.2 docs.


Yeah, I htink that's a really bad idea.

 
Instead we should encourage or force Googlebot (and other search
engines) to index/prefer the current version of docs. I hear the
sitemaps file already gives higher priority to current, but clearly
that's not having the effect it should. I suspect most of our
referrers link to versioned docs, which boosts their rank, and the
search engine has no way of telling that the individual versions are
mostly equivalent.

Yeah, that's my impression as well. Once we put the sitemap into place properly, after some time (a month or so I think) the *obsolete* documentation links started to disappear from the google search hits (they're not in the sitemap at all). But it seems to pay no attention whatsoever to the priorities given in the sitemap.

 
We might detect search engines by their User-Agent header and always
serve them redirects to /current/ (if the page exists in current) or
the latest version. But that would have the effect of de-indexing old
versions of pages that have been revised -- which isn't an issue most
of the time, but is not ideal either.

If we want to do that, we can remove the old versions from the sitemap, which is a much easier way of doing it. However, that would remove any way of searching for the old documentation there - and there are certainly many usecases where doing that makes sense.
 

Also it would require us to serve docs pages with "Vary: User-Agent",
which would make caching less efficient (but maybe there are
workarounds?)

Not really, we could fairly easily work around that specifically for the docs. No need to use the big gun of setting vary:user-agent on every page (even every docs page).

However, I still think it's a non-starter since it would make it impossible to search for old stuff. 

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: Search points to ancient manuals

From
Magnus Hagander
Date:
On Thu, Nov 1, 2012 at 7:04 PM, Greg Stark <stark@mit.edu> wrote:
On Thu, Nov 1, 2012 at 11:35 AM, Marti Raudsepp <marti@juffo.org> wrote:
> On Thu, Nov 1, 2012 at 1:13 PM, Craig Ringer <ringerc@ringerc.id.au> wrote:
>> I've often wished that the docs would redirect to the /current/interactive version when the referer is google. Not enough to write the code yet, though.
>
> That solution is backwards. You click on the Google link that says
> "Documentation: 8.1: EXPLAIN", but when you click on it, you magically
> end up in PostgreSQL 9.2 docs.

You can actually get the search string from the referrer header and
see if 8.1 is in it I suppose. But that's still not very good. The
user might have seen multiple versions in their search results and
chosen a specific version they wanted. Being redirected to a page
different from what the search engine saw and showed the snippet is a
bad idea.

Yes.
 

> Instead we should encourage or force Googlebot (and other search
> engines) to index/prefer the current version of docs. I hear the
> sitemaps file already gives higher priority to current, but clearly
> that's not having the effect it should. I suspect most of our

The documentation on sitemap.org isn't very clear on what this priority does:

>> The priority of this URL relative to other URLs on your site. Valid values range from 0.0 to 1.0. This value does not
>> affect how your pages are compared to pages on other sites—it only lets the search engines know which pages
>> you deem most important for the crawlers.
>>
>> The default priority of a page is 0.5.
>>
>> Please note that the priority you assign to a page is not likely to influence the position of your URLs in a search
>> engine's result pages. Search engines may use this information when selecting between URLs on the same site,
>> so you can use this tag to increase the likelihood that your most important pages are present in a search index.
>>
>> Also, please note that assigning a high priority to all of the URLs on your site is not likely to help you. Since the
>> priority is relative, it is only used to select between URLs on your site.

There are multiple suggestions here about what the priority might
mean. Of course it's up to the individual search engines how they use
this information.

Yeah, that's exactly the problem. It was originally supposed to work for prioritising between pages on the same site, but it seems at least google trusts their own algorithms a lot more (which may not be a bad thing in general, but it's a bad thing for us at this point)
 

Keep in mind that Google might have an opinion about which results are
most relevant to the user. When 9.2 is released it might be the case
that most uses are actually still looking for 9.1. It may even be the
case that most users today are currently looking for 8.4 or 9.0 due to
some popular distribution shipping it.

That's probably true, and it's definitely something that changes over time... 

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: Search points to ancient manuals

From
Craig Ringer
Date:
On 11/01/2012 07:35 PM, Marti Raudsepp wrote:
> On Thu, Nov 1, 2012 at 1:13 PM, Craig Ringer <ringerc@ringerc.id.au> wrote:
>> I've often wished that the docs would redirect to the /current/interactive version when the referer is google. Not
enoughto write the code yet, though.
 
> 
> That solution is backwards. You click on the Google link that says
> "Documentation: 8.1: EXPLAIN", but when you click on it, you magically
> end up in PostgreSQL 9.2 docs.

You're right, especially with people searching for a specific version or
choosing a specific version from the hit list.

It was a poorly thought out suggestion.

> I suspect most of our
> referrers link to versioned docs, which boosts their rank, and the
> search engine has no way of telling that the individual versions are
> mostly equivalent.

I'd say so, yes. That's partly because when you visit the docs site, you
have to pick a version. You have to edit the URL to get /current .

> We might detect search engines by their User-Agent header and always
> serve them redirects to /current/ (if the page exists in current) or
> the latest version. But that would have the effect of de-indexing old
> versions of pages that have been revised -- which isn't an issue most
> of the time, but is not ideal either.

Yep; in particular, that's bad when the old versions have significantly
different info, as with `custom_variable_classes`, which is gone in 9.2,
or the `shared_buffers` changes for 9.3.

Maybe if the user visits old docs there just needs to be a banner with a
prominent "This documentation is specific to version xx of PostgreSQL;
if this is not the version you are using please choose yours from the
links above."

--
Craig Ringer



Re: Search points to ancient manuals

From
Magnus Hagander
Date:

On Fri, Nov 2, 2012 at 1:43 PM, Craig Ringer <ringerc@ringerc.id.au> wrote:
On 11/01/2012 07:35 PM, Marti Raudsepp wrote:
> On Thu, Nov 1, 2012 at 1:13 PM, Craig Ringer <ringerc@ringerc.id.au> wrote:> I suspect most of our
> referrers link to versioned docs, which boosts their rank, and the
> search engine has no way of telling that the individual versions are
> mostly equivalent.

I'd say so, yes. That's partly because when you visit the docs site, you
have to pick a version. You have to edit the URL to get /current .

*That* is something we could certainly fix. We could add a "current" link to all documentation pages. To the left of "9.2" (or whatever happens to be the latst).

That'll also seed the site with lots of links to the /current/ version. I'm not sure if that actually helps the search engines, though.

 

> We might detect search engines by their User-Agent header and always
> serve them redirects to /current/ (if the page exists in current) or
> the latest version. But that would have the effect of de-indexing old
> versions of pages that have been revised -- which isn't an issue most
> of the time, but is not ideal either.

Yep; in particular, that's bad when the old versions have significantly
different info, as with `custom_variable_classes`, which is gone in 9.2,
or the `shared_buffers` changes for 9.3.

Maybe if the user visits old docs there just needs to be a banner with a
prominent "This documentation is specific to version xx of PostgreSQL;
if this is not the version you are using please choose yours from the
links above."


How intrusive do we want to make that? I think just having the links up there has actually made it a lot more user friendly in that regard already... 


--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: Search points to ancient manuals

From
Josh Kupershmidt
Date:
On Fri, Nov 2, 2012 at 7:36 AM, Magnus Hagander <magnus@hagander.net> wrote:
> On Fri, Nov 2, 2012 at 1:43 PM, Craig Ringer <ringerc@ringerc.id.au> wrote:
>> Maybe if the user visits old docs there just needs to be a banner with a
>> prominent "This documentation is specific to version xx of PostgreSQL;
>> if this is not the version you are using please choose yours from the
>> links above."

> How intrusive do we want to make that? I think just having the links up
> there has actually made it a lot more user friendly in that regard
> already...

IMO the "This page in other versions" links work pretty well as-is for
supported versions. I wouldn't mind seeing a small warning when
viewing an unsupported version page, though. It's a little too easy to
follow a search result or other link and land on a 7.x doc page
without noticing.

I'm not sure about the thornier problem of Google continuing to rank
the ancient pages higher than the new ones in many cases. I've seen a
few cases in our own wiki of links going to specific (old) versions of
doc pages, when they really should have pointed to /current, which is
easy enough to fix.

Here's a crazy idea: what about marking links to the unsupported
versions with rel="nofollow"?

Josh



Re: Search points to ancient manuals

From
Dave Page
Date:
On Thu, Nov 1, 2012 at 11:13 AM, Craig Ringer <ringerc@ringerc.id.au> wrote:
> On 11/01/2012 06:00 PM, Magnus Hagander wrote:
>
> We already do this for Google, so it shouldn't be rocket science.
>
> I still routinely see 8.1 / 8.2 / 8.3 results in searches. For example, my
> first hit for a Google search for EXPLAIN ANALYZE is

FYI, the Google Analytics traffic flow report shows this nicely.
docs/8.1/static is one of the highest ranked landing pages (depending
on the month - sometimes it's at the top, other times second in the
samples I looked at), and shows large numbers of users bouncing
straight across to the 9.1 docs (25% in October) or the 9.2 docs (20%
in October).

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: Search points to ancient manuals

From
Magnus Hagander
Date:
On Wed, Nov 7, 2012 at 5:38 PM, Dave Page <dpage@pgadmin.org> wrote:
> On Thu, Nov 1, 2012 at 11:13 AM, Craig Ringer <ringerc@ringerc.id.au> wrote:
>> On 11/01/2012 06:00 PM, Magnus Hagander wrote:
>>
>> We already do this for Google, so it shouldn't be rocket science.
>>
>> I still routinely see 8.1 / 8.2 / 8.3 results in searches. For example, my
>> first hit for a Google search for EXPLAIN ANALYZE is
>
> FYI, the Google Analytics traffic flow report shows this nicely.
> docs/8.1/static is one of the highest ranked landing pages (depending
> on the month - sometimes it's at the top, other times second in the
> samples I looked at), and shows large numbers of users bouncing
> straight across to the 9.1 docs (25% in October) or the 9.2 docs (20%
> in October).

Wouldn't it be nice if there was a feedback loop from analytics to
search for that... But AFAIK there is no way to make them do that even
if we ask for it...


--Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/



Re: Search points to ancient manuals

From
Magnus Hagander
Date:
On Wed, Nov 7, 2012 at 7:28 AM, Josh Kupershmidt <schmiddy@gmail.com> wrote:
> On Fri, Nov 2, 2012 at 7:36 AM, Magnus Hagander <magnus@hagander.net> wrote:
>> On Fri, Nov 2, 2012 at 1:43 PM, Craig Ringer <ringerc@ringerc.id.au> wrote:
>>> Maybe if the user visits old docs there just needs to be a banner with a
>>> prominent "This documentation is specific to version xx of PostgreSQL;
>>> if this is not the version you are using please choose yours from the
>>> links above."
>
>> How intrusive do we want to make that? I think just having the links up
>> there has actually made it a lot more user friendly in that regard
>> already...
>
> IMO the "This page in other versions" links work pretty well as-is for
> supported versions. I wouldn't mind seeing a small warning when
> viewing an unsupported version page, though. It's a little too easy to
> follow a search result or other link and land on a 7.x doc page
> without noticing.

Not an unreasonable thing to do. Looking forward to a patch from
someone who can figure out how to make one that's just intrusive
enough without being too intrusive :)


> I'm not sure about the thornier problem of Google continuing to rank
> the ancient pages higher than the new ones in many cases. I've seen a
> few cases in our own wiki of links going to specific (old) versions of
> doc pages, when they really should have pointed to /current, which is
> easy enough to fix.
>
> Here's a crazy idea: what about marking links to the unsupported
> versions with rel="nofollow"?

I don't think that's really that crazy :) And it's also easy to do, so
I've done it. It'll take a while before it actually hits the site
though, as it takes a while to work it's way through all the different
cached docs pages.

--Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/