On Thu, Nov 1, 2012 at 12:35 PM, Marti Raudsepp <marti@juffo.org> wrote:
On Thu, Nov 1, 2012 at 1:13 PM, Craig Ringer <ringerc@ringerc.id.au> wrote: > I've often wished that the docs would redirect to the /current/interactive version when the referer is google. Not enough to write the code yet, though.
That solution is backwards. You click on the Google link that says "Documentation: 8.1: EXPLAIN", but when you click on it, you magically end up in PostgreSQL 9.2 docs.
Yeah, I htink that's a really bad idea.
Instead we should encourage or force Googlebot (and other search engines) to index/prefer the current version of docs. I hear the sitemaps file already gives higher priority to current, but clearly that's not having the effect it should. I suspect most of our referrers link to versioned docs, which boosts their rank, and the search engine has no way of telling that the individual versions are mostly equivalent.
Yeah, that's my impression as well. Once we put the sitemap into place properly, after some time (a month or so I think) the *obsolete* documentation links started to disappear from the google search hits (they're not in the sitemap at all). But it seems to pay no attention whatsoever to the priorities given in the sitemap.
We might detect search engines by their User-Agent header and always serve them redirects to /current/ (if the page exists in current) or the latest version. But that would have the effect of de-indexing old versions of pages that have been revised -- which isn't an issue most of the time, but is not ideal either.
If we want to do that, we can remove the old versions from the sitemap, which is a much easier way of doing it. However, that would remove any way of searching for the old documentation there - and there are certainly many usecases where doing that makes sense.
Also it would require us to serve docs pages with "Vary: User-Agent", which would make caching less efficient (but maybe there are workarounds?)
Not really, we could fairly easily work around that specifically for the docs. No need to use the big gun of setting vary:user-agent on every page (even every docs page).
However, I still think it's a non-starter since it would make it impossible to search for old stuff.
