Thread: Bad order of Postgres links in Google search results and how to fix it
Bad order of Postgres links in Google search results and how to fix it
From
Nikolay Samokhvalov
Date:
Hi,
I propose:
It is very annoying that it most cases, Google search engine result page (SERP) orders links to Postgres documentation so that older versions go first and sometimes the most up-to-date version is not even listed on the 1st page.
For instance, if I try to google "postgresql string functions" (in Chrome's incognito mode to have less bias; try it yourself but keep in mind that results may vary depending on your region, settings and search history) it currently shows me links in the following order:
etc, and current (Postgres 10) documentation doesn't even present on the 1st page of SERP. (Well, it does, but only PostgresPro's Russian version https://postgrespro.ru/docs/postgrespro/10/functions-string.html – and it's not because I speak Russian – I by purpose didn't give Google any signs to suspect that; I suspect it's because PostgresPro's page has better SEO weight than corresponding official English page).
This is the very confusing situation – I hate clicking "latest" each time I use Google to get some Postgres info, and, what is much, much worse, less Postgres-experienced people even don't suspect that they work with very outdated data (fresh example: https://twitter.com/will_in_wi/status/1009813068346519552 – notice Postgres version IN URL, it's 8.2, from 2007).
Here is a quick look at SEPRs of the query "XXX string functions" for other open source database systems:
In all 3 cases, we observe that the 1st position is being held by a version-free link, which is still relevant and shows information for the current, most recent production-ready version (e.g., for MySQL it's 8.0 and the links for previous versions – 5.6 and 5.5 – go, with version hard-coded to the URL).
This is very convenient and the most user-friendly situation and I think all Postgres user would benefit from it and the daily amount of confusion in this world would be much less if we somehow influence on Google SEPR to make it place results for the most recent and production-ready version to the very top.
But how?
Postgres documentation has hardcoded links and the special link with "current" word in URL (current main documentation page: https://www.postgresql.org/docs/current/static/index.html). But why it isn't being used by Google and as the main one? The reason is that PostgreSQL website doesn't consider it as the main one. Any time I go to https://postgresql.org/docs/ or https://www.postgresql.org/docs/ and try to achieve any documentation page I don't get "current" word in URL – I need to choose version myself, so I definitely get version in the URL.
This leads to the situation when people refer to documentation pages with hardcoded version in URL, when they write articles, tweet or discuss anything in social networks (what is very imporatnt for Google and ranking of pages nowadays) or chats or forums.
What I propose:
===
1) On the main page (www.postgresql.org) refer to the current documentation (URL = https://www.postgresql.org/docs/current/static/index.html) to allow me to get there with just 1 click. This step will give great weight for this page and spread good weight among deeper pages with "static" keyword in URL. It will also – as well as the next steps in my proposal – encourage people to use "current" URLs in their discussions/articles/posts everywhere.
2) Anywhere where the list of documentation versions is given, to the "CURRENT" version to the top, emphasizing that this is the "main" one at the moment, using the entry URL with "current" keyword and using exact version only for displaying purposes, not in any URLs. Example: https://www.postgresql.org/docs/manuals/ – the URL leading to the version 10 documentation should have "current", not "10"
3) On the page https://www.postgresql.org/docs/ allow users to get to the "current" documentation as quickly as possible, without additional thinking and clicking. Now I see the word "current", click it expecting to see the documentation already but see the list of versions again because it leads me to the mage https://www.postgresql.org/docs/manuals/. I would rephrase it so the word "current" would have URL https://www.postgresql.org/docs/current/static/index.html). In general, I would redesign this page to make it more UX-friendly.
The current version of its main, central part:
This section contains current and archived manuals for PostgreSQL users. You can read the release notes, and view a listing of books written about PostgreSQL.
For each major PostgreSQL version, a separate version of manual exists:
- most recent production-ready version (10),
- the list of currently supported versions (9.3 – 11 beta),
- archived manuals (6.3 – 9.2).
Also, you can:
- read the release notes,
- view a listing of books written about PostgreSQL,
- work with PostgreSQL wiki.
4) Inside manuals, I would reverse the order of versions mentioned and would make "current" upper case. Now it is:
I propose:
===
Once these steps are done, one cannot expect immediate results from Google. Some time (months or, for some cases, even years) needs to be passed before SEPRs for most popular queries will be properly reorganized.
I can try and make the corresponding patches myself once the consensus is achieved here.
Nikolay
Re: Bad order of Postgres links in Google search results and how tofix it
From
Nikolay Samokhvalov
Date:
Any feedback on this?
The problem persists, a few minutes ago I explained to the new user who cited some part of the doc and sent me a link to the manual, that 9.1 is very, very old version – obviously, again, Google was used to find that link.
On Fri, Jun 22, 2018 at 2:09 PM Nikolay Samokhvalov <samokhvalov@gmail.com> wrote:
Hi,It is very annoying that it most cases, Google search engine result page (SERP) orders links to Postgres documentation so that older versions go first and sometimes the most up-to-date version is not even listed on the 1st page.For instance, if I try to google "postgresql string functions" (in Chrome's incognito mode to have less bias; try it yourself but keep in mind that results may vary depending on your region, settings and search history) it currently shows me links in the following order:etc, and current (Postgres 10) documentation doesn't even present on the 1st page of SERP. (Well, it does, but only PostgresPro's Russian version https://postgrespro.ru/docs/postgrespro/10/functions-string.html – and it's not because I speak Russian – I by purpose didn't give Google any signs to suspect that; I suspect it's because PostgresPro's page has better SEO weight than corresponding official English page).This is the very confusing situation – I hate clicking "latest" each time I use Google to get some Postgres info, and, what is much, much worse, less Postgres-experienced people even don't suspect that they work with very outdated data (fresh example: https://twitter.com/will_in_wi/status/1009813068346519552 – notice Postgres version IN URL, it's 8.2, from 2007).Here is a quick look at SEPRs of the query "XXX string functions" for other open source database systems:In all 3 cases, we observe that the 1st position is being held by a version-free link, which is still relevant and shows information for the current, most recent production-ready version (e.g., for MySQL it's 8.0 and the links for previous versions – 5.6 and 5.5 – go, with version hard-coded to the URL).This is very convenient and the most user-friendly situation and I think all Postgres user would benefit from it and the daily amount of confusion in this world would be much less if we somehow influence on Google SEPR to make it place results for the most recent and production-ready version to the very top.But how?Postgres documentation has hardcoded links and the special link with "current" word in URL (current main documentation page: https://www.postgresql.org/docs/current/static/index.html). But why it isn't being used by Google and as the main one? The reason is that PostgreSQL website doesn't consider it as the main one. Any time I go to https://postgresql.org/docs/ or https://www.postgresql.org/docs/ and try to achieve any documentation page I don't get "current" word in URL – I need to choose version myself, so I definitely get version in the URL.This leads to the situation when people refer to documentation pages with hardcoded version in URL, when they write articles, tweet or discuss anything in social networks (what is very imporatnt for Google and ranking of pages nowadays) or chats or forums.What I propose:===1) On the main page (www.postgresql.org) refer to the current documentation (URL = https://www.postgresql.org/docs/current/static/index.html) to allow me to get there with just 1 click. This step will give great weight for this page and spread good weight among deeper pages with "static" keyword in URL. It will also – as well as the next steps in my proposal – encourage people to use "current" URLs in their discussions/articles/posts everywhere.2) Anywhere where the list of documentation versions is given, to the "CURRENT" version to the top, emphasizing that this is the "main" one at the moment, using the entry URL with "current" keyword and using exact version only for displaying purposes, not in any URLs. Example: https://www.postgresql.org/docs/manuals/ – the URL leading to the version 10 documentation should have "current", not "10"3) On the page https://www.postgresql.org/docs/ allow users to get to the "current" documentation as quickly as possible, without additional thinking and clicking. Now I see the word "current", click it expecting to see the documentation already but see the list of versions again because it leads me to the mage https://www.postgresql.org/docs/manuals/. I would rephrase it so the word "current" would have URL https://www.postgresql.org/docs/current/static/index.html). In general, I would redesign this page to make it more UX-friendly.The current version of its main, central part:This section contains current and archived manuals for PostgreSQL users. You can read the release notes, and view a listing of books written about PostgreSQL.I propose:For each major PostgreSQL version, a separate version of manual exists:
- most recent production-ready version (10),
- the list of currently supported versions (9.3 – 11 beta),
- archived manuals (6.3 – 9.2).
Also, you can:
- read the release notes,
- view a listing of books written about PostgreSQL,
- work with PostgreSQL wiki.
4) Inside manuals, I would reverse the order of versions mentioned and would make "current" upper case. Now it is:I propose:===Once these steps are done, one cannot expect immediate results from Google. Some time (months or, for some cases, even years) needs to be passed before SEPRs for most popular queries will be properly reorganized.I can try and make the corresponding patches myself once the consensus is achieved here.Nikolay
Greeting Nikolay, * Nikolay Samokhvalov (samokhvalov@gmail.com) wrote: > Any feedback on this? At least a couple of the primary -www folks are currently more-or-less offline, so I doubt we'll hear much on this front in the next couple weeks. That said, thanks for bumping it, as I definitely think this is something we'd like to fix. I'd suggest bumping it again, if you haven't heard anything, once we get close to September. Thanks! Stephen
Attachment
Re: Bad order of Postgres links in Google search results and how tofix it
From
Daniel Gustafsson
Date:
> On 30 Jul 2018, at 01:44, Stephen Frost <sfrost@snowman.net> wrote: > * Nikolay Samokhvalov (samokhvalov@gmail.com) wrote: >> Any feedback on this? > > At least a couple of the primary -www folks are currently more-or-less > offline, so I doubt we'll hear much on this front in the next couple > weeks. That said, thanks for bumping it, as I definitely think this is > something we'd like to fix. I'd suggest bumping it again, if you > haven't heard anything, once we get close to September. +1 It should be noted that other search engines like DuckDuckGo exhibit similar behavior so this is not just a Google thing. cheers ./daniel
On Mon, Jul 30, 2018 at 1:32 AM, Nikolay Samokhvalov <samokhvalov@gmail.com> wrote: > Any feedback on this? > > The problem persists, a few minutes ago I explained to the new user who > cited some part of the doc and sent me a link to the manual, that 9.1 is > very, very old version – obviously, again, Google was used to find that > link. > > On Fri, Jun 22, 2018 at 2:09 PM Nikolay Samokhvalov <samokhvalov@gmail.com> > wrote: >> >> Hi, >> >> It is very annoying that it most cases, Google search engine result page >> (SERP) orders links to Postgres documentation so that older versions go >> first and sometimes the most up-to-date version is not even listed on the >> 1st page. >> >> For instance, if I try to google "postgresql string functions" (in >> Chrome's incognito mode to have less bias; try it yourself but keep in mind >> that results may vary depending on your region, settings and search history) >> it currently shows me links in the following order: >> >> 1. https://www.postgresql.org/docs/9.1/static/functions-string.html >> 2. https://www.postgresql.org/docs/9.6/static/functions-string.html >> 3. https://www.postgresql.org/docs/9.5/static/functions-string.html >> 4. https://www.postgresql.org/docs/8.1/static/functions-string.html >> 5. http://www.postgresqltutorial.com/postgresql-string-functions/ >> >> >> etc, and current (Postgres 10) documentation doesn't even present on the >> 1st page of SERP. (Well, it does, but only PostgresPro's Russian version >> https://postgrespro.ru/docs/postgrespro/10/functions-string.html – and it's >> not because I speak Russian – I by purpose didn't give Google any signs to >> suspect that; I suspect it's because PostgresPro's page has better SEO >> weight than corresponding official English page). >> >> This is the very confusing situation – I hate clicking "latest" each time >> I use Google to get some Postgres info, and, what is much, much worse, less >> Postgres-experienced people even don't suspect that they work with very >> outdated data (fresh example: >> https://twitter.com/will_in_wi/status/1009813068346519552 – notice Postgres >> version IN URL, it's 8.2, from 2007). >> >> Here is a quick look at SEPRs of the query "XXX string functions" for >> other open source database systems: >> >> A. "mysql string functions" >> >> 1. https://dev.mysql.com/doc/en/string-functions.html >> 2. https://dev.mysql.com/doc/refman/5.6/en/string-functions.html >> 3. https://dev.mysql.com/doc/refman/5.5/en/string-functions.html >> 4. https://www.w3schools.com/sql/sql_ref_mysql.asp >> 5. https://www.w3schools.com/sql/func_mysql_substring.asp >> >> >> B. "mariadb string functions" >> >> 1. https://mariadb.com/kb/en/library/string-functions/ >> 2. https://mariadb.com/kb/en/library/built-in-functions/ >> 3. https://mariadb.com/kb/en/library/locate/ >> 4. https://mariadb.com/kb/en/library/like/ >> 5. https://www.techonthenet.com/mariadb/functions/position.php >> >> >> C. "mongodb string functions" >> >> 1. https://docs.mongodb.com/manual/reference/method/db.collection.find/ >> 2. https://docs.mongodb.com/manual/text-search/ >> 3. https://docs.mongodb.com/manual/reference/operator/query/ >> 4. https://docs.mongodb.com/manual/tutorial/query-documents/ >> 5. >> https://code.tutsplus.com/tutorials/full-text-search-in-mongodb--cms-24835 >> >> >> In all 3 cases, we observe that the 1st position is being held by a >> version-free link, which is still relevant and shows information for the >> current, most recent production-ready version (e.g., for MySQL it's 8.0 and >> the links for previous versions – 5.6 and 5.5 – go, with version hard-coded >> to the URL). >> >> This is very convenient and the most user-friendly situation and I think >> all Postgres user would benefit from it and the daily amount of confusion in >> this world would be much less if we somehow influence on Google SEPR to make >> it place results for the most recent and production-ready version to the >> very top. >> >> But how? >> >> Postgres documentation has hardcoded links and the special link with >> "current" word in URL (current main documentation page: >> https://www.postgresql.org/docs/current/static/index.html). But why it isn't >> being used by Google and as the main one? The reason is that PostgreSQL >> website doesn't consider it as the main one. Any time I go to >> https://postgresql.org/docs/ or https://www.postgresql.org/docs/ and try to >> achieve any documentation page I don't get "current" word in URL – I need to >> choose version myself, so I definitely get version in the URL. >> >> This leads to the situation when people refer to documentation pages with >> hardcoded version in URL, when they write articles, tweet or discuss >> anything in social networks (what is very imporatnt for Google and ranking >> of pages nowadays) or chats or forums. >> >> What I propose: >> >> === >> 1) On the main page (www.postgresql.org) refer to the current >> documentation (URL = >> https://www.postgresql.org/docs/current/static/index.html) to allow me to >> get there with just 1 click. This step will give great weight for this page >> and spread good weight among deeper pages with "static" keyword in URL. It >> will also – as well as the next steps in my proposal – encourage people to >> use "current" URLs in their discussions/articles/posts everywhere. >> >> 2) Anywhere where the list of documentation versions is given, to the >> "CURRENT" version to the top, emphasizing that this is the "main" one at the >> moment, using the entry URL with "current" keyword and using exact version >> only for displaying purposes, not in any URLs. Example: >> https://www.postgresql.org/docs/manuals/ – the URL leading to the version 10 >> documentation should have "current", not "10" >> >> 3) On the page https://www.postgresql.org/docs/ allow users to get to the >> "current" documentation as quickly as possible, without additional thinking >> and clicking. Now I see the word "current", click it expecting to see the >> documentation already but see the list of versions again because it leads me >> to the mage https://www.postgresql.org/docs/manuals/. I would rephrase it so >> the word "current" would have URL >> https://www.postgresql.org/docs/current/static/index.html). In general, I >> would redesign this page to make it more UX-friendly. >> >> The current version of its main, central part: >> >> This section contains current and archived manuals for PostgreSQL users. >> You can read the release notes, and view a listing of books written about >> PostgreSQL. >> >> >> I propose: >> >> For each major PostgreSQL version, a separate version of manual exists: >> >> most recent production-ready version (10), >> the list of currently supported versions (9.3 – 11 beta), >> archived manuals (6.3 – 9.2). >> >> Also, you can: >> >> read the release notes, >> view a listing of books written about PostgreSQL, >> work with PostgreSQL wiki. >> >> >> 4) Inside manuals, I would reverse the order of versions mentioned and >> would make "current" upper case. Now it is: >> >> This page in other versions: 9.3 / 9.4 / 9.5 / 9.6 / current (10) >> >> >> I propose: >> >> This page in other versions: CURRENT (10) / 9.6 / 9.5 / 9.4 / 9.3 >> >> === >> >> Once these steps are done, one cannot expect immediate results from >> Google. Some time (months or, for some cases, even years) needs to be passed >> before SEPRs for most popular queries will be properly reorganized. >> >> I can try and make the corresponding patches myself once the consensus is >> achieved here. >> >> Nikolay >> >> >> >> >> >> > The problem is well known for a long time, for example, this thread from 2009 https://www.postgresql.org/message-id/flat/1252074856.13736.17.camel%40fsopti579.F-Secure.com It might helps if we generate sitemap.xml (https://www.postgresql.org/sitemap.xml) with links to the pages in "right" order. Oleg -- Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
Re: Bad order of Postgres links in Google search results and how tofix it
From
Nikolay Samokhvalov
Date:
On Mon, Jul 30, 2018 at 6:35 AM Oleg Bartunov <obartunov@postgrespro.ru> wrote:
The problem is well known for a long time, for example, this thread from 2009
https://www.postgresql.org/message-id/flat/1252074856.13736.17.camel%40fsopti579.F-Secure.com
It might helps if we generate sitemap.xml
(https://www.postgresql.org/sitemap.xml) with links
to the pages in "right" order.
Sitemaps won't be really useful here since their primary goal is to deliver up-to-date lists of pages to search engines, with lastmod timestamps.
To influence on the search engines, it is very important to change the real structure of website, to rework existing navigation for humans. This is the key to solve this problem. The problem is not huge – I've shown "good" examples in my original email and explained why they are "good".
We need to fix real users' behaviour – the most imporant thing is what people use when they share documentation URLs in social networks, blogs and articles.
Re: Bad order of Postgres links in Google search results and how tofix it
From
Damien Clochard
Date:
Le 31.07.2018 10:57, Nikolay Samokhvalov a écrit : > On Mon, Jul 30, 2018 at 6:35 AM Oleg Bartunov > <obartunov@postgrespro.ru> wrote: > >> The problem is well known for a long time, for example, this thread >> from 2009 >> > https://www.postgresql.org/message-id/flat/1252074856.13736.17.camel%40fsopti579.F-Secure.com >> >> It might helps if we generate sitemap.xml >> (https://www.postgresql.org/sitemap.xml) with links >> to the pages in "right" order. > > Sitemaps won't be really useful here since their primary goal is to > deliver up-to-date lists of pages to search engines, with lastmod > timestamps. > > To influence on the search engines, it is very important to change the > real structure of website, to rework existing navigation for humans. > This is the key to solve this problem. The problem is not huge – > I've shown "good" examples in my original email and explained why they > are "good". > > We need to fix real users' behaviour – the most imporant thing is > what people use when they share documentation URLs in social networks, > blogs and articles. +1 For the record, Google provides some tools (links below) to understand how a website is perceived and gives hindsight on how to improve search results : https://www.google.com/webmasters/tools/ https://search.google.com/search-console -- Damien Clochard
Re: Bad order of Postgres links in Google search results and how tofix it
From
Nikolay Samokhvalov
Date:
On Sun, Jul 29, 2018 at 4:44 PM Stephen Frost <sfrost@snowman.net> wrote:
Greeting Nikolay,
* Nikolay Samokhvalov (samokhvalov@gmail.com) wrote:
> Any feedback on this?
At least a couple of the primary -www folks are currently more-or-less
offline, so I doubt we'll hear much on this front in the next couple
weeks. That said, thanks for bumping it, as I definitely think this is
something we'd like to fix. I'd suggest bumping it again, if you
haven't heard anything, once we get close to September.
– I've got a link to Postgres 8.1 documentation.
Fwiw I used to work at Google and raised this internally. It is definitely something the search quality team was aware of and it affects other projects as well. Try searching for docs on Java APIs for example.
They seemed really interested in improving it and there have been some improvements but it's obviously still not there. It's a harder problem than you might expect because there are lots of links to old versions and they have lots of data that users are happy with the results from 9.6 or 10 (and indeed even after 11 is released most users are probably still looking for 9.6 or 10 docs...)
A few things we could do is add a prominent coloured banner across the top of every page saying "you're looking at docs for an old version, click here for the docs for the current version".
We could actually make the current version "canonical" but then it would be actually awkward if anyone wanted to search for older docs. And it's not true in general that every page is present in every version. We sometimes add or remove pages from the docs... That still might be the only really useful option today.
Greg Stark <stark@mit.edu> writes:
> We could actually make the current version "canonical" but then it would be
> actually awkward if anyone wanted to search for older docs. And it's not
> true in general that every page is present in every version. We sometimes
> add or remove pages from the docs... That still might be the only really
> useful option today.
I'm not quite following why that's such a bad option?
People would get sent to the current version, sure, but there's a header
right there with a link to the prior version they actually want.
Pages being deleted might be a problem, but how would a nonexistent
"canonical" marking on a page that doesn't exist stop indexing of older
pages? Maybe we could even notice "page XYZ doesn't exist after 9.3,
so mark 9.3's version as canonical".
regards, tom lane