Thread: Rough estimate of number of downloads per year/month/day?

Rough estimate of number of downloads per year/month/day?

From
Gabriele Bartolini
Date:
I know this is the 1 million dollar question and we miss most of the data.

But I'd love to have some numbers about the downloads from the community
website. Something like: number of downloads in 2012 so far, or a daily
average in 2012 or something like that. If you have a series of data per
day, that'd be cool.

The reason is that today an open source conference is takin place in
Italy and it was reported that MySQL has 70000 downloads per day. I have
asked more information about the source and the specific product, but
I'd love to have a number about PostgreSQL (even if partial).

Thanks.

Cheers,
Gabriele


Re: Rough estimate of number of downloads per year/month/day?

From
Josh Berkus
Date:
> The reason is that today an open source conference is takin place in
> Italy and it was reported that MySQL has 70000 downloads per day. I have
> asked more information about the source and the specific product, but
> I'd love to have a number about PostgreSQL (even if partial).

I'm reasonably sure that the data we have doesn't cover the majority of
download sources for PostgreSQL (e.g. package repositories).

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com


Re: Rough estimate of number of downloads per year/month/day?

From
"Jonathan S. Katz"
Date:
On Oct 25, 2012, at 2:07 PM, Josh Berkus wrote:

>> The reason is that today an open source conference is takin place in
>> Italy and it was reported that MySQL has 70000 downloads per day. I have
>> asked more information about the source and the specific product, but
>> I'd love to have a number about PostgreSQL (even if partial).
>
> I'm reasonably sure that the data we have doesn't cover the majority of
> download sources for PostgreSQL (e.g. package repositories).

But presumably, the numbers that MySQL etc. would be producing would be similar, since they would not be able to
accuratelyobtain numbers from package repos, etc. 

Jonathan



Re: Rough estimate of number of downloads per year/month/day?

From
Devrim GÜNDÜZ
Date:
Hi,

On Thu, 2012-10-25 at 14:12 +0200, Jonathan S. Katz wrote:
> But presumably, the numbers that MySQL etc. would be producing would
> be similar, since they would not be able to accurately obtain numbers
> from package repos, etc.

I think what Josh refers is our official package repositories, like RPM
and the upcoming DEB packages, plus 1-click installers from
EnterpriseDB downloads. Not sure that MySQL has such repos.

OTOH, PostgreSQL has also a wide usage from the distro repositories for
sure, like MySQL.

Regards,
--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz

Attachment

Re: Rough estimate of number of downloads per year/month/day?

From
Gabriele Bartolini
Date:
Il 25/10/12 14:07, Josh Berkus ha scritto:
> I'm reasonably sure that the data we have doesn't cover the majority of
> download sources for PostgreSQL (e.g. package repositories).
I perfectly know that it is partial. But still a partial (but reliable)
source, is much better than no information.

I plan to reply something like: "we cannot count Windows installer,
other binaries, RPM packages, Debian packages, etc. but regular
downloads from the Community site are XX".

I think this would be great.

Ciao,
Gabriele


--
  Gabriele Bartolini - 2ndQuadrant Italia
  PostgreSQL Training, Services and Support
  gabriele.bartolini@2ndQuadrant.it | www.2ndQuadrant.it



Re: Rough estimate of number of downloads per year/month/day?

From
Josh Berkus
Date:
> I plan to reply something like: "we cannot count Windows installer,
> other binaries, RPM packages, Debian packages, etc. but regular
> downloads from the Community site are XX".

It's not helpful if the exceptions make our numbers look smaller than
MySQL's ...


--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com


Re: Rough estimate of number of downloads per year/month/day?

From
Jean-Paul Argudo
Date:
Hi there,

Le 25/10/2012 14:31, Josh Berkus a écrit :
>
>> I plan to reply something like: "we cannot count Windows installer,
>> other binaries, RPM packages, Debian packages, etc. but regular
>> downloads from the Community site are XX".
>
> It's not helpful if the exceptions make our numbers look smaller than
> MySQL's ...

This idea is not new but.. Could we imagine some kind of automatic
survey on install?

Depending packaging, OSes and such, this would be implemented
differently, for sure, but the basic would be that we ask the one
installing if (s)he wants to participate in a survey of few questions or
send some anonymous infos to the project (like hardware specs, OS specs
for example).

This is done in various other projects.

Just wondering what you think of this kind of thing...


Cheers,


--
Jean-Paul Argudo
www.PostgreSQL.(fr|eu)
www.Dalibo.com


Re: Rough estimate of number of downloads per year/month/day?

From
Gabriele Bartolini
Date:
Hi Josh,

Il 25/10/12 14:31, Josh Berkus ha scritto:
> It's not helpful if the exceptions make our numbers look smaller than
> MySQL's ...
I have no idea about the numbers we already have, so I don't know if
that's below MySQL.

But I would simply take them for what they are: download hits from the
PostgreSQL community website. If we are afraid of the absolute numbers,
I think it is important also to monitor the trend.

I would not mind - if possible - an automated process that every day
aggregates this data based (just making it up) on UTC. Then publish this
data on the website somewhere.

I volunteer to help with this process, if it does not exist yet.

Cheers,
Gabriele

--
  Gabriele Bartolini - 2ndQuadrant Italia
  PostgreSQL Training, Services and Support
  gabriele.bartolini@2ndQuadrant.it | www.2ndQuadrant.it



Re: Rough estimate of number of downloads per year/month/day?

From
"Jonathan S. Katz"
Date:
On Oct 25, 2012, at 2:58 PM, Gabriele Bartolini <gabriele.bartolini@2ndQuadrant.it> wrote:

> Hi Josh,
>
> Il 25/10/12 14:31, Josh Berkus ha scritto:
>> It's not helpful if the exceptions make our numbers look smaller than MySQL's ...
> I have no idea about the numbers we already have, so I don't know if that's below MySQL.
>
> But I would simply take them for what they are: download hits from the PostgreSQL community website. If we are afraid
ofthe absolute numbers, I think it is important also to monitor the trend. 
>
> I would not mind - if possible - an automated process that every day aggregates this data based (just making it up)
onUTC. Then publish this data on the website somewhere. 
>
> I volunteer to help with this process, if it does not exist yet.
>
> Cheers,
> Gabriele

+1.  Particularly to look at the trends and see where we are increasing/decreasing on downloads.  We can also try to
identifyany correlation with other marketing efforts and see, for instance, if press coverage or other mentions are in
factdriving more downloads. 

Jonathan

Re: Rough estimate of number of downloads per year/month/day?

From
Josh Kupershmidt
Date:
On Thu, Oct 25, 2012 at 3:32 AM, Gabriele Bartolini
<gabriele.bartolini@2ndquadrant.it> wrote:
> I know this is the 1 million dollar question and we miss most of the data.
>
> But I'd love to have some numbers about the downloads from the community
> website. Something like: number of downloads in 2012 so far, or a daily
> average in 2012 or something like that. If you have a series of data per
> day, that'd be cool.
>
> The reason is that today an open source conference is takin place in Italy
> and it was reported that MySQL has 70000 downloads per day. I have asked
> more information about the source and the specific product, but I'd love to
> have a number about PostgreSQL (even if partial).

If you're looking for a rough (but still apples-to-apples) Postgres
vs. MySQL estimate, what about using Debian popcon? Looking at this
listing:
  http://popcon.debian.org/by_inst

For MySQL client:
$ cat by_inst.txt | grep -i mysql | grep -i client  | perl -pi -e
's/\d+\s+[\S]+\s+(\d+).+$/$1/' | awk '{s+=$1} END {print s}'
183654

For PostgreSQL client:
$ cat by_inst.txt | grep -i postgres | grep -i -E '(client)|(libpq)' |
perl -pi -e 's/\d+\s+[\S]+\s+(\d+).+$/$1/' | awk '{s+=$1} END {print
s}'
122026

For MySQL server:
$ cat by_inst.txt | grep -i mysql | grep -i server  | perl -pi -e
's/\d+\s+[\S]+\s+(\d+).+$/$1/' | awk '{s+=$1} END {print s}'
113256

For PostgreSQL server, tallying by hand*:
postgresql-common (19355) + postgresql (14895) + postgresql-8.4
(11058) + postgresql-9.1 (4315) + postgrsql-8.3 (3059) +
postgresql-9.0 (865) = 53547
* ignoring some uncommon variants less popular than all these

I'm not totally familiar with popcon and the Debian packaging
conventions, so apologies if I double-counted some packages or
otherwise mangled the counts.

Josh


Re: Rough estimate of number of downloads per year/month/day?

From
Shane Ambler
Date:
On 26/10/2012 01:41, Josh Kupershmidt wrote:

> If you're looking for a rough (but still apples-to-apples) Postgres
> vs. MySQL estimate, what about using Debian popcon? Looking at this
>
> For MySQL client:
> 183654
>
> For PostgreSQL client:
> 122026
>
> For MySQL server:
> 113256
>
> For PostgreSQL server, tallying by hand*:
> 53547

So MySQL has 183654 clients to 113256 servers - 1.62 clients to a server

Postgres has 122026 clients to 53547 servers - 2.28 clients to a server

Does that represent that postgres naturally handles more clients? or
just how many apps support postgres?

It will be hard to measure but I think you will find that mysql has a
similar effect to windows oem - some apps fail to provide an option to
switch off mysql support so you get forced to install mysql even when
you don't want to use it. This is the only reason that I have mysql
installed on my machine - programs won't build without it. But that
doesn't mean I have ever bothered to get mysql server running.
(Well not for about 6 years now)



Re: Rough estimate of number of downloads per year/month/day?

From
Selena Deckelmann
Date:
On Fri, Oct 26, 2012 at 1:18 AM, Shane Ambler <pgsql@sheeky.biz> wrote:

It will be hard to measure but I think you will find that mysql has a
similar effect to windows oem - some apps fail to provide an option to
switch off mysql support so you get forced to install mysql even when
you don't want to use it. This is the only reason that I have mysql
installed on my machine - programs won't build without it. But that
doesn't mean I have ever bothered to get mysql server running.
(Well not for about 6 years now)

Clearly, we need to get more applications to require postgres installs. ;)

Honestly, i think it just shows how useless the raw number is.

Far more interesting would be change over time. "The trend is XX% increase/decrease over X years."

So, a great project for someone: start collecting these stats in a way that can be reproduced by others and publish the data.

-selena

--
http://chesnok.com

Re: Rough estimate of number of downloads per year/month/day?

From
Josh Berkus
Date:
> Far more interesting would be change over time. "The trend is XX%
> increase/decrease over X years."
>
> So, a great project for someone: start collecting these stats in a way that
> can be reproduced by others and publish the data.

So, here's the reason I haven't been interested in pursuing these
numbers for several years.  I used to track them.

In May of 2005, we had a sudden 70% drop in Windows downloads.  WTH?

After some digging, it turned out that in April the downloads site
TuCows started offering PostgreSQL 8.0 for download, and by May this was
the top link for "postgres windows" on Google.  But if you just looked
at the numbers, it looked like Windows adoption had fallen off a cliff.

The stats we have are counts on download redirects from
www.postgresql.org, and EnterpriseDB has been happy to furnish counts
from the one-click downloads on request.  I don't know if we have stats
for yum.postgresql.org.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com


Re: Rough estimate of number of downloads per year/month/day?

From
"Greg Sabino Mullane"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160


> The reason is that today an open source conference is takin place in
> Italy and it was reported that MySQL has 70000 downloads per day. I have
> asked more information about the source and the specific product, but
> I'd love to have a number about PostgreSQL (even if partial).

Did you (or anyone else) ever get a response? It's easy to find this 70K
number on the web, but not so easy to find out more details. Frankly, it's
an unbelievable claim at face value, so I am assuming there are some
major caveats. It is certainly not a number I suspect we can even come close
to competing with, even were we to be able to gather that information (which
seems a Sisyphean given the many and shifting ways of installing Postgres).

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 201211042152
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAlCXKiwACgkQvJuQZxSWSsi6KwCeJmwEc7aDRIV5FkQt6cR3Wiul
n/MAmgItvwXRSmg/mZG3EUtvlzh5QBA/
=2JOd
-----END PGP SIGNATURE-----




Re: Rough estimate of number of downloads per year/month/day?

From
damien clochard
Date:

I agree with Josh : the number of downloads is not relevant for us. Just
like the "market share" estimations always fail to demonstrate
PostgreSQL presence in the industry.

However we need to provide numbers to represent the growth of our user
base. There's tons of blogs saying that PostgreSQL is wonderful but when
MySQL announces  70000 downloads/day or when Oracle says they own 50% of
the market, we are speechless because we don't have anything to oppose
to this.

I thinks we need to define new metrics to monitor the evolution of the
project in the industry. It's not easy but there must some way to mesure
that. For example, the job trends or ML traffic could be more
informative that the download numbers...

http://www.indeed.com/jobtrends?q=postgres%2C++oracle&l=&relative=1
http://markmail.blogspot.fr/2008/02/postgresql-more-traffic-than-mysql-and.html

Any other ideas ?


Re: Rough estimate of number of downloads per year/month/day?

From
Gabriele Bartolini
Date:
Il 05/11/12 03:54, Greg Sabino Mullane ha scritto:
> Did you (or anyone else) ever get a response? It's easy to find this 70K
> number on the web, but not so easy to find out more details.
As expected, I am still waiting for an answer. :)

Cheers,
Gabriele

--
  Gabriele Bartolini - 2ndQuadrant Italia
  PostgreSQL Training, Services and Support
  gabriele.bartolini@2ndQuadrant.it | www.2ndQuadrant.it



Re: Rough estimate of number of downloads per year/month/day?

From
"Jonathan S. Katz"
Date:
On Nov 6, 2012, at 6:26 AM, damien clochard wrote:

> http://www.indeed.com/jobtrends?q=postgres%2C++oracle&l=&relative=1
> http://markmail.blogspot.fr/2008/02/postgresql-more-traffic-than-mysql-and.html
>
> Any other ideas ?

It would be good to see the trend of website views.  Regardless of how people download Postgres, at some point they do
needto come to the site, e.g. for documentation or community support, etc.  It would be good to see the trendline for
websitegrowth, not only as a metric of more Postgres "awareness" but we can also break down the results further and
analyzewhat people are actually looking at on the website. 

Jonathan

Re: Rough estimate of number of downloads per year/month/day?

From
Gabriele Bartolini
Date:
Hi Damien,

Il 06/11/12 12:26, damien clochard ha scritto:
> I thinks we need to define new metrics to monitor the evolution of the
> project in the industry. It's not easy but there must some way to mesure
> that. For example, the job trends or ML traffic could be more
> informative that the download numbers...
An objection I got on this is that most of their supported users use
corporate ticketing systems and not public mailing lists, which
invalidates the comparison.

It is however a valid measure of the growth trend we see in PostgreSQL.

Cheers,
Gabriele

--
  Gabriele Bartolini - 2ndQuadrant Italia
  PostgreSQL Training, Services and Support
  gabriele.bartolini@2ndQuadrant.it | www.2ndQuadrant.it



Re: Rough estimate of number of downloads per year/month/day?

From
Magnus Hagander
Date:
On Tue, Nov 6, 2012 at 3:55 PM, Jonathan S. Katz <jonathan.katz@excoventures.com> wrote:
It would be good to see the trend of website views.  Regardless of how people download Postgres, at some point they do need to come to the site, e.g. for documentation or community support, etc.  It would be good to see the trendline for website growth, not only as a metric of more Postgres "awareness" but we can also break down the results further and analyze what people are actually looking at on the website.

Probably a fair amount of people actually never go even to the website.

We do have google analytics data for quite a few years around somewhere. It will only track those people who don't block google analytics of course, which we're probably quite overrepresented when it comes to, but it's something.


--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: Rough estimate of number of downloads per year/month/day?

From
Kris Pennella
Date:
Magnus -
If you'd be willing to get me access to the google analytics, I'd be willing to cull through them and do some analysis. I think any of the data found would be useful to the group at large (thoughts of what everyone would like to see, what's important would be helpful/encouraged). 
I've noticed in doing the analytics analysis for 2Q that many pages under postgres.org/  drive a decent percentage traffic to our site and it varies slightly from month to month which pages are most popular. 


---------------------------------------
Kris Pennella     2ndQuadrant US    
Operations/Client Services  kris@2ndQuadrant.com   
 503.236.8196
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com



On Nov 6, 2012, at 7:25 AM, Magnus Hagander wrote:

On Tue, Nov 6, 2012 at 3:55 PM, Jonathan S. Katz <jonathan.katz@excoventures.com> wrote:
It would be good to see the trend of website views.  Regardless of how people download Postgres, at some point they do need to come to the site, e.g. for documentation or community support, etc.  It would be good to see the trendline for website growth, not only as a metric of more Postgres "awareness" but we can also break down the results further and analyze what people are actually looking at on the website.

Probably a fair amount of people actually never go even to the website.

We do have google analytics data for quite a few years around somewhere. It will only track those people who don't block google analytics of course, which we're probably quite overrepresented when it comes to, but it's something.


--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: Rough estimate of number of downloads per year/month/day?

From
Magnus Hagander
Date:
We are fairly restrictive about giving access to that data. You'd have
to check that with -core.

Through a quick discussion with Jonathan yesterday it seems the data
in there isn't actually that correct anyway - it looks like we're
hitting the limit of what the free tier of google analytics will give
us, and have been hitting that for quite some time, so we don't really
know what the numbers are...

But it might be a good idea to have somebody who actually knows how GA
works very well take a look at things and see if we can work something
out from it.

//Magnus


On Tue, Nov 6, 2012 at 4:55 PM, Kris Pennella <kris@2ndquadrant.com> wrote:
> Magnus -
> If you'd be willing to get me access to the google analytics, I'd be willing
> to cull through them and do some analysis. I think any of the data found
> would be useful to the group at large (thoughts of what everyone would like
> to see, what's important would be helpful/encouraged).
> I've noticed in doing the analytics analysis for 2Q that many pages under
> postgres.org/  drive a decent percentage traffic to our site and it varies
> slightly from month to month which pages are most popular.
>
>
> ---------------------------------------
> Kris Pennella     2ndQuadrant US
> Operations/Client Services  kris@2ndQuadrant.com
>  503.236.8196
> PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com
>
>
>
> On Nov 6, 2012, at 7:25 AM, Magnus Hagander wrote:
>
> On Tue, Nov 6, 2012 at 3:55 PM, Jonathan S. Katz
> <jonathan.katz@excoventures.com> wrote:
>>
>> On Nov 6, 2012, at 6:26 AM, damien clochard wrote:
>>
>> > http://www.indeed.com/jobtrends?q=postgres%2C++oracle&l=&relative=1
>> >
>> > http://markmail.blogspot.fr/2008/02/postgresql-more-traffic-than-mysql-and.html
>> >
>> > Any other ideas ?
>>
>> It would be good to see the trend of website views.  Regardless of how
>> people download Postgres, at some point they do need to come to the site,
>> e.g. for documentation or community support, etc.  It would be good to see
>> the trendline for website growth, not only as a metric of more Postgres
>> "awareness" but we can also break down the results further and analyze what
>> people are actually looking at on the website.
>
>
> Probably a fair amount of people actually never go even to the website.
>
> We do have google analytics data for quite a few years around somewhere. It
> will only track those people who don't block google analytics of course,
> which we're probably quite overrepresented when it comes to, but it's
> something.
>
>
> --
>  Magnus Hagander
>  Me: http://www.hagander.net/
>  Work: http://www.redpill-linpro.com/
>
>



--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: Rough estimate of number of downloads per year/month/day?

From
Dave Page
Date:
On Wed, Nov 7, 2012 at 3:26 PM, Magnus Hagander <magnus@hagander.net> wrote:
> We are fairly restrictive about giving access to that data. You'd have
> to check that with -core.

We've always said no in the past, though we have shared specific
pre-generated reports from time to time.

> Through a quick discussion with Jonathan yesterday it seems the data
> in there isn't actually that correct anyway - it looks like we're
> hitting the limit of what the free tier of google analytics will give
> us, and have been hitting that for quite some time, so we don't really
> know what the numbers are...

Hmm, that would explain a thing or two...

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: Rough estimate of number of downloads per year/month/day?

From
"Jonathan S. Katz"
Date:
On Nov 7, 2012, at 10:42 AM, Dave Page wrote:

> On Wed, Nov 7, 2012 at 3:26 PM, Magnus Hagander <magnus@hagander.net> wrote:
>> We are fairly restrictive about giving access to that data. You'd have
>> to check that with -core.
>
> We've always said no in the past, though we have shared specific
> pre-generated reports from time to time.
>
>> Through a quick discussion with Jonathan yesterday it seems the data
>> in there isn't actually that correct anyway - it looks like we're
>> hitting the limit of what the free tier of google analytics will give
>> us, and have been hitting that for quite some time, so we don't really
>> know what the numbers are...
>
> Hmm, that would explain a thing or two...

Unfortunately, the next level up plan for Google Analytics that would get us all the data starts at $150K / year.  But
Iwonder if they have some options for nonprofits? 

Jonathan

Re: Rough estimate of number of downloads per year/month/day?

From
Dave Page
Date:
On Wed, Nov 7, 2012 at 3:44 PM, Jonathan S. Katz
<jonathan.katz@excoventures.com> wrote:
> On Nov 7, 2012, at 10:42 AM, Dave Page wrote:
>
>> On Wed, Nov 7, 2012 at 3:26 PM, Magnus Hagander <magnus@hagander.net> wrote:
>>> We are fairly restrictive about giving access to that data. You'd have
>>> to check that with -core.
>>
>> We've always said no in the past, though we have shared specific
>> pre-generated reports from time to time.
>>
>>> Through a quick discussion with Jonathan yesterday it seems the data
>>> in there isn't actually that correct anyway - it looks like we're
>>> hitting the limit of what the free tier of google analytics will give
>>> us, and have been hitting that for quite some time, so we don't really
>>> know what the numbers are...
>>
>> Hmm, that would explain a thing or two...
>
> Unfortunately, the next level up plan for Google Analytics that would get us all the data starts at $150K / year.
ButI wonder if they have some options for nonprofits? 

Wow. That's insane.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: Rough estimate of number of downloads per year/month/day?

From
Kris Pennella
Date:
Heh, I didn't think this would be a simple process.
If the site is maxing out the "free" version of Google Analytics, that means 10M+ impressions/hits (not page views) a month which is already saying something.
There is certainly some filtering that can be done in the free version that would arrive at some useful metrics for everyone.
Jonathan, what makes you think the data isn't "correct" (or being collected correctly)? 

---------------------------------------
Kris Pennella     2ndQuadrant US    
Client Services  kris@2ndQuadrant.com   
 503.236.8196
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com



On Nov 7, 2012, at 7:46 AM, Dave Page wrote:

On Wed, Nov 7, 2012 at 3:44 PM, Jonathan S. Katz
<jonathan.katz@excoventures.com> wrote:
On Nov 7, 2012, at 10:42 AM, Dave Page wrote:

On Wed, Nov 7, 2012 at 3:26 PM, Magnus Hagander <magnus@hagander.net> wrote:
We are fairly restrictive about giving access to that data. You'd have
to check that with -core.

We've always said no in the past, though we have shared specific
pre-generated reports from time to time.

Through a quick discussion with Jonathan yesterday it seems the data
in there isn't actually that correct anyway - it looks like we're
hitting the limit of what the free tier of google analytics will give
us, and have been hitting that for quite some time, so we don't really
know what the numbers are...

Hmm, that would explain a thing or two...

Unfortunately, the next level up plan for Google Analytics that would get us all the data starts at $150K / year.  But I wonder if they have some options for nonprofits?

Wow. That's insane.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


--
Sent via pgsql-advocacy mailing list (pgsql-advocacy@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-advocacy

Re: Rough estimate of number of downloads per year/month/day?

From
"Jonathan S. Katz"
Date:

On Nov 7, 2012, at 10:52 AM, Kris Pennella wrote:

On Nov 7, 2012, at 7:46 AM, Dave Page wrote:

On Wed, Nov 7, 2012 at 3:44 PM, Jonathan S. Katz
<jonathan.katz@excoventures.com> wrote:
On Nov 7, 2012, at 10:42 AM, Dave Page wrote:

On Wed, Nov 7, 2012 at 3:26 PM, Magnus Hagander <magnus@hagander.net> wrote:
We are fairly restrictive about giving access to that data. You'd have
to check that with -core.

We've always said no in the past, though we have shared specific
pre-generated reports from time to time.

Through a quick discussion with Jonathan yesterday it seems the data
in there isn't actually that correct anyway - it looks like we're
hitting the limit of what the free tier of google analytics will give
us, and have been hitting that for quite some time, so we don't really
know what the numbers are...

Hmm, that would explain a thing or two...

Unfortunately, the next level up plan for Google Analytics that would get us all the data starts at $150K / year.  But I wonder if they have some options for nonprofits?

Wow. That's insane.

Heh, I didn't think this would be a simple process.
If the site is maxing out the "free" version of Google Analytics, that means 10M+ impressions/hits (not page views) a month which is already saying something.
There is certainly some filtering that can be done in the free version that would arrive at some useful metrics for everyone.
Jonathan, what makes you think the data isn't "correct" (or being collected correctly)? 


Magnus did some rough calculations off the cuff based on some of the major-viewed pages within the GA token that the site uses and figured we are above those limits.

Jonathan

Re: Rough estimate of number of downloads per year/month/day?

From
Dave Page
Date:
On Wed, Nov 7, 2012 at 3:57 PM, Jonathan S. Katz
<jonathan.katz@excoventures.com> wrote:
>
> Heh, I didn't think this would be a simple process.
> If the site is maxing out the "free" version of Google Analytics, that means
> 10M+ impressions/hits (not page views) a month which is already saying
> something.
> There is certainly some filtering that can be done in the free version that
> would arrive at some useful metrics for everyone.
> Jonathan, what makes you think the data isn't "correct" (or being collected
> correctly)?
>
>
> Kris: http://support.google.com/analytics/bin/answer.py?hl=en&answer=1070983
>
> Magnus did some rough calculations off the cuff based on some of the
> major-viewed pages within the GA token that the site uses and figured we are
> above those limits.

Hmm, well my math doesn't quite tally with Magnus'. A quick look at a
few different recent months tells me we're hitting around 3 - 3.5M
page views per month on www.postgresql.org, with archives getting just
under 1M.

We're not tracking anything other than pages though, so I'm not sure
how impressions would actually differ from page views. Or am I missing
something?

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: Rough estimate of number of downloads per year/month/day?

From
Magnus Hagander
Date:
On Wed, Nov 7, 2012 at 5:26 PM, Dave Page <dpage@pgadmin.org> wrote:
> On Wed, Nov 7, 2012 at 3:57 PM, Jonathan S. Katz
> <jonathan.katz@excoventures.com> wrote:
>>
>> Heh, I didn't think this would be a simple process.
>> If the site is maxing out the "free" version of Google Analytics, that means
>> 10M+ impressions/hits (not page views) a month which is already saying
>> something.
>> There is certainly some filtering that can be done in the free version that
>> would arrive at some useful metrics for everyone.
>> Jonathan, what makes you think the data isn't "correct" (or being collected
>> correctly)?
>>
>>
>> Kris: http://support.google.com/analytics/bin/answer.py?hl=en&answer=1070983
>>
>> Magnus did some rough calculations off the cuff based on some of the
>> major-viewed pages within the GA token that the site uses and figured we are
>> above those limits.
>
> Hmm, well my math doesn't quite tally with Magnus'. A quick look at a
> few different recent months tells me we're hitting around 3 - 3.5M
> page views per month on www.postgresql.org, with archives getting just
> under 1M.

There are a bunch of more sites that track under the same account, I
believe. If you sum them all up, you get close enough to that
explaining things.


> We're not tracking anything other than pages though, so I'm not sure
> how impressions would actually differ from page views. Or am I missing
> something?

I don't know GA enough to actually comment on details :)

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: Rough estimate of number of downloads per year/month/day?

From
Dave Page
Date:
On Wed, Nov 7, 2012 at 4:40 PM, Magnus Hagander <magnus@hagander.net> wrote:
> On Wed, Nov 7, 2012 at 5:26 PM, Dave Page <dpage@pgadmin.org> wrote:
>> On Wed, Nov 7, 2012 at 3:57 PM, Jonathan S. Katz
>> <jonathan.katz@excoventures.com> wrote:
>>>
>>> Heh, I didn't think this would be a simple process.
>>> If the site is maxing out the "free" version of Google Analytics, that means
>>> 10M+ impressions/hits (not page views) a month which is already saying
>>> something.
>>> There is certainly some filtering that can be done in the free version that
>>> would arrive at some useful metrics for everyone.
>>> Jonathan, what makes you think the data isn't "correct" (or being collected
>>> correctly)?
>>>
>>>
>>> Kris: http://support.google.com/analytics/bin/answer.py?hl=en&answer=1070983
>>>
>>> Magnus did some rough calculations off the cuff based on some of the
>>> major-viewed pages within the GA token that the site uses and figured we are
>>> above those limits.
>>
>> Hmm, well my math doesn't quite tally with Magnus'. A quick look at a
>> few different recent months tells me we're hitting around 3 - 3.5M
>> page views per month on www.postgresql.org, with archives getting just
>> under 1M.
>
> There are a bunch of more sites that track under the same account, I
> believe. If you sum them all up, you get close enough to that
> explaining things.

There are, but www and archives are *by far* the busiest. The rest of
them are tiny in comparison, and at most would put us around the 5M
mark.

>> We're not tracking anything other than pages though, so I'm not sure
>> how impressions would actually differ from page views. Or am I missing
>> something?
>
> I don't know GA enough to actually comment on details :)

:-)

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: Rough estimate of number of downloads per year/month/day?

From
Shane Ambler
Date:
On 08/11/2012 02:14, Jonathan S. Katz wrote:

> Unfortunately, the next level up plan for Google Analytics that would
> get us all the data starts at $150K / year.  But I wonder if they
> have some options for nonprofits?
>

Anyone had a look at piwik.org ?
It aims to be an OSS alternative to google analytics.

While there appears to be some work on including PDO_PGSQL support it
is still not functional yet.
Any volunteers?



Re: Rough estimate of number of downloads per year/month/day?

From
Magnus Hagander
Date:


On Nov 8, 2012 7:44 PM, "Stefan Kaltenbrunner" <stefan@kaltenbrunner.cc> wrote:
>
> On 11/08/2012 03:38 AM, Shane Ambler wrote:
> > On 08/11/2012 02:14, Jonathan S. Katz wrote:
> >
> >> Unfortunately, the next level up plan for Google Analytics that would
> >> get us all the data starts at $150K / year.  But I wonder if they
> >> have some options for nonprofits?
> >>
> >
> > Anyone had a look at piwik.org ?
> > It aims to be an OSS alternative to google analytics.
>
> piwik is a pretty reasonable substitute for google analytics

A good start is probably to just start collecting the raw logs. If nothing else, it would let us confirm if the Google analytics numbers match or not.

And yes, this is now on our todo...

> > While there appears to be some work on including PDO_PGSQL support it
> > is still not functional yet.
> > Any volunteers?
>
> however - not having postgresql support is a blocker for the main
> infrastructure.

Indeed.

/Magnus

Re: Rough estimate of number of downloads per year/month/day?

From
Stefan Kaltenbrunner
Date:
On 11/08/2012 03:38 AM, Shane Ambler wrote:
> On 08/11/2012 02:14, Jonathan S. Katz wrote:
>
>> Unfortunately, the next level up plan for Google Analytics that would
>> get us all the data starts at $150K / year.  But I wonder if they
>> have some options for nonprofits?
>>
>
> Anyone had a look at piwik.org ?
> It aims to be an OSS alternative to google analytics.

piwik is a pretty reasonable substitute for google analytics

>
> While there appears to be some work on including PDO_PGSQL support it
> is still not functional yet.
> Any volunteers?

however - not having postgresql support is a blocker for the main
infrastructure.



Stefan


Re: Rough estimate of number of downloads per year/month/day?

From
Cédric Villemain
Date:
Le jeudi 8 novembre 2012 03:38:43, Shane Ambler a écrit :
> On 08/11/2012 02:14, Jonathan S. Katz wrote:
> > Unfortunately, the next level up plan for Google Analytics that would
> > get us all the data starts at $150K / year.  But I wonder if they
> > have some options for nonprofits?
>
> Anyone had a look at piwik.org ?
> It aims to be an OSS alternative to google analytics.

I maintained a fork of piwik for PostgreSQL (so I know it can work with
PostgreSQL). Now piwik is more 'open' and there are some efforts done in Piwik
to ease the use of PostgreSQL.

> While there appears to be some work on including PDO_PGSQL support it
> is still not functional yet.
> Any volunteers?

I am sure that if a goal is to use it for postgresql.org, then people in piwik
community will enjoy helping us.

--
Cédric Villemain +33 (0)6 20 30 22 52
http://2ndQuadrant.fr/
PostgreSQL: Support 24x7 - Développement, Expertise et Formation

Attachment

Re: Rough estimate of number of downloads per year/month/day?

From
damien clochard
Date:
>
> I thinks we need to define new metrics to monitor the evolution of the
> project in the industry. It's not easy but there must some way to mesure
> that. For example, the job trends or ML traffic could be more
> informative that the download numbers...
>
> http://www.indeed.com/jobtrends?q=postgres%2C++oracle&l=&relative=1
> http://markmail.blogspot.fr/2008/02/postgresql-more-traffic-than-mysql-and.html
>

I also found this :

http://qa.debian.org/popcon-graph.php?packages=postgresql-8.4+postgresql-9.0+postgresql-9.1+postgresql-9.2&show_installed=on&want_legend=on&want_ticks=on&from_date=2007-07-01&to_date=&hlght_date=&date_fmt=%25Y-%25m&beenhere=1

I should have thought about it earlier... The numbers are pretty low and
they're quite debian-specific (8.4 is dominant because of Squeeze). but
my guess is that the trends may be the same for other distributions. I
don't know if there's similar stats for other distributions, if you have
similar links for Ubuntu, Arch or CentOS please share :)

Anyway to go back to the good-old mysql-vs-pgsql battle, the graph below
shows clearly that the gap is still big and that it keeps growing


http://qa.debian.org/popcon-graph.php?packages=postgresql+mysql-server&show_installed=on&want_legend=on&want_ticks=on&from_date=2007-07-01&to_date=&hlght_date=&date_fmt=%25Y-%25m&beenhere=1


Re: Rough estimate of number of downloads per year/month/day?

From
Magnus Hagander
Date:
On Thu, Nov 29, 2012 at 10:38 AM, damien clochard <damien@dalibo.info> wrote:
>
>>
>> I thinks we need to define new metrics to monitor the evolution of the
>> project in the industry. It's not easy but there must some way to mesure
>> that. For example, the job trends or ML traffic could be more
>> informative that the download numbers...
>>
>> http://www.indeed.com/jobtrends?q=postgres%2C++oracle&l=&relative=1
>> http://markmail.blogspot.fr/2008/02/postgresql-more-traffic-than-mysql-and.html
>>
>
> I also found this :
>
http://qa.debian.org/popcon-graph.php?packages=postgresql-8.4+postgresql-9.0+postgresql-9.1+postgresql-9.2&show_installed=on&want_legend=on&want_ticks=on&from_date=2007-07-01&to_date=&hlght_date=&date_fmt=%25Y-%25m&beenhere=1
>
> I should have thought about it earlier... The numbers are pretty low and
> they're quite debian-specific (8.4 is dominant because of Squeeze). but
> my guess is that the trends may be the same for other distributions. I
> don't know if there's similar stats for other distributions, if you have
> similar links for Ubuntu, Arch or CentOS please share :)

I'd say they're quite useless. AFAIK it requires specific opt-in. And
as an example, while we have many clients who run PostgreSQL on
Debian, I'm pretty sure not a single one is counted there. And I think
looking at the trend is wrong too - because the more installations you
get in large companies etc, the less likely are they to opt-in.


--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: Rough estimate of number of downloads per year/month/day?

From
damien clochard
Date:
Le 29/11/2012 14:14, Magnus Hagander a écrit :
> On Thu, Nov 29, 2012 at 10:38 AM, damien clochard <damien@dalibo.info> wrote:
>>
>>>
>>> I thinks we need to define new metrics to monitor the evolution of the
>>> project in the industry. It's not easy but there must some way to mesure
>>> that. For example, the job trends or ML traffic could be more
>>> informative that the download numbers...
>>>
>>> http://www.indeed.com/jobtrends?q=postgres%2C++oracle&l=&relative=1
>>> http://markmail.blogspot.fr/2008/02/postgresql-more-traffic-than-mysql-and.html
>>>
>>
>> I also found this :
>>
http://qa.debian.org/popcon-graph.php?packages=postgresql-8.4+postgresql-9.0+postgresql-9.1+postgresql-9.2&show_installed=on&want_legend=on&want_ticks=on&from_date=2007-07-01&to_date=&hlght_date=&date_fmt=%25Y-%25m&beenhere=1
>>
>> I should have thought about it earlier... The numbers are pretty low and
>> they're quite debian-specific (8.4 is dominant because of Squeeze). but
>> my guess is that the trends may be the same for other distributions. I
>> don't know if there's similar stats for other distributions, if you have
>> similar links for Ubuntu, Arch or CentOS please share :)
>
> I'd say they're quite useless. AFAIK it requires specific opt-in. And
> as an example, while we have many clients who run PostgreSQL on
> Debian, I'm pretty sure not a single one is counted there. And I think
> looking at the trend is wrong too - because the more installations you
> get in large companies etc, the less likely are they to opt-in.
>

I agree with that but this argument applies to mysql too. So the trends
seem relevant to me if you want to compare mysql and postgres. Unless if
you assume that mysql users are more likely to opt-in than psql users,
but I can't see no evidence of that.




Re: Rough estimate of number of downloads per year/month/day?

From
Josh Berkus
Date:
> I agree with that but this argument applies to mysql too. So the trends
> seem relevant to me if you want to compare mysql and postgres. Unless if
> you assume that mysql users are more likely to opt-in than psql users,
> but I can't see no evidence of that.

Actually, there's a different bias here.  Many, if not most, people who
install Postgres on Debian get it from other sources because the version
of Debian is old.  Whereas people are more likely to install MySQL 5.1
and leave it alone, since Debian users are unlikely to be onboard with
the Oracle releases.

However, it is interesting that the number of MySQL installs on Debian
is still growing  ... says that the demise of MySQL may be premature.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com