Thread: Some download statistics
(crossposting this to hackers, I'm sure there are interested people there as well) Since Dave set the site up for tracking clickthroughs, I hit the db with a couple of queries to count our downloads. This is what I came up with: I did some simple pivoting in Excel and split it into categories win32, source, sig (MD5 or PGP signatures), RPMs, split (the split tarballs), pgadmin and ODBC. Other stuff was so little that I cut it. The numbers for just-past-release may be off a bit because of the move of wwwmaster. Not sure what happened to the db. Just before and about 24 hours after the release of 8.0, we had the following: win32 16480 source 5223 sig 2016 rpm 1908 split 1689 pgadmin 370 odbc 261 Grand Total 27947 Total tally since the tracking started (2005-01-11): win32 106878 56,37% source 39058 20,60% rpm 11703 6,17% split 9135 4,82% sig 8931 4,71% pgadmin 7281 3,84% odbc 6601 3,48% Total 189587 And the versions being downloaded (total): 8.0.1 85511 8.0.0 82517 notpg 14526 (pgadmin, odbc etc) 7.4.6 4297 7.4.7 2736 Grand Total 189587 This does *not* include bittorrent downloads. Some conclusions that I draw from this: 1) The win32 distribution is very popular. Some of this is no doubt because it's new. Also, about 3500 of those downloads are RC5 downloads. RC5 downloads of the source were during this time so few that they didn't show up after my cutoff. 2) Most of the people who use RPMs on their linux systems wait for their distribution to ship the RPMs and don't get it. Or they are alraedy set up to download directly from their mirror and not from the website (the tracker only counts downloads that were initiated from the website) 3) There doesnt' seem to be much point to the distribution splits. A total of less than 5% the *number* of downloads. And most people probably get more than one file, so in reality that number shuold proably be divided by 4 or 5. I know several people who downloaded source *plus* the split ones, because "hey, I need postgresql. And I certainly need base too. And I need docs.". They don't realise it's included in the main tarball. Frankly, I'd suggest dropping the splits. Thoughts? 4) People upgraded to 8.0.1. Not a lot of people have upgraded their 7.4 systems. As always, it's statistics. You can say anything you want with it... Someone who chews numbers for a living might be able to get more out of it. If someone is interested in the excel pivot tables with this stuff in it, let me know. But it's nothing exciting there :) //Magnus
On Thu, 24 Feb 2005, Magnus Hagander wrote: > 3) There doesnt' seem to be much point to the distribution splits. A > total of less than 5% the *number* of downloads. And most people > probably get more than one file, so in reality that number shuold > proably be divided by 4 or 5. > I know several people who downloaded source *plus* the split ones, > because "hey, I need postgresql. And I certainly need base too. And I > need docs.". They don't realise it's included in the main tarball. > Frankly, I'd suggest dropping the splits. Thoughts? this tracks only those going in through the web ... this doesn't track those using an ftp client going in and downloading files ... specifically, I know that all of the FreeBSD ports are based on the split distributions ... in the case of the postgresql80-server port, it only downloads base and opt and ignores the rest ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Hi Magnus, On Feb 24, 2005, at 11:35 AM, Magnus Hagander wrote: > I did some simple pivoting in Excel and split it into categories win32, > source, sig (MD5 or PGP signatures), RPMs, split (the split tarballs), > pgadmin and ODBC. Other stuff was so little that I cut it. Assuming this is from a HTTP log, I'm just curious if you took into account that many Windows users have "download managers" that make repeated requests to get a single file. You see this in the log with a 206 (partial content) HTTP response code. So you might have anywhere from 10 to 20 hits in the log to download a single file. This makes it a pain to get accurate download statistics. John DeSoi, Ph.D. http://pgedit.com/ Power Tools for PostgreSQL
On Thu, 24 Feb 2005 17:35:57 +0100, Magnus Hagander wrote: > I know several people who downloaded source *plus* the split ones, > because "hey, I need postgresql. And I certainly need base too. And I > need docs.". They don't realise it's included in the main tarball. > Frankly, I'd suggest dropping the splits. Thoughts? I also found the split sources + a non-split sources version to be confusing. As you, I think that splitting should be dropped. -- Greetings from Troels Arvin, Copenhagen, Denmark
Troels Arvin <troels@arvin.dk> writes:
> On Thu, 24 Feb 2005 17:35:57 +0100, Magnus Hagander wrote:
>> Frankly, I'd suggest dropping the splits. Thoughts?
> I also found the split sources + a non-split sources version to be
> confusing. As you, I think that splitting should be dropped.
Perhaps the confusion issue could be addressed by keeping the split
sources in a separate subdirectory:
pub/source/v.8.0.1/ postgresql-8.0.1.tar.bz2 postgresql-8.0.1.tar.bz2.md5 postgresql-8.0.1.tar.gz
postgresql-8.0.1.tar.gz.md5 split-tarballs/ postgresql-base-8.0.1.tar.bz2
postgresql-base-8.0.1.tar.bz2.md5 etc...
I suspect the demand for the split tarballs is mighty low nowaways,
but it's probably not zero yet.
Also: I notice that the README file that's supposed to tell people about
the split-tarball scheme is not present in any of the recent-version
subdirectories, so it's no wonder that they are confused.
regards, tom lane
On Thu, 24 Feb 2005, Tom Lane wrote: > Troels Arvin <troels@arvin.dk> writes: >> On Thu, 24 Feb 2005 17:35:57 +0100, Magnus Hagander wrote: >>> Frankly, I'd suggest dropping the splits. Thoughts? > >> I also found the split sources + a non-split sources version to be >> confusing. As you, I think that splitting should be dropped. > > Perhaps the confusion issue could be addressed by keeping the split > sources in a separate subdirectory: > > pub/source/v.8.0.1/ > postgresql-8.0.1.tar.bz2 > postgresql-8.0.1.tar.bz2.md5 > postgresql-8.0.1.tar.gz > postgresql-8.0.1.tar.gz.md5 > split-tarballs/ > postgresql-base-8.0.1.tar.bz2 > postgresql-base-8.0.1.tar.bz2.md5 > etc... That sounds like a reasonable compromise ... I could do that right away, if nobody is in disagreement ... ? > I suspect the demand for the split tarballs is mighty low nowaways, > but it's probably not zero yet. Considering that every new FreeBSD install uses them exclusively, I don't expect it to drop to zero in the near future .. > Also: I notice that the README file that's supposed to tell people about > the split-tarball scheme is not present in any of the recent-version > subdirectories, so it's no wonder that they are confused. Actually, its never been ... just checked, its in the root directory ... but, with that in mind, I've added putting a copy in there to the release script ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
On Thursday 24 February 2005 21:43, Marc G. Fournier wrote:
[.......]
> >
> > pub/source/v.8.0.1/
> > postgresql-8.0.1.tar.bz2
> > postgresql-8.0.1.tar.bz2.md5
> > postgresql-8.0.1.tar.gz
> > postgresql-8.0.1.tar.gz.md5
> > split-tarballs/
> > postgresql-base-8.0.1.tar.bz2
> > postgresql-base-8.0.1.tar.bz2.md5
> > etc...
>
> That sounds like a reasonable compromise ... I could do that right away,
> if nobody is in disagreement ... ?
Hm, there is a distribution that uses the split tarballs: gentoo.
snipped from /usr/portage/dev-db/postgresql/postgresql-8.0.1-r1.ebuild :
SRC_URI="mirror://postgresql/source/v${PV}/${PN}-base-${MY_PV}.tar.bz2
mirror://postgresql/source/v${PV}/${PN}-opt-${MY_PV}.tar.bz2 doc? (
mirror://postgresql/source/v${PV}/${PN}-docs-${MY_PV}.tar.bz2)"
Better is to drop a 'bug' to bugs.gentoo.org so that the ebuild will be
fixed;-)
<Earny>
>>>> Frankly, I'd suggest dropping the splits. Thoughts? >> >>> I also found the split sources + a non-split sources version to be >>> confusing. As you, I think that splitting should be dropped. >> >> Perhaps the confusion issue could be addressed by keeping the split >> sources in a separate subdirectory: >> >> pub/source/v.8.0.1/ >> postgresql-8.0.1.tar.bz2 >> postgresql-8.0.1.tar.bz2.md5 >> postgresql-8.0.1.tar.gz >> postgresql-8.0.1.tar.gz.md5 >> split-tarballs/ >> postgresql-base-8.0.1.tar.bz2 >> postgresql-base-8.0.1.tar.bz2.md5 >> etc... > >That sounds like a reasonable compromise ... I could do that >right away, >if nobody is in disagreement ... ? Seems reasonable to me. >> I suspect the demand for the split tarballs is mighty low nowaways, >> but it's probably not zero yet. > >Considering that every new FreeBSD install uses them >exclusively, I don't >expect it to drop to zero in the near future .. Do they pull fromt eh pg mirrors, or from their own? I realise they still need it, of course, but it'd be interesting to know. //Magnus
> -----Original Message----- > From: pgsql-hackers-owner@postgresql.org > [mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of John DeSoi > Sent: 24 February 2005 19:20 > To: Magnus Hagander > Cc: pgsql-hackers@postgresql.org; pgsql-www@postgresql.org > Subject: Re: [HACKERS] Some download statistics > > Hi Magnus, > > On Feb 24, 2005, at 11:35 AM, Magnus Hagander wrote: > > > I did some simple pivoting in Excel and split it into > categories win32, > > source, sig (MD5 or PGP signatures), RPMs, split (the split > tarballs), > > pgadmin and ODBC. Other stuff was so little that I cut it. > > > Assuming this is from a HTTP log, It's not. When you click on a mirror flag after selecting a file under http://www.postgresql.org/ftp/ the click is logged and the client redirected to the requested file on the selected mirror. /D
"Marc G. Fournier" <scrappy@postgresql.org> writes:
> On Thu, 24 Feb 2005, Tom Lane wrote:
>> Also: I notice that the README file that's supposed to tell people about
>> the split-tarball scheme is not present in any of the recent-version
>> subdirectories, so it's no wonder that they are confused.
> Actually, its never been ... just checked, its in the root directory ...
> but, with that in mind, I've added putting a copy in there to the release
> script ...
It is present in the subdirectories for some older releases, eg v7.3.
regards, tom lane
On Thu, 24 Feb 2005, Magnus Hagander wrote:
>>>>> Frankly, I'd suggest dropping the splits. Thoughts?
>>>
>>>> I also found the split sources + a non-split sources version to be
>>>> confusing. As you, I think that splitting should be dropped.
>>>
>>> Perhaps the confusion issue could be addressed by keeping the split
>>> sources in a separate subdirectory:
>>>
>>> pub/source/v.8.0.1/
>>> postgresql-8.0.1.tar.bz2
>>> postgresql-8.0.1.tar.bz2.md5
>>> postgresql-8.0.1.tar.gz
>>> postgresql-8.0.1.tar.gz.md5
>>> split-tarballs/
>>> postgresql-base-8.0.1.tar.bz2
>>> postgresql-base-8.0.1.tar.bz2.md5
>>> etc...
>>
>> That sounds like a reasonable compromise ... I could do that
>> right away,
>> if nobody is in disagreement ... ?
>
> Seems reasonable to me.
>
>
>>> I suspect the demand for the split tarballs is mighty low nowaways,
>>> but it's probably not zero yet.
>>
>> Considering that every new FreeBSD install uses them
>> exclusively, I don't
>> expect it to drop to zero in the near future ..
>
> Do they pull fromt eh pg mirrors, or from their own? I realise they
> still need it, of course, but it'd be interesting to know.
MASTER_SITES= ${MASTER_SITE_PGSQL}
which expands to:
MASTER_SITE_PGSQL+= \ ftp://ftp3.us.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp8.us.postgresql.org/postgresql/%SUBDIR%/\ ftp://ftp9.us.postgresql.org/pub/mirrors/postgresql/%SUBDIR%/
\ ftp://ftp10.us.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp13.us.postgresql.org/mirror/postgresql/%SUBDIR%/\
${MASTER_SITE_RINGSERVER:S,%SUBDIR%,misc/db/postgresql/&,}\ ftp://ftp.au.postgresql.org/pub/postgresql/%SUBDIR%/
\ ftp://ftp.at.postgresql.org/db/www.postgresql.org/pub/%SUBDIR%/ \
ftp://ftp.be.postgresql.org/postgresql/%SUBDIR%/\ ftp://ftp.ba.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp.br.postgresql.org/pub/PostgreSQL/%SUBDIR%/\ ftp://ftp3.ca.postgresql.org/pub/%SUBDIR%/ \
ftp://ftp2.cr.postgresql.org/pub/Unix/postgres/%SUBDIR%/\
ftp://ftp.cz.postgresql.org/pub/ftp.postgresql.org/%SUBDIR%/\
ftp://ftp2.cz.postgresql.org/pub/postgresql/%SUBDIR%/\ ftp://ftp.ee.postgresql.org/mirrors/postgresql/%SUBDIR%/
\ ftp://ftp.fr.postgresql.org/%SUBDIR%/ \ ftp://ftp2.fr.postgresql.org/postgresql/%SUBDIR%/ \
ftp://ftp.de.postgresql.org/mirror/postgresql/%SUBDIR%/\
ftp://ftp2.de.postgresql.org/pub/comp/os/unix/database/postgresql/%SUBDIR%/\
ftp://ftp3.de.postgresql.org/pub/Mirrors/ftp.postgresql.org/%SUBDIR%/\
ftp://ftp4.de.postgresql.org/pub/postgresql/%SUBDIR%/\
ftp://ftp.gr.postgresql.org/pub/databases/postgresql/%SUBDIR%/\ ftp://ftp.hk.postgresql.org/postgresql/%SUBDIR%/
\ ftp://ftp2.is.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp.ie.postgresql.org/mirrors/ftp.postgresql.org/pub/%SUBDIR%/\
ftp://ftp2.it.postgresql.org/mirrors/postgres/%SUBDIR%/\
ftp://ftp.kr.postgresql.org/mirror/database/postgresql/%SUBDIR%/\
ftp://ftp.lv.postgresql.org/pub/software/postgresql/%SUBDIR%/\
ftp://ftp.eu.postgresql.org/pub/unix/db/postgresql/%SUBDIR%/\
ftp://ftp2.nl.postgresql.org/mirror/postgresql/%SUBDIR%/\
ftp://ftp4.nl.postgresql.org/postgresql.zeelandnet.nl/%SUBDIR%/\
ftp://ftp.no.postgresql.org/pub/databases/postgresql/%SUBDIR%/\
ftp://ftp6.pl.postgresql.org/pub/postgresql/%SUBDIR%/\
ftp://ftp7.pl.postgresql.org/pub/mirror/ftp.postgresql.org/%SUBDIR%/\
ftp://ftp.pt.postgresql.org/postgresql/%SUBDIR%/\
ftp://ftp6.ro.postgresql.org/pub/mirrors/ftp.postgresql.org/%SUBDIR%/\
ftp://ftp.ru.postgresql.org/pub/unix/database/pgsql/%SUBDIR%/\
ftp://ftp2.ru.postgresql.org/pub/databases/postgresql/%SUBDIR%/\
ftp://ftp3.ru.postgresql.org/pub/mirror/postgresql/pub/%SUBDIR%/\
ftp://ftp5.es.postgresql.org/mirror/postgresql/%SUBDIR%/\
ftp://ftp.se.postgresql.org/pub/databases/relational/postgresql/%SUBDIR%/\
ftp://ftp2.ch.postgresql.org/pub/postgresql/%SUBDIR%/\ ftp://ftp.tw.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp3.tw.postgresql.org/pub/postgresql/%SUBDIR%/ \
ftp://ftp.tr.postgresql.org/pub/SQL/postgresql/%SUBDIR%/\ ftp://ftp.postgresql.org/pub/%SUBDIR%/
----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
On Thu, 24 Feb 2005, Tom Lane wrote: > "Marc G. Fournier" <scrappy@postgresql.org> writes: >> On Thu, 24 Feb 2005, Tom Lane wrote: >>> Also: I notice that the README file that's supposed to tell people about >>> the split-tarball scheme is not present in any of the recent-version >>> subdirectories, so it's no wonder that they are confused. > >> Actually, its never been ... just checked, its in the root directory ... >> but, with that in mind, I've added putting a copy in there to the release >> script ... > > It is present in the subdirectories for some older releases, eg v7.3. 'k, now I see it ... I was looking for a README file .. fixed ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
On Thu, Feb 24, 2005 at 10:27:03PM +0100, Ernst Herzberg wrote:
> On Thursday 24 February 2005 21:43, Marc G. Fournier wrote:
> [.......]
> > >
> > > pub/source/v.8.0.1/
> > > postgresql-8.0.1.tar.bz2
> > > postgresql-8.0.1.tar.bz2.md5
> > > postgresql-8.0.1.tar.gz
> > > postgresql-8.0.1.tar.gz.md5
> > > split-tarballs/
> > > postgresql-base-8.0.1.tar.bz2
> > > postgresql-base-8.0.1.tar.bz2.md5
> > > etc...
> >
> > That sounds like a reasonable compromise ... I could do that right away,
> > if nobody is in disagreement ... ?
>
> Hm, there is a distribution that uses the split tarballs: gentoo.
>
> snipped from /usr/portage/dev-db/postgresql/postgresql-8.0.1-r1.ebuild :
>
> SRC_URI="mirror://postgresql/source/v${PV}/${PN}-base-${MY_PV}.tar.bz2
> mirror://postgresql/source/v${PV}/${PN}-opt-${MY_PV}.tar.bz2
> doc? ( mirror://postgresql/source/v${PV}/${PN}-docs-${MY_PV}.tar.bz2 )"
>
> Better is to drop a 'bug' to bugs.gentoo.org so that the ebuild will be
> fixed;-)
How exactly is that a bug? Why should someone spend time downloading the
docs if they don't want to install them? And gentoo isn't the only OS
that uses them.
Unless generating the splits requires a non-trivial amount of work, I
see no reason not to have them.
--
Jim C. Nasby, Database Consultant decibel@decibel.org
Give your computer some brain candy! www.distributed.net Team #1828
Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"
On Fri, 25 Feb 2005, Jim C. Nasby wrote: > Unless generating the splits requires a non-trivial amount of work, I > see no reason not to have them. its all automated ... still wish I could figure out a way of splitting off *just* libppq, since that would drop downloads of the full package significantly ... in my case, I need postgresql "server" on maybe one machine, but need just libpq on every other one that I run .. I've gotta sit down and see if I can figure out how to do thatt .. would really be interesting to see the stats if we had a seperate 'libpq' download :) ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664