Thread: New site - minor problem

New site - minor problem

From
"Dave Page"
Date:
I suspected this would be a problem:

The new website uses ip2country to figure out where to send users.
However, the site is spidered and static pages generated for mirroring
in .uk

I would rather remove the ip2country than have every hit directed to my
server. Any major objections?

/D

Re: New site - minor problem

From
"Dave Page"
Date:

> -----Original Message-----
> From: pgsql-www-owner@postgresql.org
> [mailto:pgsql-www-owner@postgresql.org] On Behalf Of Dave Page
> Sent: 06 December 2004 11:02
> To: pgsql-www@postgresql.org
> Subject: [pgsql-www] New site - minor problem
>
> I suspected this would be a problem:
>
> The new website uses ip2country to figure out where to send users.
> However, the site is spidered and static pages generated for
> mirroring in .uk
>
> I would rather remove the ip2country than have every hit
> directed to my server. Any major objections?

Should add to that - all this will do is change the format of the mirror
selection pages by not listing local servers at the top.

/D

Re: New site - minor problem

From
Alexey Borzov
Date:
Hi,

Dave Page wrote:
> I suspected this would be a problem:
>
> The new website uses ip2country to figure out where to send users.
> However, the site is spidered and static pages generated for mirroring
> in .uk
>
> I would rather remove the ip2country than have every hit directed to my
> server. Any major objections?

If you have problems with directing traffic to your server, then we'll
probably need to find "real" hosting. Because we'll still need to direct
all form submissions to your server (uncluding voting in surveys).

A major objection to mirroring the lists of mirrors is that we risk
having a stale mirror having a list of even more stale (and non-working)
mirrors.


Re: New site - minor problem

From
"Dave Page"
Date:

> -----Original Message-----
> From: Alexey Borzov [mailto:borz_off@cs.msu.su]
> Sent: 06 December 2004 11:22
> To: Dave Page
> Cc: pgsql-www@postgresql.org
> Subject: Re: [pgsql-www] New site - minor problem
>
> If you have problems with directing traffic to your server,
> then we'll probably need to find "real" hosting. Because
> we'll still need to direct all form submissions to your
> server (uncluding voting in surveys).

The form submissions will not be a problem as they are a minimal amount
of traffic. Handling tens or hundreds of thousands of hits per day for
mirrors is a lot more. The box I'm using is on a 2Mb leased line (which
costs an arm and a leg here btw). If the traffic impacts our normal
business in any way, it will have to be removed - hence I'm being
cautious.

> A major objection to mirroring the lists of mirrors is that
> we risk having a stale mirror having a list of even more
> stale (and non-working) mirrors.

That happens quite rarely, and even when it does, 99% of users will
simply choose another mirror. This is also how it is setup now...

/D

Re: New site - minor problem

From
Alexey Borzov
Date:
Hi,

Dave Page wrote:
>>If you have problems with directing traffic to your server,
>>then we'll probably need to find "real" hosting. Because
>>we'll still need to direct all form submissions to your
>>server (uncluding voting in surveys).
>
> The form submissions will not be a problem as they are a minimal amount
> of traffic. Handling tens or hundreds of thousands of hits per day for
> mirrors is a lot more. The box I'm using is on a 2Mb leased line (which
> costs an arm and a leg here btw). If the traffic impacts our normal
> business in any way, it will have to be removed - hence I'm being
> cautious.

Ok, I see.

But then I have another question: why are we moving the dynamic site
from one inadequate server (due to overload) to the other (due to
bandwidth limits)? Especially when there were proposals of better
hosting options on this list?

Re: New site - minor problem

From
"Dave Page"
Date:

> -----Original Message-----
> From: Alexey Borzov [mailto:borz_off@cs.msu.su]
> Sent: 06 December 2004 13:19
> To: Dave Page
> Cc: pgsql-www@postgresql.org
> Subject: Re: [pgsql-www] New site - minor problem
>
> Ok, I see.
>
> But then I have another question: why are we moving the
> dynamic site from one inadequate server (due to overload) to
> the other (due to bandwidth limits)?

Bandwidth should not be an issue except for *possibly* the ip2country
thing, which as I've stated on a number of occasions is, in my opinion,
a lot of overhead for very little gain on any server.

Regardless however, I doubt 2megs will cause any problems - like I said,
I'm being cautious.

> Especially when there
> were proposals of better hosting options on this list?

None that I'm aware of that gave us the level of control over software
installation/backups etc. that we like.

Regards, Dave.

Re: New site - minor problem

From
Justin Clift
Date:
Dave Page wrote:
<snip>
 > If the traffic impacts our normal
> business in any way, it will have to be removed - hence I'm being
> cautious.

Ok.  If it starts to be a problem then it can be put elsewhere.

If no-one else with dedicated infrastructure steps up, it'd be no
problem for us.  We've got a server in the US with a couple of TB's of
unused traffic per month pre-paid for it.

Regards and best wishes,

Justin Clift

Re: New site - minor problem

From
"Gavin M. Roy"
Date:
Two things, I have the resources to host it, and I would be happy to
donate my apache module "Paleface IP2C" which can do IP2Country lookups
with very little overhead.  With it installed there should be very
little overhead in IP2Country lookups (read 100's of times faster) than
anything else I've played with module, or php based.  On a P4-3ghz one
can expect lookups in the 0.0007 second range.

Gavin

Dave Page wrote:

>
>
>
>
>>-----Original Message-----
>>From: Alexey Borzov [mailto:borz_off@cs.msu.su]
>>Sent: 06 December 2004 13:19
>>To: Dave Page
>>Cc: pgsql-www@postgresql.org
>>Subject: Re: [pgsql-www] New site - minor problem
>>
>>Ok, I see.
>>
>>But then I have another question: why are we moving the
>>dynamic site from one inadequate server (due to overload) to
>>the other (due to bandwidth limits)?
>>
>>
>
>Bandwidth should not be an issue except for *possibly* the ip2country
>thing, which as I've stated on a number of occasions is, in my opinion,
>a lot of overhead for very little gain on any server.
>
>Regardless however, I doubt 2megs will cause any problems - like I said,
>I'm being cautious.
>
>
>
>>Especially when there
>>were proposals of better hosting options on this list?
>>
>>
>
>None that I'm aware of that gave us the level of control over software
>installation/backups etc. that we like.
>
>Regards, Dave.
>
>---------------------------(end of broadcast)---------------------------
>TIP 6: Have you searched our list archives?
>
>               http://archives.postgresql.org
>
>


Re: New site - minor problem

From
"Dave Page"
Date:

> -----Original Message-----
> From: Gavin M. Roy [mailto:gmr@ehpg.net]
> Sent: 06 December 2004 16:17
> To: Dave Page
> Cc: Alexey Borzov; pgsql-www@postgresql.org
> Subject: Re: [pgsql-www] New site - minor problem
>
> Two things, I have the resources to host it, and I would be
> happy to donate my apache module "Paleface IP2C" which can do
> IP2Country lookups with very little overhead.  With it
> installed there should be very little overhead in IP2Country
> lookups (read 100's of times faster) than anything else I've
> played with module, or php based.  On a P4-3ghz one can
> expect lookups in the 0.0007 second range.

OK, some questions:

1) What backups do you run?
2) Will we (ie. myself, Marc and Robert) have root access?
3) I take it a PostgreSQL installation is no problem?
4) Where do I get back the hours I just spent setting up this server?
:-)

I would also add that I'm not committing to anything at this point, just
asking the questions. I've also spent too long setting this all up over
here to be overly enthusiastic to do it all again for the sake of one
very minor feature. The resources I have are certainly more than enough
without it, and most likely wouldn't cause problems even with it
enabled. Like I said, I'm just being cautious.

Personally, I would prefer for you to host something larger, such as
pgFoundry - but we'd need to convince Marc about that one!

Regards, Dave

Re: New site - minor problem

From
Robert Bernier
Date:
Is that the cvs on http://software77.net/cgi-bin/ip-country/geo-ip.pl ?

I've been auditing it to put on the SRA America site.

On December 6, 2004 11:16 am, Gavin M. Roy wrote:
> Two things, I have the resources to host it, and I would be happy to
> donate my apache module "Paleface IP2C" which can do IP2Country lookups
> with very little overhead.  With it installed there should be very
> little overhead in IP2Country lookups (read 100's of times faster) than
> anything else I've played with module, or php based.  On a P4-3ghz one
> can expect lookups in the 0.0007 second range.

Re: New site - minor problem

From
"Marc G. Fournier"
Date:
On Mon, 6 Dec 2004, Dave Page wrote:

>
>
>> -----Original Message-----
>> From: Alexey Borzov [mailto:borz_off@cs.msu.su]
>> Sent: 06 December 2004 11:22
>> To: Dave Page
>> Cc: pgsql-www@postgresql.org
>> Subject: Re: [pgsql-www] New site - minor problem
>>
>> If you have problems with directing traffic to your server,
>> then we'll probably need to find "real" hosting. Because
>> we'll still need to direct all form submissions to your
>> server (uncluding voting in surveys).
>
> The form submissions will not be a problem as they are a minimal amount
> of traffic.

I thought form submissions went to scripts.postgresql.org anyway .. ?

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664

Re: New site - minor problem

From
"Gavin M. Roy"
Date:
I'll address hosting in another email.  My primary offer, knowing the
politics of PgSQL related site hosting, is a closed-source prorpietary
module that should very easily handle the ip to country needs of the
launch.  It provides the country as an environment variable, and has
very, very little overhead.  It is production stable and currently
handles 20 million plus hits per day for some of my customers.

Gavin

Dave Page wrote:

>
>
>
>
>>-----Original Message-----
>>From: Gavin M. Roy [mailto:gmr@ehpg.net]
>>Sent: 06 December 2004 16:17
>>To: Dave Page
>>Cc: Alexey Borzov; pgsql-www@postgresql.org
>>Subject: Re: [pgsql-www] New site - minor problem
>>
>>Two things, I have the resources to host it, and I would be
>>happy to donate my apache module "Paleface IP2C" which can do
>>IP2Country lookups with very little overhead.  With it
>>installed there should be very little overhead in IP2Country
>>lookups (read 100's of times faster) than anything else I've
>>played with module, or php based.  On a P4-3ghz one can
>>expect lookups in the 0.0007 second range.
>>
>>
>
>OK, some questions:
>
>1) What backups do you run?
>2) Will we (ie. myself, Marc and Robert) have root access?
>3) I take it a PostgreSQL installation is no problem?
>4) Where do I get back the hours I just spent setting up this server?
>:-)
>
>I would also add that I'm not committing to anything at this point, just
>asking the questions. I've also spent too long setting this all up over
>here to be overly enthusiastic to do it all again for the sake of one
>very minor feature. The resources I have are certainly more than enough
>without it, and most likely wouldn't cause problems even with it
>enabled. Like I said, I'm just being cautious.
>
>Personally, I would prefer for you to host something larger, such as
>pgFoundry - but we'd need to convince Marc about that one!
>
>Regards, Dave
>
>


Re: New site - minor problem

From
"Dave Page"
Date:

> -----Original Message-----
> From: Marc G. Fournier [mailto:scrappy@postgresql.org]
> Sent: 06 December 2004 19:19
> To: Dave Page
> Cc: Alexey Borzov; pgsql-www@postgresql.org
> Subject: Re: [pgsql-www] New site - minor problem
>
> I thought form submissions went to scripts.postgresql.org anyway .. ?
>

They do at the moment, but they'll need to come to the master site as
that's where the database is.

BTW, we'll also need to figure out how to get the DNS zonefile from the
master to you. How about I provide a section of the zonefile for you to
rsync an import as required?

Regards, Dave

Re: New site - minor problem

From
"Dave Page"
Date:

> -----Original Message-----
> From: Gavin M. Roy [mailto:gmr@ehpg.net]
> Sent: 06 December 2004 19:52
> To: Dave Page
> Cc: pgsql-www@postgresql.org
> Subject: Re: [pgsql-www] New site - minor problem
>
> I'll address hosting in another email.  My primary offer,
> knowing the politics of PgSQL related site hosting, is a
> closed-source prorpietary module that should very easily
> handle the ip to country needs of the launch.  It provides
> the country as an environment variable, and has very, very
> little overhead.  It is production stable and currently
> handles 20 million plus hits per day for some of my customers.

Personally I'm not fanatically against closed source code - I'll use
whatever is best for the job at an appropriate price. Is this your own
code, and would you donate it wherever we ran the master site?

Regards, Dave.

Re: New site - minor problem

From
"Gavin M. Roy"
Date:
This is my own code from a start company I am working on -
http://www.palefacemodules.com/ - pardon the very sparse website... ;)
I would be donating the license, and providing free database updates, as
needed.

Gavin

Dave Page wrote:

>
>
>
>
>>-----Original Message-----
>>From: Gavin M. Roy [mailto:gmr@ehpg.net]
>>Sent: 06 December 2004 19:52
>>To: Dave Page
>>Cc: pgsql-www@postgresql.org
>>Subject: Re: [pgsql-www] New site - minor problem
>>
>>I'll address hosting in another email.  My primary offer,
>>knowing the politics of PgSQL related site hosting, is a
>>closed-source prorpietary module that should very easily
>>handle the ip to country needs of the launch.  It provides
>>the country as an environment variable, and has very, very
>>little overhead.  It is production stable and currently
>>handles 20 million plus hits per day for some of my customers.
>>
>>
>
>Personally I'm not fanatically against closed source code - I'll use
>whatever is best for the job at an appropriate price. Is this your own
>code, and would you donate it wherever we ran the master site?
>
>Regards, Dave.
>
>


Re: New site - minor problem

From
"Joshua D. Drake"
Date:
Dave Page wrote:
>
>
>
>>-----Original Message-----
>>From: Gavin M. Roy [mailto:gmr@ehpg.net]
>>Sent: 06 December 2004 19:52
>>To: Dave Page
>>Cc: pgsql-www@postgresql.org
>>Subject: Re: [pgsql-www] New site - minor problem
>>
>>I'll address hosting in another email.  My primary offer,
>>knowing the politics of PgSQL related site hosting, is a
>>closed-source prorpietary module that should very easily
>>handle the ip to country needs of the launch.  It provides
>>the country as an environment variable, and has very, very
>>little overhead.  It is production stable and currently
>>handles 20 million plus hits per day for some of my customers.
>
>
> Personally I'm not fanatically against closed source code - I'll use
> whatever is best for the job at an appropriate price. Is this your own
> code, and would you donate it wherever we ran the master site?

I would say, if there is not a reasonably open source alternative then
go for it... I have not problem with us using a closed source module
if it is the best for the job.

Sincerely,

Joshua D. Drake




>
> Regards, Dave.
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
>       subscribe-nomail command to majordomo@postgresql.org so that your
>       message can get through to the mailing list cleanly


--
Command Prompt, Inc., home of PostgreSQL Replication, and plPHP.
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL

Attachment

Re: New site - minor problem

From
John Hansen
Date:
> I would say, if there is not a reasonably open source alternative then
> go for it... I have not problem with us using a closed source module
> if it is the best for the job.

What's wrong with using maxmind's ip2country database?

Having read the thread, I'm still unclear as to what the problem really
is.

... John


Re: New site - minor problem

From
"Dave Page"
Date:

> -----Original Message-----
> From: John Hansen [mailto:john@geeknet.com.au]
> Sent: 06 December 2004 21:31
> To: Joshua D. Drake
> Cc: Dave Page; Gavin M. Roy; pgsql-www@postgresql.org
> Subject: Re: [pgsql-www] New site - minor problem
>
> > I would say, if there is not a reasonably open source
> alternative then
> > go for it... I have not problem with us using a closed
> source module
> > if it is the best for the job.
>
> What's wrong with using maxmind's ip2country database?
>
> Having read the thread, I'm still unclear as to what the
> problem really is.

The problem is that we go to great lengths to mirror the site and
minimise the points of failure, and then add code in that forces all
selection of those mirrors downloads of our software through 1 database
dependent script that provides very little benefit to the end user.

My earlier concern was that it would have too great of an effect on our
leased line, but having looked at the server logs and done a little
math, I realise now I was worrying about nothing.

Regards, Dave.

Re: New site - minor problem

From
John Hansen
Date:
> > Having read the thread, I'm still unclear as to what the
> > problem really is.
>
> The problem is that we go to great lengths to mirror the site and
> minimise the points of failure, and then add code in that forces all
> selection of those mirrors downloads of our software through 1 database
> dependent script that provides very little benefit to the end user.

Maxminds API's are not nescessarily database dependant. I even think
they have an apache module. If not, writing one would be trivial.

However all these solutions would require mirrors to install the modules
on their servers no? or are we generating the mirror selection pages
statically, customised to the mirror region? if so, what exactly would
the ip2country modules be used for?

Sorry if I sound a bit lost... :)

... John


Re: New site - minor problem

From
"Dave Page"
Date:

> -----Original Message-----
> From: John Hansen [mailto:john@geeknet.com.au]
> Sent: 06 December 2004 21:55
> To: Dave Page
> Cc: Joshua D. Drake; Gavin M. Roy; pgsql-www@postgresql.org
> Subject: RE: [pgsql-www] New site - minor problem
>
> Maxminds API's are not nescessarily database dependant. I
> even think they have an apache module. If not, writing one
> would be trivial.
>
> However all these solutions would require mirrors to install
> the modules on their servers no? or are we generating the
> mirror selection pages statically, customised to the mirror
> region? if so, what exactly would the ip2country modules be used for?

No, that's the point. All the mirrors would be selected on the master
server, giving a single point of failure. I'm far from convinced that
it's worth it. The *only* benefit to users is that the flags for their
country would be shown seperately on the mirror selection pages, saving
them may 2 seconds to spot their flag in the main grid.

Regards, Dave.

Re: New site - minor problem

From
John Hansen
Date:
> No, that's the point. All the mirrors would be selected on the master
> server, giving a single point of failure. I'm far from convinced that
> it's worth it. The *only* benefit to users is that the flags for their
> country would be shown seperately on the mirror selection pages, saving
> them may 2 seconds to spot their flag in the main grid.

In that case I vote to send that idea to the trash can..... :)

Unless mirrors are prepared to perform the ip2ccode lokups...
Single point of failure kinda defeats the purpose I reckon.

... John



Re: New site - minor problem

From
"Gavin M. Roy"
Date:
As I understand it, the new site will examine the person's ip address
and redirect them to the proper mirror for their country, if available.
The module I am offering will allow this to happen faster with less
overhead than current open source implementations.

Gavin

On Mon, 2004-12-06 at 22:01 +0000, Dave Page wrote:
>
> > -----Original Message-----
> > From: John Hansen [mailto:john@geeknet.com.au]
> > Sent: 06 December 2004 21:55
> > To: Dave Page
> > Cc: Joshua D. Drake; Gavin M. Roy; pgsql-www@postgresql.org
> > Subject: RE: [pgsql-www] New site - minor problem
> >
> > Maxminds API's are not nescessarily database dependant. I
> > even think they have an apache module. If not, writing one
> > would be trivial.
> >
> > However all these solutions would require mirrors to install
> > the modules on their servers no? or are we generating the
> > mirror selection pages statically, customised to the mirror
> > region? if so, what exactly would the ip2country modules be used for?
>
> No, that's the point. All the mirrors would be selected on the master
> server, giving a single point of failure. I'm far from convinced that
> it's worth it. The *only* benefit to users is that the flags for their
> country would be shown seperately on the mirror selection pages, saving
> them may 2 seconds to spot their flag in the main grid.
>
> Regards, Dave.
>
> ---------------------------(end of broadcast)---------------------------
> TIP 7: don't forget to increase your free space map settings


Re: New site - minor problem

From
"Dave Page"
Date:

> -----Original Message-----
> From: John Hansen [mailto:john@geeknet.com.au]
> Sent: 06 December 2004 22:09
> To: Dave Page
> Cc: Joshua D. Drake; Gavin M. Roy; pgsql-www@postgresql.org
> Subject: RE: [pgsql-www] New site - minor problem
>
> > No, that's the point. All the mirrors would be selected on
> the master
> > server, giving a single point of failure. I'm far from
> convinced that
> > it's worth it. The *only* benefit to users is that the
> flags for their
> > country would be shown seperately on the mirror selection pages,
> > saving them may 2 seconds to spot their flag in the main grid.
>
> In that case I vote to send that idea to the trash can..... :)
>
> Unless mirrors are prepared to perform the ip2ccode lokups...

No, we keep them pure HTML for simplicity.

> Single point of failure kinda defeats the purpose I reckon.

Yup, my feelings precisely.

/D

Re: New site - minor problem

From
"Dave Page"
Date:

> -----Original Message-----
> From: pgsql-www-owner@postgresql.org
> [mailto:pgsql-www-owner@postgresql.org] On Behalf Of Gavin M. Roy
> Sent: 06 December 2004 22:15
> To: pgsql-www@postgresql.org
> Subject: Re: [pgsql-www] New site - minor problem
>
> As I understand it, the new site will examine the person's ip
> address and redirect them to the proper mirror for their
> country, if available.

No, it doesn't. All it does is reorganise the page of flags so those in
the user's country are shown at the top.

Automatic selection of mirrors has caused big problems in the (distant)
past for the project, and only about a week ago for me on php.net :-(
Whatever we do, the user must have some control so they can easily
select a different mirror in the case of a problem with any site, even
the primary.

Regards, Dave.

Re: New site - minor problem

From
John Hansen
Date:
> Automatic selection of mirrors has caused big problems in the (distant)
> past for the project, and only about a week ago for me on php.net :-(

hehe, I always use www.dk.php.net, since the search function on
www.au.php.net is so sluggish it's unusable....

> Whatever we do, the user must have some control so they can easily
> select a different mirror in the case of a problem with any site, even
> the primary.

agreed...

... John