Re: Static mirror generation - Mailing list pgsql-www

From Dave Page
Subject Re: Static mirror generation
Date
Msg-id E7F85A1B5FF8D44C8A1AF6885BC9A0E40C360D@ratbert.vale-housing.co.uk
Whole thread Raw
In response to Static mirror generation  (Alexey Borzov <borz_off@cs.msu.su>)
Responses Re: Static mirror generation
List pgsql-www

> -----Original Message-----
> From: pgsql-www-owner@postgresql.org
> [mailto:pgsql-www-owner@postgresql.org] On Behalf Of Alexey Borzov
> Sent: 17 June 2004 15:19
> To: pgsql-www@postgresql.org
> Subject: [pgsql-www] Static mirror generation
>
> Greetings.
>
> I've commited a script for static mirror generation. Unlike
> previous such scripts, it is just a generic spider that
> follows links and does not have any knowledge about the site
> structure. This immediately helped to fix several problems.
>
> The mirror of website not including docs is generated in ~5
> minutes. I don't yet know how long it will take with all the
> docs, got tired after the first 1,5 hours.

Meep, that's slow. The current build takes just a few minutes when the
server is behaving. Still, the idea of using a crawler is a good one -
at least that way nothing will get forgotten, and presumably it will
create a report of any broken links?

> I've set up a proof-of-concept static mirror @
> http://oc.cs.msu.su/pgorg/
>
> The mirror uses Apache's content negotiation, so that if you
> have Russian set up as the preferred language in your
> browser, it'll come up in Russian, else in English.

OK, sounds good. Nice work :-)

Regards, Dave.

pgsql-www by date:

Previous
From: "Marc G. Fournier"
Date:
Subject: Re: New News Entry
Next
From: Alexey Borzov
Date:
Subject: Re: Static mirror generation