Re: Mass-Data question - Mailing list pgsql-general

From Curt Sampson
Subject Re: Mass-Data question
Date
Msg-id Pine.NEB.4.43.0204161209400.498-100000@angelic.cynic.net
Whole thread Raw
In response to Re: Mass-Data question  (Gunther Schadow <gunther@aurora.regenstrief.org>)
Responses Re: Mass-Data question  (Boris Köster <koester@x-itec.de>)
List pgsql-general
> Boris Kster wrote:
>
> > What should I do if the disk-space is not enough? Is there something
> > to distribute data over several machines and to collect the data with
> > a select statement if required?

> Hmm, interesting. I have similar needs.

As do I. Unfortuantely, I'm not a guru. But I'll be testing out
something like this in the next few weeks if all goes well. I was
planning to do some fairly simple data partitioning. My initial
plan is to drop the data into multiple tables across multiple
servers, partitioned by date, and have a master table indicating
the names of the various tables and the date ranges they cover.
The application will then deal with determining which tables the
query will be spread across, construct and submit the appropriate
queries (eventually in parallel, if I'm getting a lot of queries
crossing multiple tables), and collate the results.

cjs
--
Curt Sampson  <cjs@cynic.net>   +81 90 7737 2974   http://www.netbsd.org
    Don't you know, in this new Dark Age, we're all light.  --XTC


pgsql-general by date:

Previous
From: Tom Lane
Date:
Subject: Re: speeding up subqueries
Next
From: Curt Sampson
Date:
Subject: Re: psql and passwords