Re: break table into portions for writing to separate files - Mailing list pgsql-general

From Szymon Guz
Subject Re: break table into portions for writing to separate files
Date
Msg-id CAFjNrYs=uFC73uXymR_M8DVmDVVWZW_wJ34Za629qJNO_pga1g@mail.gmail.com
Whole thread Raw
In response to Re: break table into portions for writing to separate files  (Seb <spluque@gmail.com>)
List pgsql-general

On 1 May 2014 22:24, Seb <spluque@gmail.com> wrote:
On Thu, 1 May 2014 22:17:24 +0200,
Szymon Guz <mabewlun@gmail.com> wrote:

> On 1 May 2014 21:01, Seb <spluque@gmail.com> wrote:
>     On Thu, 1 May 2014 20:22:26 +0200,
>     Szymon Guz <mabewlun@gmail.com> wrote:

>> Hi, several Gb is about 1GB, that's not too much. In case you meant
>> 'several GB', that shouldn't be a problem as well.

>     Sorry, I meant several GB.  Although that may not be a problem for
> PostgreSQL, it is for post-processing the output file with other
> tools.

>> The first thing I'd do would be creating an index on the column used
>> for dividing the data. Then I'd just use the command COPY with a
>> proper select to save the data to a file.

>     I should have mentioned that this is quite a complex view (not a
> table), which joins several other views of similar complexity.  I'm
> not sure whether indexes are useful/feasible in this case.  I'll
> investigate.

> Yes, indexes can be used to speed up the view as well. Such a view is
> nothing more than just a query.

Help for CREATE INDEX says that its target should be a table or
materialized view, so I'm guessing you mean indexes on the relevant
fields of the underlying tables.  In that case, I already have indexes
on those, especially the timestamp fields which are the ones that are
used for the heavy query work.



Can you show us the query plan for the queries you are using, the view definition, and how you query that view?

Szymon

pgsql-general by date:

Previous
From: Seb
Date:
Subject: Re: break table into portions for writing to separate files
Next
From: Seb
Date:
Subject: Re: break table into portions for writing to separate files