Re: dumping 8M bit fields - Mailing list pgsql-general

From Jeff Davis
Subject Re: dumping 8M bit fields
Date
Msg-id 1158944469.7578.30.camel@dogma.v10.wvs
Whole thread Raw
In response to dumping 8M bit fields  (Rajarshi Guha <rguha@indiana.edu>)
List pgsql-general
On Thu, 2006-09-21 at 20:14 -0400, Rajarshi Guha wrote:
> Hi, I have a table with 8M rows and one of the fields is a bit (1024
> bits) field. I am trying to dump the bit field for 8M rows to a file:
>
> psql -U cicc3 -A -F " " -t -o pubchem.fp -c "select cid,gfp from
> pubchem_compound;
>
> However I get
>
> out of memory for query result
>

psql is trying to load all of the data into RAM before outputting any of
it. More specifically, it's trying to load all of the output
representations of all the data into RAM before outputting it.

For 1024 bits, the output representation will be 1k. For 8M rows that's
a lot of RAM needed.

It would be better to use something like COPY or a cursor.

Regards,
    Jeff Davis


pgsql-general by date:

Previous
From: "Jim C. Nasby"
Date:
Subject: Re: CLUSTERing on Insert
Next
From: "Jim C. Nasby"
Date:
Subject: Re: postgresql rising