Re: Import File - Mailing list pgsql-general

From Michael Fuhr
Subject Re: Import File
Date
Msg-id 20050823151805.GA9651@winnie.fuhr.org
Whole thread Raw
In response to Import File  ("Rodrigo Africani" <rafricani@fotinisystems.com.br>)
List pgsql-general
On Tue, Aug 23, 2005 at 11:47:57AM -0300, Rodrigo Africani wrote:
> I'm trying to import a sequence txt file.
> The file has fix columns ... i.e the first 10 positions is the primary key ...
> The comand copy need a delimitier but the file is seq without a delimitier.
> I look in the manual but i don't won't if it is possible.
> My problem is the file is very large ... > 1G and I don't have a lot of memory
> to change the file and put delimitiers.

If you have enough disk space then you don't need a lot of memory
to change the file: simply run it through a filter that reads one
line at a time, changes it, and writes it to a new file.  Such
filters are trivial to write in Perl and other scripting languages.
If you don't have enough disk space then you could pipe the filter's
output into psql and use "COPY FROM stdin" -- this might be faster
anyway because it involves less disk I/O.

--
Michael Fuhr

pgsql-general by date:

Previous
From: Michael Fuhr
Date:
Subject: Re: Question about Foreign key constraint causes "costly sequential scans"?
Next
From: "Kevin Grittner"
Date:
Subject: Where to get 8.1 beta test version?