Re: Newbie: PG8 and text file parsing - Mailing list pgsql-general

From Tatsuo Ishii
Subject Re: Newbie: PG8 and text file parsing
Date
Msg-id 20050219.111501.104033145.t-ishii@sra.co.jp
Whole thread Raw
In response to Newbie: PG8 and text file parsing  (Paul R <blueaxon@gmail.com>)
List pgsql-general
> I need some advice.. I am new to PG8 - I am working on the project
> that needs to load data daily to PostgreSQL from the text file.. the
> text file is 40MB comma delimited file with row consisting of 20-30
> fields and couple thousands of rows..
>
> so the data needs to be parsed and if the row exists it needs to be
> updated, otherwise new record is going to be created. My problem is
> with parsing:
>
> 1-what would be a good approach to do it? is plpgSQL suitable for
> this? or should I use PHP or other language to to that and run as a
> cron job/service?

There are some tools which can read a text file and treat it as if
it's a real table. Using them would make your job much easier.

ftp://ftp.sra.co.jp/pub/cmd/postgres/exttable/exttable-0.5.tar.gz
http://pgfoundry.org/projects/dbi-link/

> 2-how about committing transactions?  would there be a problem if it
> is done at the end of all the updates?

Not at all, I think.

pgsql-general by date:

Previous
From: "Joshua D. Drake"
Date:
Subject: Re: Newbie: PG8 and text file parsing
Next
From: Richard Huxton
Date:
Subject: Re: I'm newbie