Skipping duplicate records? - Mailing list pgsql-general

From Steve Micallef
Subject Skipping duplicate records?
Date
Msg-id 20010607094751.S20209-100000@toaster.syd.ot
Whole thread Raw
Responses Re: Skipping duplicate records?  (Marc SCHAEFER <schaefer@alphanet.ch>)
List pgsql-general
Hi,

I've recently migrated from MySQL to PostgreSQL and as impressed as I am
with Postgres, I have found one seemingly missing feature to be a little
bothersome..

'mysqlimport' has the ability to skip duplicate records when doing bulk
imports from non-binary files. PostgreSQL doesn't seem to have this
feature, and it causes a problem for me as I import extremely large
amounts of data into Postgres using 'copy' and it rejects the whole file
if one record breaches the primary key.

I have managed to get around this by hacking
src/backend/access/nbtree/nbtinsert.c to call elog with NOTICE instead of
ERROR, causing it to skip the duplicate record and continue importing.

Is there a way to get around this without changing the code? If not, will
a future release of Postgres optionally implement this?

Thanks in advance,

Steve Micallef


pgsql-general by date:

Previous
From: Stephan Szabo
Date:
Subject: Re: using create constraint trigger
Next
From: will trillich
Date:
Subject: Re: Re: Updating views