Hello!
I have a huge dictionary table with series data generated by a third-party
service. The table consists of 2 columns
- id : serial, primary key
- series : varchar, not null, indexed
From time to time I need to apply a "patch" to the dictionary, the patch file
consists of "series" data, one per line.
Now I need to import the patch into the database, and produce another file as
- if the passed "series" field exists in the database, then return ID:series
- otherwise insert a new row to the table and generate new ID and return ID:series
for each row in the source file.
So the new file will contain both ID and series data, separated by tab or
something.
While reading and writing the data is not a question (I simply described the
whole task just in case), I wonder what is the most efficient way of importing
such a data into a table, keeping in mind that
- the dictionary table already consists of ~200K records
- the patch could be ~1-50K of records long
- records could not be removed from the dictionary, only added if not exist
Thanks!
--
Eugene Dzhurinsky
