Re: Bulk Insert/Update Scenario - Mailing list pgsql-general

From Peter J. Holzer
Subject Re: Bulk Insert/Update Scenario
Date
Msg-id 20180131173251.lxiqp3yyy5q7fgvn@hjp.at
Whole thread Raw
In response to Re: Bulk Insert/Update Scenario  (Mana M <manan.gcs@gmail.com>)
List pgsql-general
On 2018-01-04 13:47:49 -0800, Mana M wrote:
> I was thinking about dumping everything into TEMP table and using that
> as source for INSERT ... ON CONFLICT. However, I was not sure on how
> to get thousands of rows from my Python application into TEMP table in
> one shot. Or is there any better alternatives?

Psycopg2 can decent support for copy:
http://initd.org/psycopg/docs/cursor.html#cursor.copy_from
(but I had problems copying more than 2GB from a StringIO buffer. Not
sure whether that was a problem with psycopg or StringIO.)

Alternatively you can just invoke psql -c '\copy ...'

        hp

--
   _  | Peter J. Holzer    | we build much bigger, better disasters now
|_|_) |                    | because we have much more sophisticated
| |   | hjp@hjp.at         | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson <https://www.edge.org/>

Attachment

pgsql-general by date:

Previous
From: Stefan Blanke
Date:
Subject: Re: ERROR: invalid memory alloc request size 1073741824
Next
From: HIRTZ Jorge Alberto TENARIS
Date:
Subject: RE: PostgreSQL Kerberos Authentication