Re: [psycopg] speed concerns with executemany() - Mailing list psycopg

From Oswaldo
Subject Re: [psycopg] speed concerns with executemany()
Date
Msg-id d64343e3-05b6-1385-8322-12a0650f21af@soft-com.es
Whole thread Raw
In response to Re: [psycopg] speed concerns with executemany()  (Oswaldo <listas@soft-com.es>)
List psycopg
El 02/01/17 a las 20:33, Oswaldo escribió:
> El 02/01/17 a las 17:07, Daniele Varrazzo escribió:
>> On Mon, Jan 2, 2017 at 4:35 PM, Adrian Klaver
>> <adrian.klaver@aklaver.com> wrote:
>>>
>>> With NRECS=10000 and page size=100:
>>>
>>> aklaver@tito:~> python psycopg_executemany.py -p 100
>>> classic: 427.618795156 sec
>>> joined: 7.55754685402 sec
>>
>
> Hello,
>
> There is a third option that provides a small improvement: generate a
> single sql with multple values.
>
> - Test with local database:
>
> classic: 1.53970813751 sec
> joined: 0.564052820206 sec
> joined values: 0.175103187561 sec
>
> - Test with db on an internet server
>
> classic: 236.342775822 sec
> joined: 6.08789801598 sec
> joined values: 4.49090409279 sec
>
> I often need to move data between different internet servers (sql server
> <-> Postgresql). In my experience this is the fastest way to move
> hundreds of thousands of data records.
>
> I attach the sample modified with it executemany3 function.
>

Sorry i forgot the file.



Attachment

psycopg by date:

Previous
From: Oswaldo
Date:
Subject: Re: [psycopg] speed concerns with executemany()
Next
From: Jim Nasby
Date:
Subject: Re: [psycopg] Solving the SQL composition problem