Thread: Fastest way to load data in a table

Hi all.

I need to clean and fill periodically a database with several hundred thousands
records readed from a plain ascii file.

I have a Python script that opens the file and then, after a 'DELETE *
FROM table'  submits an INSERT for each registry, but this process is very slow.

How can I accelerate this uploading process ?

Best regards,

Antonio Navarro Navarro
BemarNet Management
http://www.bemarnet.es
hostmaster@bemarnet.es

On Wed, Sep 26, 2001 at 11:17:10AM +0200, Antonio Navarro Navarro wrote:
> Hi all.
>
> I need to clean and fill periodically a database with several hundred thousands
> records readed from a plain ascii file.
>
> I have a Python script that opens the file and then, after a 'DELETE *
> FROM table'  submits an INSERT for each registry, but this process is very slow.
>
> How can I accelerate this uploading process ?

Are you using BEGIN and COMMIT to avoid transaction overhead?
What about using COPY?
What about using TRUNCATE?

HTH,
--
Martijn van Oosterhout <kleptog@svana.org>
http://svana.org/kleptog/
> Magnetism, electricity and motion are like a three-for-two special offer:
> if you have two of them, the third one comes free.