any solution for doing a data file import spawning it on multiple processes - Mailing list pgsql-general

From hb@101-factory.eu
Subject any solution for doing a data file import spawning it on multiple processes
Date
Msg-id 8AEB78AA-CF8E-4F0F-AECF-0333451B6C3A@101-factory.eu
Whole thread Raw
Responses Re: any solution for doing a data file import spawning it on multiple processes  (Edson Richter <edsonrichter@hotmail.com>)
List pgsql-general
hi there,

I am trying to import large data files into pg.
for now i used the. xarg linux command to spawn the file line for line and set  and use the  maximum available
connections. 

we use pg pool as connection pool to the database, and so try to maximize the concurrent data import of the file.

problem for now that it seems to work well but we miss a line once in a while, and that is not acceptable. also it
createszombies ;(.  

does anybody have any other tricks that will do the job?

thanks,

Henk

pgsql-general by date:

Previous
From: Gabriele Bartolini
Date:
Subject: Re: Smaller multiple tables or one large table?
Next
From: Edson Richter
Date:
Subject: Re: any solution for doing a data file import spawning it on multiple processes