In order to test a real life scenario (and use it for benchmarking) I want to load large number of data from csv files. The requirement is that the load should happen like an application writing to the database ( that is, no COPY command).
Once you have parsed the data it is fairly easy to use PostgreSQL "COPY FROM stdin" format. If you have all data with a tabulator separator. A simple table (t1) could look like:
COPY t1 (f1,f2) FROM stdin;
3<tab>Joe
7<tab>Jane
\.
These data can be piped directly to psql and it will be fast.
