Thread: Database Loading

Database Loading

From
Carol Walter
Date:
Hello,

I have to load a very large database.  Is there a command that can
load data in bulk faster than the copy command does?

Carol Walter

Re: Database Loading

From
Andrew Sullivan
Date:
On Tue, Feb 19, 2008 at 01:47:01PM -0500, Carol Walter wrote:
> I have to load a very large database.  Is there a command that can
> load data in bulk faster than the copy command does?

No.

A


Re: Database Loading

From
"Spiegelberg, Greg"
Date:
> Andrew Sullivan wrote:
>
> On Tue, Feb 19, 2008 at 01:47:01PM -0500, Carol Walter wrote:
> > I have to load a very large database.  Is there a command that can
> > load data in bulk faster than the copy command does?
>
> No.
>

There was a thread a while back regarding this.

http://archives.postgresql.org/pgsql-performance/2005-07/msg00372.php

I don't recall though if an answer was found.

Good luck.

CONFIDENTIALITY NOTICE: This e-mail communication and any attachments may contain confidential and privileged
informationfor the use of the designated recipients named above.  If you are not the intended recipient, you are hereby
notifiedthat you have received this communication in error and that any review, disclosure, dissemination, distribution
orcopying of it or its contents is prohibited.  If you have received this communication in error; please notify Cranel
Incorporatedimmediately by telephone at 614-431-8000 or 800-288-3475 and destroy all copies of this communication and
anyattachments. 


Re: Database Loading

From
"Scott Marlowe"
Date:
On Feb 19, 2008 12:47 PM, Carol Walter <walterc@indiana.edu> wrote:
> Hello,
>
> I have to load a very large database.  Is there a command that can
> load data in bulk faster than the copy command does?

pg_bulkload

http://pgfoundry.org/projects/pgbulkload/

I've not used it, but have seen it mentioned several times.

Re: Database Loading

From
Carol Walter
Date:
I went to the page.  It said the OS is Linux.  Is there anything for
Solaris. I've got Solaris 10.

Carol
On Feb 19, 2008, at 3:19 PM, Scott Marlowe wrote:

> On Feb 19, 2008 12:47 PM, Carol Walter <walterc@indiana.edu> wrote:
>> Hello,
>>
>> I have to load a very large database.  Is there a command that can
>> load data in bulk faster than the copy command does?
>
> pg_bulkload
>
> http://pgfoundry.org/projects/pgbulkload/
>
> I've not used it, but have seen it mentioned several times.
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster


Re: Database Loading

From
"Peter Koczan"
Date:
On Feb 19, 2008 3:04 PM, Carol Walter <walterc@indiana.edu> wrote:
> I went to the page.  It said the OS is Linux.  Is there anything for
> Solaris. I've got Solaris 10.

It looks like it's just some C code, a shell file, and some sql
scripts. It should compile and run fine on any postgresql supported
platform (the operative word being "should"). They probably just
listed Linux as the OS as it's likely the primary OS on which they
wrote and tested it.

Peter

Re: Database Loading

From
Dimitri Fontaine
Date:
Hi,

Le mardi 19 février 2008, Carol Walter a écrit :
> I have to load a very large database.  Is there a command that can
> load data in bulk faster than the copy command does?

The next version of pgloader has been asked to implement some new parallel
loading of a single file, which could speed up things by having several
backends importing data at once. The implementation of this is now done and
ready to get some tests, but still at a beta or release candidate status.

If you want to try it (and report about your tests?), here's the current
version of it:
 http://pgloader.projects.postgresql.org/dev/
 http://pgloader.projects.postgresql.org/dev/pgloader.1.html#_parallel_loading

Regards,
--
dim

Attachment