Concurrent Restores - Mailing list pgsql-hackers

From Volkan YAZICI
Subject Concurrent Restores
Date
Msg-id 87abgzpev7.fsf@alamut.mobiliz.com.tr
Whole thread Raw
Responses Re: Concurrent Restores  (Zdenek Kotala <Zdenek.Kotala@Sun.COM>)
List pgsql-hackers
Hi,

[I've searched archives for the subject, but couldn't find a related
discussion. If there is any, sorry for duplication.]

We're migrating nearly a dozen of MSSQL servers of size ~100GiB per
cluster. For this purpose, we dump MSSQL data to COPY files using a Java
program. We have database schemas for PostgreSQL which are equivalent to
their correponding ones in MSSQL side. The problem is, while we're
creating primary key, foreign key and index relations, I'm manually
partitioning related SQL files into separate files to gain performance
from CPU usage. One can argue that, concurrent processes will consume
larger disk I/O in this scheme and cause I/O bottleneck this time. But
as far as I monitored the system statistics, during concurrent
restoration, in our situation operation is CPU bounded, not disk
I/O. (Thanks SAN!)

pg_dump is capable of dumping objects with respect to their dependency
relations. It'd be really awesome if pg_dump can also handle
parallelizing primary key, foreign key and index creation queries into
separate files. Would such a think be possible? Comments?


Regards.


pgsql-hackers by date:

Previous
From: Teodor Sigaev
Date:
Subject: Re: PATCH: CITEXT 2.0
Next
From: "Marko Kreen"
Date:
Subject: Re: [PATCHES] pg_dump lock timeout