Thread: pgsql: Add ideas for concurrent pg_dump and pg_restore: < * pg_dump > *
pgsql: Add ideas for concurrent pg_dump and pg_restore: < * pg_dump > *
From
momjian@postgresql.org (Bruce Momjian)
Date:
Log Message: ----------- Add ideas for concurrent pg_dump and pg_restore: < * pg_dump > * pg_dump / pg_restore > o Allow pg_dump to utilize multiple CPUs and I/O channels by dumping > multiple objects simultaneously > > The difficulty with this is getting multiple dump processes to > produce a single dump output file. > http://archives.postgresql.org/pgsql-hackers/2008-02/msg00205.php > > o Allow pg_restore to utilize multiple CPUs and I/O channels by > restoring multiple objects simultaneously > > This might require a pg_restore flag to indicate how many > simultaneous operations should be performed. Only pg_dump's > -Fc format has the necessary dependency information. > > o To better utilize resources, restore data, primary keys, and > indexes for a single table before restoring the next table > > Hopefully this will allow the CPU-I/O load to be more uniform > for simultaneous restores. The idea is to start data restores > for several objects, and once the first object is done, to move > on to its primary keys and indexes. Over time, simultaneous > data loads and index builds will be running. > > o To better utilize resources, allow pg_restore to check foreign > keys simultaneously, where possible > o Allow pg_restore to create all indexes of a table > concurrently, via a single heap scan > > This requires a pg_dump -Fc file because that format contains > the required dependency information. > http://archives.postgresql.org/pgsql-general/2007-05/msg01274.php > > o Allow pg_restore to load different parts of the COPY data > simultaneously < single heap scan, and have a restore of a pg_dump somehow use it > single heap scan, and have pg_restore use it < http://archives.postgresql.org/pgsql-general/2007-05/msg01274.php Modified Files: -------------- pgsql/doc: TODO (r1.2251 -> r1.2252) (http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/doc/TODO?r1=1.2251&r2=1.2252) pgsql/doc/src/FAQ: TODO.html (r1.760 -> r1.761) (http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/doc/src/FAQ/TODO.html?r1=1.760&r2=1.761)