Thread: Modified pg_dump & new pg_restore need testing...
If anyone is interested in being sent the current sources for the new pg_dump & pg_restore, please let me know. The utilities now seem to work, but need testing. The basic idea is to use pg_dump to dump an *entire* database, and then use pg_restore to choose what is restored. The salient features are as follows: - pg_dump still used to dump database; all output is via new interface (virtually all of the pg_dump code is changed, but not the logic). The changes are relatively minor, all the same. - the '-c' option is not used in pg-dump: it now dumps the commands to delete the schema, and it is up to the user of pg_restore to decide if they are output. - the default output file format is a custom format with compressed sections (the data dumps). It is NOT a text file. - pg_restore reads the backup file and, depending on the options chosen, produces a script (to stdout) that can be sent to psql. - by default pg_restore outputs the schema/data in the order it was sent from pg_dump, but the --oid flag will send the output in order of increasing OID, and the --rearrange flag will put all 'non-parental' (??) items at the end, after the data. (eg. indexes, acls, triggers etc). Needless to say that the best results com from using both of these options. - If the -c (clear) option is chosen in pg_restore, it also dumps the 'drop' commands in reverse order at the start of the script. This *should* make it more reliable than dumping them when the item is defined. It also means that triggers can be dropped. - The --toc option shows a summary of the restore operation that would be performed if the --toc were not there. Please send me an email if you are interested and have the time to test them. Thanks, Philip Warner. ---------------------------------------------------------------- Philip Warner | __---_____ Albatross Consulting Pty. Ltd. |----/ - \ (A.C.N. 008 659 498) | /(@) ______---_ Tel: (+61) 0500 83 82 81 | _________ \ Fax: (+61) 0500 83 82 82 | ___________ | Http://www.rhyme.com.au | / \| | --________-- PGP key available upon request, | / and from pgp5.ai.mit.edu:11371 |/
Sounds like something that is a suitable replacement for what we have now
... I particularly like the 'compressed output' *but* is there some way,
for the paranoid, of doing an uncompressed format
(--dont-compress?)? Then again, how does this handle someone wanting to
dump the database for reloading into a non-PgSQL database?
On Fri, 30 Jun 2000, Philip Warner wrote:
>
> If anyone is interested in being sent the current sources for the new
> pg_dump & pg_restore, please let me know.
>
> The utilities now seem to work, but need testing.
>
> The basic idea is to use pg_dump to dump an *entire* database, and then use
> pg_restore to choose what is restored.
>
> The salient features are as follows:
>
> - pg_dump still used to dump database; all output is via new interface
> (virtually all of the pg_dump code is changed, but not the logic). The
> changes are relatively minor, all the same.
>
> - the '-c' option is not used in pg-dump: it now dumps the commands to
> delete the schema, and it is up to the user of pg_restore to decide if they
> are output.
>
> - the default output file format is a custom format with compressed
> sections (the data dumps). It is NOT a text file.
>
> - pg_restore reads the backup file and, depending on the options chosen,
> produces a script (to stdout) that can be sent to psql.
>
> - by default pg_restore outputs the schema/data in the order it was sent
> from pg_dump, but the --oid flag will send the output in order of
> increasing OID, and the --rearrange flag will put all 'non-parental' (??)
> items at the end, after the data. (eg. indexes, acls, triggers etc).
> Needless to say that the best results com from using both of these options.
>
> - If the -c (clear) option is chosen in pg_restore, it also dumps the
> 'drop' commands in reverse order at the start of the script. This *should*
> make it more reliable than dumping them when the item is defined. It also
> means that triggers can be dropped.
>
> - The --toc option shows a summary of the restore operation that would be
> performed if the --toc were not there.
>
> Please send me an email if you are interested and have the time to test them.
>
> Thanks,
>
> Philip Warner.
>
>
> ----------------------------------------------------------------
> Philip Warner | __---_____
> Albatross Consulting Pty. Ltd. |----/ - \
> (A.C.N. 008 659 498) | /(@) ______---_
> Tel: (+61) 0500 83 82 81 | _________ \
> Fax: (+61) 0500 83 82 82 | ___________ |
> Http://www.rhyme.com.au | / \|
> | --________--
> PGP key available upon request, | /
> and from pgp5.ai.mit.edu:11371 |/
>
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
At 13:53 30/06/00 -0300, The Hermit Hacker wrote: > >Sounds like something that is a suitable replacement for what we have now >... I particularly like the 'compressed output' *but* is there some way, >for the paranoid, of doing an uncompressed format >(--dont-compress?)? Then again, how does this handle someone wanting to >dump the database for reloading into a non-PgSQL database? > There will be; you can specify -z0 on the pg_dump, but it still has binary junk in the output. I will add a '--plain-text, -p' option to dump it into text. You can also use pg_restore to dump the file to text. ---------------------------------------------------------------- Philip Warner | __---_____ Albatross Consulting Pty. Ltd. |----/ - \ (A.C.N. 008 659 498) | /(@) ______---_ Tel: (+61) 0500 83 82 81 | _________ \ Fax: (+61) 0500 83 82 82 | ___________ | Http://www.rhyme.com.au | / \| | --________-- PGP key available upon request, | / and from pgp5.ai.mit.edu:11371 |/
At 13:53 30/06/00 -0300, The Hermit Hacker wrote: > >Sounds like something that is a suitable replacement for what we have now >... I particularly like the 'compressed output' *but* is there some way, >for the paranoid, of doing an uncompressed format >(--dont-compress?)? Then again, how does this handle someone wanting to >dump the database for reloading into a non-PgSQL database? > As suggested I have now added a '--plain-text, -p' option to pg_dump to dump it into text. Needless to say the output can not be used by pg_restore (although, I suppose, it is feasible). This also supports the -Z flag, to dump in gzip format. ---------------------------------------------------------------- Philip Warner | __---_____ Albatross Consulting Pty. Ltd. |----/ - \ (A.C.N. 008 659 498) | /(@) ______---_ Tel: (+61) 0500 83 82 81 | _________ \ Fax: (+61) 0500 83 82 82 | ___________ | Http://www.rhyme.com.au | / \| | --________-- PGP key available upon request, | / and from pgp5.ai.mit.edu:11371 |/
Philip Warner writes: > As suggested I have now added a '--plain-text, -p' option to pg_dump > to dump it into text. The *default* action of pg_dump should be to write plain text and to standard output. Changing that will certainly make a lot of people very unhappy. -- Peter Eisentraut Sernanders väg 10:115 peter_e@gmx.net 75262 Uppsala http://yi.org/peter-e/ Sweden
[ Charset ISO-8859-1 unsupported, converting... ] > Philip Warner writes: > > > As suggested I have now added a '--plain-text, -p' option to pg_dump > > to dump it into text. > > The *default* action of pg_dump should be to write plain text and to > standard output. Changing that will certainly make a lot of people very > unhappy. Agreed. We need a enable-compress switch. -- Bruce Momjian | http://www.op.net/~candle pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
At 23:23 1/07/00 +0200, Peter Eisentraut wrote:
>> As suggested I have now added a '--plain-text, -p' option to pg_dump
>> to dump it into text.
>
>The *default* action of pg_dump should be to write plain text and to
>standard output. Changing that will certainly make a lot of people very
>unhappy.
Good point; I'll make it the default. The -Z {0..9] switch and -F {p|c|f}
switches can be used to select output compression levels and output format.
----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.C.N. 008 659 498) | /(@) ______---_
Tel: (+61) 0500 83 82 81 | _________ \
Fax: (+61) 0500 83 82 82 | ___________ |
Http://www.rhyme.com.au | / \| | --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/
Philip, where did we leave this? > > If anyone is interested in being sent the current sources for the new > pg_dump & pg_restore, please let me know. > > The utilities now seem to work, but need testing. > > The basic idea is to use pg_dump to dump an *entire* database, and then use > pg_restore to choose what is restored. > > The salient features are as follows: > > - pg_dump still used to dump database; all output is via new interface > (virtually all of the pg_dump code is changed, but not the logic). The > changes are relatively minor, all the same. > > - the '-c' option is not used in pg-dump: it now dumps the commands to > delete the schema, and it is up to the user of pg_restore to decide if they > are output. > > - the default output file format is a custom format with compressed > sections (the data dumps). It is NOT a text file. > > - pg_restore reads the backup file and, depending on the options chosen, > produces a script (to stdout) that can be sent to psql. > > - by default pg_restore outputs the schema/data in the order it was sent > from pg_dump, but the --oid flag will send the output in order of > increasing OID, and the --rearrange flag will put all 'non-parental' (??) > items at the end, after the data. (eg. indexes, acls, triggers etc). > Needless to say that the best results com from using both of these options. > > - If the -c (clear) option is chosen in pg_restore, it also dumps the > 'drop' commands in reverse order at the start of the script. This *should* > make it more reliable than dumping them when the item is defined. It also > means that triggers can be dropped. > > - The --toc option shows a summary of the restore operation that would be > performed if the --toc were not there. > > Please send me an email if you are interested and have the time to test them. > > Thanks, > > Philip Warner. > > > ---------------------------------------------------------------- > Philip Warner | __---_____ > Albatross Consulting Pty. Ltd. |----/ - \ > (A.C.N. 008 659 498) | /(@) ______---_ > Tel: (+61) 0500 83 82 81 | _________ \ > Fax: (+61) 0500 83 82 82 | ___________ | > Http://www.rhyme.com.au | / \| > | --________-- > PGP key available upon request, | / > and from pgp5.ai.mit.edu:11371 |/ > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
At 21:01 10/10/00 -0400, Bruce Momjian wrote: >Philip, where did we leave this? In CVS. ;-). The longer answer is that it's been in CVS for a while, and I have a version for 7.0.2 which I have also been using for a while. Various people have tested it, but not as many or as much as I would like (or lots of people have tested it with no problems, which seems unlikely). The docs for pg_dump are with Thomas and I am about to start working on an unrelated bug in the way pg_dump handles sequences. When that's done, I will document pg_restore. Hopefully, all in time for beta... ---------------------------------------------------------------- Philip Warner | __---_____ Albatross Consulting Pty. Ltd. |----/ - \ (A.B.N. 75 008 659 498) | /(@) ______---_ Tel: (+61) 0500 83 82 81 | _________ \ Fax: (+61) 0500 83 82 82 | ___________ | Http://www.rhyme.com.au | / \| | --________-- PGP key available upon request, | / and from pgp5.ai.mit.edu:11371 |/