Thread: [GENERAL] Restoring tables with circular references dumped to separate files

I was dumping each table to a separate file so I could pick and choose when
restoring. However, seems this was not a great idea, since two of my tables
happened to reference each other via FOREIGN KEYs, and I am not able to
restore them. Is there a way to do this without manually merging the dump
files? Thanks for guidance in advance. 



--
Sent from: http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

I would alter the tables and remove the foreign key constraint, restore the data and add the constraint back. If the data is consistent, adding the foreign key should work without error.

On Friday, October 20, 2017 8:15:27 PM PDT doganmeh wrote:

I was dumping each table to a separate file so I could pick and choose when restoring. However, seems this was not a great idea, since two of my tables happened to reference each other via FOREIGN KEYs, and I am not able to restore them. Is there a way to do this without manually merging the dump files? Thanks for guidance in advance.

— Sent from: http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html

Seems that would be easier and less error prone. Thanks, 



--
Sent from: http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Re: Restoring tables with circular references dumped toseparate files

From
Melvin Davidson
Date:


On Sat, Oct 21, 2017 at 8:24 AM, doganmeh <mehmet@edgle.com> wrote:
Seems that would be easier and less error prone. Thanks,



--
Sent from: http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


>I was dumping each table to a separate file so I could pick and choose when restoring...

It seems to me that instead on multiple single table dumps, you could take advantage of restoring from a list-file.

Just
1. do a dump with custom format ( -F c)
EG: pg_dump -U postgres -F c -t table1 -t table2 -t table3 yourdb > yourdb.dmp

2. use pg_restore -l to create a list-file
EG: pg_restore -l yourdb.dmp > yourdb.lis

3. edit yourdb.lis and comment out ( prefix with ; ) any line you don't want

4. then use pg_restore with the edited list-file 
EG:  pg_restore -L yourdb.lis yourdb.dmp

See examples at the end of

--
Melvin Davidson
I reserve the right to fantasize.  Whether or not you
wish to share my fantasy is entirely up to you.

The list approach for partial restore is also useful, thank you. 

On another note, I used to take full backups (entire database), however
switched to table by table scheme in order to make it more VCS friendly.
Namely, so I only check into github the dumps of the tables that are updated
only. 
So, from that perspective, is there a dump-restore scenario that is widely
used, but is also VCS friendly? To my knowledge, pg_restore does not restore
backups that are in "plain text" format, and compressed formats such as
"tar" would not be github friendly. 



--
Sent from: http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Re: Restoring tables with circular references dumped toseparate files

From
Melvin Davidson
Date:


On Sat, Oct 21, 2017 at 4:48 PM, doganmeh <mehmet@edgle.com> wrote:
The list approach for partial restore is also useful, thank you.

On another note, I used to take full backups (entire database), however
switched to table by table scheme in order to make it more VCS friendly.
Namely, so I only check into github the dumps of the tables that are updated
only.

So, from that perspective, is there a dump-restore scenario that is widely
used, but is also VCS friendly? To my knowledge, pg_restore does not restore
backups that are in "plain text" format, and compressed formats such as
"tar" would not be github friendly.



--
Sent from: http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general


>...is there a dump-restore scenario that is widely used, but is also VCS friendly?

You might want to give consideration to pg_extractor.



--
Melvin Davidson
I reserve the right to fantasize.  Whether or not you
wish to share my fantasy is entirely up to you.

Re: [GENERAL] Re: Restoring tables with circular references dumped toseparate files

From
Francisco Olarte
Date:
On Sat, Oct 21, 2017 at 10:48 PM, doganmeh <mehmet@edgle.com> wrote:
...
> On another note, I used to take full backups (entire database), however
> switched to table by table scheme in order to make it more VCS friendly.
> Namely, so I only check into github the dumps of the tables that are updated
> only.
> So, from that perspective, is there a dump-restore scenario that is widely
> used, but is also VCS friendly? To my knowledge, pg_restore does not restore
> backups that are in "plain text" format, and compressed formats such as
> "tar" would not be github friendly.

Not widely used, but you have the directory format ( disclaimer: have
not tested it for VCS friendliness ). It populates a directory similar
to what uncompressing a tar format would, but I do not know if it
renames the files from run to run, but should be easy to test.

Also note it is documented as compressed BY DEFAULT, but you can use
options to avoid compression, and it is the only one which supports
paralell dumps.

Also, custom and tar can be made uncompressed, but I do not think
that's a great idea.

Francisco Olarte.


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general