Thread: TR: Issue: --exclude-schema flag not working with pgrestore

TR: Issue: --exclude-schema flag not working with pgrestore

From
ROCHER Julien
Date:

Hi,

 

I don’t expect the --exclude-schema flag (-N) to restore chosen schema with pgrestore tool, but it actually tries to create the schema then its table.

I deliberately keep the same db name in the below example because this is actually what I’m trying to do (restore a dump from a similar db but excluding some already existing schemas on it), but I guess you don’t need to deal with this detail.

Did I misunderstand this command ?

 

  ~ psql -U julien -W -p 5432 -d postgres  -c "create schema schemaA; create schema schemaB;"

Password:

CREATE SCHEMA

CREATE SCHEMA

 

  ~ psql -U julien -W -p 5432 -d postgres  -c "create table schemaA.tableA(f int); create table schemaB.tableB(f int);"

Password:

CREATE TABLE

CREATE TABLE

 

  ~ pg_dump -p 5432 -W -d postgres --format=c > dump.sql

Password:

 

  ~ pg_restore -p 5432 -W -d postgres --verbose  -N schemaB dump.sql

pg_restore: connecting to database for restore

Password:

pg_restore: creating SCHEMA "schemaa"

pg_restore: while PROCESSING TOC:

pg_restore: from TOC entry 6; 2615 16389 SCHEMA schemaa julien

pg_restore: error: could not execute query: ERROR:  schema "schemaa" already exists

Command was: CREATE SCHEMA schemaa;

 

pg_restore: creating SCHEMA "schemab"

pg_restore: from TOC entry 7; 2615 16390 SCHEMA schemab julien

pg_restore: error: could not execute query: ERROR:  schema "schemab" already exists

Command was: CREATE SCHEMA schemab;

 

pg_restore: creating TABLE "schemaa.tablea"

pg_restore: from TOC entry 216; 1259 16391 TABLE tablea julien

pg_restore: error: could not execute query: ERROR:  relation "tablea" already exists

Command was: CREATE TABLE schemaa.tablea (

    f integer

);

 

pg_restore: creating TABLE "schemab.tableb"

pg_restore: from TOC entry 217; 1259 16394 TABLE tableb julien

pg_restore: error: could not execute query: ERROR:  relation "tableb" already exists

Command was: CREATE TABLE schemab.tableb (

    f integer

);

 

pg_restore: processing data for table "schemaa.tablea"

pg_restore: processing data for table "schemab.tableb"

pg_restore: warning: errors ignored on restore: 4

 

 

Julien ROCHER

Développeur Team Data - Migration

 

Emeria Technologies

76-78 Avenue des Champs Elysées – 75008 Paris 

06 17 17 60 64

julien.rocher@emeria.eu

 

Re: TR: Issue: --exclude-schema flag not working with pgrestore

From
Julien Rouhaud
Date:
Hi,

On Wed, Jan 11, 2023 at 09:07:08AM +0000, ROCHER Julien wrote:
> 
> I don’t expect the --exclude-schema flag (-N) to restore chosen schema with
> pgrestore<https://docs.postgresql.fr/11/app-pgrestore.html> tool, but it
> actually tries to create the schema then its table.  I deliberately keep the
> same db name in the below example because this is actually what I’m trying to
> do (restore a dump from a similar db but excluding some already existing
> schemas on it), but I guess you don’t need to deal with this detail.  Did I
> misunderstand this command ?

You didn't, but made an error in how SQL object names work:

> ➜  ~ psql -U julien -W -p 5432 -d postgres  -c "create schema schemaA; create schema schemaB;"

this creates schemaa and schemab, not schemaA and schemaB.  If you want objects
to be case sensitive at the SQL level, you need to double quote identifiers,
like "schemaB".

> ➜  ~ pg_restore -p 5432 -W -d postgres --verbose  -N schemaB dump.sql

this on the other hand is a command line utility, and will use the given case
(and properly quote identifers at the SQL leve), so it means "schemaB" which
isn't found and thus not filtered.



Re: TR: Issue: --exclude-schema flag not working with pgrestore

From
ROCHER Julien
Date:

Hi again,

 

I was wondering though, do you know why it is different from pgdump  -N behavior (Do not dump any schemas matching pattern) ?

 

 

Julien ROCHER

Développeur Team Data - Migration

 

Emeria Technologies

76-78 Avenue des Champs Elysées – 75008 Paris 

06 17 17 60 64

julien.rocher@emeria.eu

 

 

De : ROCHER Julien <julien.rocher@emeria.eu>
Date : mercredi, 11 janvier 2023 à 15:16
À : Julien Rouhaud <rjuju123@gmail.com>
Objet : Re: TR: Issue: --exclude-schema flag not working with pgrestore

Ok thank you for you feedback

 

Julien ROCHER

Développeur Team Data - Migration

 

Emeria Technologies

76-78 Avenue des Champs Elysées – 75008 Paris 

06 17 17 60 64

julien.rocher@emeria.eu

 

 

 

 

 

 

 

 

De : Julien Rouhaud <rjuju123@gmail.com>
Date : mercredi, 11 janvier 2023 à 14:27
À : ROCHER Julien <julien.rocher@emeria.eu>
Objet : Re: TR: Issue: --exclude-schema flag not working with pgrestore

On Wed, Jan 11, 2023 at 10:43:10AM +0000, ROCHER Julien wrote:
>
> Indeed this partially solved the problem. However, although it doesn’t try to
> restore the “country.number" table, it still tries to create the “country”
> schema. Do you think it’s wanted ?

Yes that's the wanted behavior.  Per documentation:

 -N schema / --exclude-schema=schema
    Do not restore objects that are in the named schema

So it filters objects **in** the schema but not the schema itself.

If you want to also ignore the CREATE SCHEMA order, or some fancier behavior,
you need to use the --list / --use-list and cherry pick the parts you want to
keep or discard.