Thread: Proposal to include --exclude-extension Flag in pg_dump

Proposal to include --exclude-extension Flag in pg_dump

From
Ayush Vatsa
Date:
Hi PostgreSQL Community,
Recently I have been working on pg_dump regarding my project and wanted to exclude an extension from the dump generated. I wonder why it doesn't have --exclude-extension type of support whereas --extension exists!
Since I needed that support, I took the initiative to contribute to the community by adding the --exclude-extension flag.
Attached is the patch for the same. Looking forward to your feedback.

Regards
Ayush Vatsa
Amazon Web services (AWS)
Attachment

Re: Proposal to include --exclude-extension Flag in pg_dump

From
Ayush Vatsa
Date:
Added a CF entry for the same https://commitfest.postgresql.org/46/4721/

Regards
Ayush Vatsa
Amazon Web Services (AWS)

On Mon, 25 Dec 2023 at 15:48, Ayush Vatsa <ayushvatsa1810@gmail.com> wrote:
Hi PostgreSQL Community,
Recently I have been working on pg_dump regarding my project and wanted to exclude an extension from the dump generated. I wonder why it doesn't have --exclude-extension type of support whereas --extension exists!
Since I needed that support, I took the initiative to contribute to the community by adding the --exclude-extension flag.
Attached is the patch for the same. Looking forward to your feedback.

Regards
Ayush Vatsa
Amazon Web services (AWS)

Re: Proposal to include --exclude-extension Flag in pg_dump

From
Junwang Zhao
Date:
Hi

On Mon, Dec 25, 2023 at 6:22 PM Ayush Vatsa <ayushvatsa1810@gmail.com> wrote:
>
> Added a CF entry for the same https://commitfest.postgresql.org/46/4721/
>
> Regards
> Ayush Vatsa
> Amazon Web Services (AWS)
>
> On Mon, 25 Dec 2023 at 15:48, Ayush Vatsa <ayushvatsa1810@gmail.com> wrote:
>>
>> Hi PostgreSQL Community,
>> Recently I have been working on pg_dump regarding my project and wanted to exclude an extension from the dump
generated.I wonder why it doesn't have --exclude-extension type of support whereas --extension exists! 
>> Since I needed that support, I took the initiative to contribute to the community by adding the --exclude-extension
flag.
>> Attached is the patch for the same. Looking forward to your feedback.
>>
>> Regards
>> Ayush Vatsa
>> Amazon Web services (AWS)

  printf(_("  -e, --extension=PATTERN      dump the specified
extension(s) only\n"));
+ printf(_("  --exclude-extension=PATTERN  do NOT dump the specified
extension(s)\n"));
  printf(_("  -E, --encoding=ENCODING      dump the data in encoding
ENCODING\n"));

long options should not mess with short options, does the following
make sense to you?

 printf(_(" --enable-row-security enable row security (dump only
content user has\n"
             "                                    access to)\n"));
+ printf(_("  --exclude-extension=PATTERN  do NOT dump the specified
extension(s)\n"));
 printf(_(" --exclude-table-and-children=PATTERN\n"

--
Regards
Junwang Zhao



Re: Proposal to include --exclude-extension Flag in pg_dump

From
Ayush Vatsa
Date:
Hi,
> long options should not mess with short options, does the following
> make sense to you?
Yes that makes sense, a reason to keep them together is that they are of the same kind
But I will update the patch accordingly.

One more thing I wanted to ask is, Should I separate them in the pg_dump.sgml file too? Like writing documentation of --exclude-extension with other long options?

After your feedback I will update on both places in a single patch

Regards,
Ayush Vatsa
Amazon Web Services (AWS)

On Mon, 25 Dec 2023 at 16:28, Junwang Zhao <zhjwpku@gmail.com> wrote:
Hi

On Mon, Dec 25, 2023 at 6:22 PM Ayush Vatsa <ayushvatsa1810@gmail.com> wrote:
>
> Added a CF entry for the same https://commitfest.postgresql.org/46/4721/
>
> Regards
> Ayush Vatsa
> Amazon Web Services (AWS)
>
> On Mon, 25 Dec 2023 at 15:48, Ayush Vatsa <ayushvatsa1810@gmail.com> wrote:
>>
>> Hi PostgreSQL Community,
>> Recently I have been working on pg_dump regarding my project and wanted to exclude an extension from the dump generated. I wonder why it doesn't have --exclude-extension type of support whereas --extension exists!
>> Since I needed that support, I took the initiative to contribute to the community by adding the --exclude-extension flag.
>> Attached is the patch for the same. Looking forward to your feedback.
>>
>> Regards
>> Ayush Vatsa
>> Amazon Web services (AWS)

  printf(_("  -e, --extension=PATTERN      dump the specified
extension(s) only\n"));
+ printf(_("  --exclude-extension=PATTERN  do NOT dump the specified
extension(s)\n"));
  printf(_("  -E, --encoding=ENCODING      dump the data in encoding
ENCODING\n"));

long options should not mess with short options, does the following
make sense to you?

 printf(_(" --enable-row-security enable row security (dump only
content user has\n"
             "                                    access to)\n"));
+ printf(_("  --exclude-extension=PATTERN  do NOT dump the specified
extension(s)\n"));
 printf(_(" --exclude-table-and-children=PATTERN\n"

--
Regards
Junwang Zhao

Re: Proposal to include --exclude-extension Flag in pg_dump

From
Ashutosh Bapat
Date:
On Mon, Dec 25, 2023 at 3:48 PM Ayush Vatsa <ayushvatsa1810@gmail.com> wrote:
>
> Hi PostgreSQL Community,
> Recently I have been working on pg_dump regarding my project and wanted to exclude an extension from the dump
generated.I wonder why it doesn't have --exclude-extension type of support whereas --extension exists! 
> Since I needed that support, I took the initiative to contribute to the community by adding the --exclude-extension
flag.

Aren't extensions excluded by default? That's why we have --extension.
Why do we need to explicitly exclude extensions?

--
Best Wishes,
Ashutosh Bapat



Re: Proposal to include --exclude-extension Flag in pg_dump

From
Ayush Vatsa
Date:

On Mon, 1 Jan 2024 at 18:18, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> wrote:
On Mon, Dec 25, 2023 at 3:48 PM Ayush Vatsa <ayushvatsa1810@gmail.com> wrote:
>
> Hi PostgreSQL Community,
> Recently I have been working on pg_dump regarding my project and wanted to exclude an extension from the dump generated. I wonder why it doesn't have --exclude-extension type of support whereas --extension exists!
> Since I needed that support, I took the initiative to contribute to the community by adding the --exclude-extension flag.

Aren't extensions excluded by default? That's why we have --extension.
Why do we need to explicitly exclude extensions?

--
Best Wishes,
Ashutosh Bapat

Re: Proposal to include --exclude-extension Flag in pg_dump

From
Dean Rasheed
Date:
On Mon, 1 Jan 2024 at 13:28, Ayush Vatsa <ayushvatsa1810@gmail.com> wrote:
>
> According to the documentation of pg_dump when the --extension option is not specified, all non-system extensions in
thetarget database will get dumped.
 
> > Why do we need to explicitly exclude extensions?
> Hence to include only a few we use --extension, but to exclude a few I am proposing --exclude-extension.
>

Thanks for working on this. It seems like a useful feature to have.
The code changes look good, and it appears to work as expected.

In my opinion the order of options in pg_dump.sgml and the --help
output is fine. Keeping this new option together with -e/--extension
makes it easier to see, while otherwise it would get lost much further
down. Other opinions on that might differ though.

There are a couple of things missing from the patch, that should be added:

1). The --filter option should be extended to support "exclude
extension pattern" lines in the filter file. That syntax is already
accepted, but it throws a not-supported error, but it's hopefully not
too hard to make that work now.

2). It ought to have some tests in the test script.

Regards,
Dean



Re: Proposal to include --exclude-extension Flag in pg_dump

From
Ayush Vatsa
Date:

Hi,

> In my opinion the order of options in pg_dump.sgml and the --help

> output is fine. Keeping this new option together with -e/--extension

> makes it easier to see, while otherwise it would get lost much further

> down.

I agree with your suggestion, so I'll maintain the original order as proposed.



> The --filter option should be extended to support "exclude

> extension pattern" lines in the filter file. That syntax is already

> accepted, but it throws a not-supported error, but it's hopefully not

> too hard to make that work now.

While proposing the --exclude-extension flag in pg_dump, the --filter option

wasn't there, so it got overlooked. But now that I've familiarized myself with it

and have tried to enhance its functionality with exclude-extension in the

provided patch. Thank you for bringing this to my attention.



> It ought to have some tests in the test script.

Correct, I must have added it in the first patch itself.

As a newcomer to the database, I explored how testing works and tried

to include test cases for the newly added flag. I've confirmed that all test

cases, including the ones I added, are passing.



Attached is the complete patch with all the required code changes.

Looking forward to your review and feedback.


On Wed, 6 Mar 2024 at 16:04, Dean Rasheed <dean.a.rasheed@gmail.com> wrote:
On Mon, 1 Jan 2024 at 13:28, Ayush Vatsa <ayushvatsa1810@gmail.com> wrote:
>
> According to the documentation of pg_dump when the --extension option is not specified, all non-system extensions in the target database will get dumped.
> > Why do we need to explicitly exclude extensions?
> Hence to include only a few we use --extension, but to exclude a few I am proposing --exclude-extension.
>

Thanks for working on this. It seems like a useful feature to have.
The code changes look good, and it appears to work as expected.

In my opinion the order of options in pg_dump.sgml and the --help
output is fine. Keeping this new option together with -e/--extension
makes it easier to see, while otherwise it would get lost much further
down. Other opinions on that might differ though.

There are a couple of things missing from the patch, that should be added:

1). The --filter option should be extended to support "exclude
extension pattern" lines in the filter file. That syntax is already
accepted, but it throws a not-supported error, but it's hopefully not
too hard to make that work now.

2). It ought to have some tests in the test script.

Regards,
Dean
Attachment

Re: Proposal to include --exclude-extension Flag in pg_dump

From
Dean Rasheed
Date:
On Sat, 16 Mar 2024 at 17:36, Ayush Vatsa <ayushvatsa1810@gmail.com> wrote:
>
> Attached is the complete patch with all the required code changes.
> Looking forward to your review and feedback.
>

This looks good to me. I tested it and everything worked as expected.

I ran it through pgindent to fix some whitespace issues and added
another test for the filter option, based on the test case you added.

I'm marking this ready-for-commit (which I'll probably do myself in a
day or two, unless anyone else claims it first).

Regards,
Dean

Attachment

Re: Proposal to include --exclude-extension Flag in pg_dump

From
Daniel Gustafsson
Date:
> On 19 Mar 2024, at 12:19, Dean Rasheed <dean.a.rasheed@gmail.com> wrote:

> I'm marking this ready-for-commit (which I'll probably do myself in a
> day or two, unless anyone else claims it first).

LGTM too from a read through.  I did notice a few mistakes in the --filter
documentation portion for other keywords but that's unrelated to this patch,
will fix them once this is in to avoid conflicts.

--
Daniel Gustafsson




Re: Proposal to include --exclude-extension Flag in pg_dump

From
Ayush Vatsa
Date:
> I ran it through pgindent to fix some whitespace issues and added
> another test for the filter option, based on the test case you added.

Thank you for addressing those whitespaces issues and adding more tests. I appreciate your attention to detail and will certainly be more vigilant in future.


> I'm marking this ready-for-commit (which I'll probably do myself in a
> day or two, unless anyone else claims it first).

Thank you very much, this marks my first contribution to the open-source community, and I'm enthusiastic about making further meaningful contributions to PostgreSQL in the future.


> LGTM too from a read through.  I did notice a few mistakes in the --filter
> documentation portion for other keywords but that's unrelated to this patch,
> will fix them once this is in to avoid conflicts.

Thanks Daniel for your review. It's gratifying to see that my patch not only introduced the intended feature but also brought other minor mistakes to light.

Regards
Ayush Vatsa
Amazon Web services (AWS)


On Tue, 19 Mar 2024 at 17:23, Daniel Gustafsson <daniel@yesql.se> wrote:
> On 19 Mar 2024, at 12:19, Dean Rasheed <dean.a.rasheed@gmail.com> wrote:

> I'm marking this ready-for-commit (which I'll probably do myself in a
> day or two, unless anyone else claims it first).

LGTM too from a read through.  I did notice a few mistakes in the --filter
documentation portion for other keywords but that's unrelated to this patch,
will fix them once this is in to avoid conflicts.

--
Daniel Gustafsson

Re: Proposal to include --exclude-extension Flag in pg_dump

From
Dean Rasheed
Date:
On Tue, 19 Mar 2024 at 19:17, Ayush Vatsa <ayushvatsa1810@gmail.com> wrote:
>
> > I'm marking this ready-for-commit (which I'll probably do myself in a
> > day or two, unless anyone else claims it first).
>
> Thank you very much, this marks my first contribution to the open-source community, and I'm enthusiastic about making
furthermeaningful contributions to PostgreSQL in the future.
 
>

Committed. Congratulations on your first contribution to PostgreSQL!
May it be the first of many to come.

Regards,
Dean



Re: Proposal to include --exclude-extension Flag in pg_dump

From
Dean Rasheed
Date:
On Tue, 19 Mar 2024 at 11:53, Daniel Gustafsson <daniel@yesql.se> wrote:
>
> I did notice a few mistakes in the --filter
> documentation portion for other keywords but that's unrelated to this patch,
> will fix them once this is in to avoid conflicts.
>

Attached is a patch for the --filter docs, covering the omissions I can see.

Regards,
Dean

Attachment

Re: Proposal to include --exclude-extension Flag in pg_dump

From
Ayush Vatsa
Date:
> Attached is a patch for the --filter docs, covering the omissions I can see.
Thanks Dean for working on this.
I have reviewed the changes and they look good to me.

Regards,
Ayush Vatsa
Amazon Web services (AWS)

On Fri, 7 Jun 2024 at 15:50, Dean Rasheed <dean.a.rasheed@gmail.com> wrote:
On Tue, 19 Mar 2024 at 11:53, Daniel Gustafsson <daniel@yesql.se> wrote:
>
> I did notice a few mistakes in the --filter
> documentation portion for other keywords but that's unrelated to this patch,
> will fix them once this is in to avoid conflicts.
>

Attached is a patch for the --filter docs, covering the omissions I can see.

Regards,
Dean

Re: Proposal to include --exclude-extension Flag in pg_dump

From
Dean Rasheed
Date:
On Sat, 8 Jun 2024 at 19:39, Ayush Vatsa <ayushvatsa1810@gmail.com> wrote:
>
> > Attached is a patch for the --filter docs, covering the omissions I can see.
> Thanks Dean for working on this.
> I have reviewed the changes and they look good to me.
>

Thanks for checking. I have committed this now.

Regards,
Dean