Thread: test_pg_dump missing cleanup actions
Hi Stephen, While hacking another patch, I have noticed that triggerring multiple times in a row installcheck on test_pg_dump results in a failure because it is missing clean up actions on the role regress_dump_test_role. Roles are shared objects, so I think that we ought to not let traces of it when doing any regression tests on a running instance. Attached is a patch to clean up things. Thanks, -- Michael
Attachment
Michael Paquier <michael@paquier.xyz> writes:
> While hacking another patch, I have noticed that triggerring multiple
> times in a row installcheck on test_pg_dump results in a failure because
> it is missing clean up actions on the role regress_dump_test_role.
> Roles are shared objects, so I think that we ought to not let traces of
> it when doing any regression tests on a running instance.
> Attached is a patch to clean up things.
I'm confused. Isn't the point of that script exactly to create a modified
extension for testing pg_dump with?
What I'd do is leave the final state as-is and add a "drop role if exists"
at the start, similar to what some of the core regression tests do.
regards, tom lane
Greetings, * Tom Lane (tgl@sss.pgh.pa.us) wrote: > Michael Paquier <michael@paquier.xyz> writes: > > While hacking another patch, I have noticed that triggerring multiple > > times in a row installcheck on test_pg_dump results in a failure because > > it is missing clean up actions on the role regress_dump_test_role. > > Roles are shared objects, so I think that we ought to not let traces of > > it when doing any regression tests on a running instance. > > > Attached is a patch to clean up things. > > I'm confused. Isn't the point of that script exactly to create a modified > extension for testing pg_dump with? > > What I'd do is leave the final state as-is and add a "drop role if exists" > at the start, similar to what some of the core regression tests do. I've not followed this thread but based on Tom's response, I agree with his suggestion of what to do here. Thanks! Stephen
Attachment
On Tue, Sep 04, 2018 at 06:02:51PM -0400, Stephen Frost wrote: > * Tom Lane (tgl@sss.pgh.pa.us) wrote: >> Michael Paquier <michael@paquier.xyz> writes: >>> While hacking another patch, I have noticed that triggerring multiple >>> times in a row installcheck on test_pg_dump results in a failure because >>> it is missing clean up actions on the role regress_dump_test_role. >>> Roles are shared objects, so I think that we ought to not let traces of >>> it when doing any regression tests on a running instance. >> >>> Attached is a patch to clean up things. >> >> I'm confused. Isn't the point of that script exactly to create a modified >> extension for testing pg_dump with? Not really, the regression tests run for pg_regress and the TAP test suite are two completely isolated things and share no dependencies. e54f757 has actually changed test_pg_dump.sql so as it adds regression tests for pg_init_privs for ALTER EXTENSION ADD/DROP, and visibly those have been added to test_pg_dump because they were easier to add there, and this has no interactions with pg_dump. What I think should have been done initially is to add those new tests in test_extensions instead. >> What I'd do is leave the final state as-is and add a "drop role if exists" >> at the start, similar to what some of the core regression tests do. > > I've not followed this thread but based on Tom's response, I agree with > his suggestion of what to do here. Not as far as I can see.. Please note that using installcheck on the main regression test suite does not leave around any extra roles. I can understand the role of having a DROP ROLE IF EXISTS though: if you get a crash while testing, then the beginning of the tests are repeatable, so independently of the root issue Tom's suggestion makes sense to me. -- Michael
Attachment
Hi Stephen, On Tue, Sep 04, 2018 at 04:14:15PM -0700, Michael Paquier wrote: > On Tue, Sep 04, 2018 at 06:02:51PM -0400, Stephen Frost wrote: >> * Tom Lane (tgl@sss.pgh.pa.us) wrote: >>> I'm confused. Isn't the point of that script exactly to create a modified >>> extension for testing pg_dump with? > > Not really, the regression tests run for pg_regress and the TAP test > suite are two completely isolated things and share no dependencies. > e54f757 has actually changed test_pg_dump.sql so as it adds regression > tests for pg_init_privs for ALTER EXTENSION ADD/DROP, and visibly those > have been added to test_pg_dump because they were easier to add there, > and this has no interactions with pg_dump. What I think should have > been done initially is to add those new tests in test_extensions > instead. I am able to come back to this thread, and I still don't grep from where sql/test_pg_dump.sql is called. Stephen, if the test suite is aiming at tracking that pg_init_privs is correctly set up with ALTER EXTENSION ADD/DROP, shouldn't it also query the catalog to make sure that the correct entries are added and removed after running the so-said command? At least that's one way I could see to perform the necessary sanity checks without running directly pg_dump. Perhaps I am missing something? >>> What I'd do is leave the final state as-is and add a "drop role if exists" >>> at the start, similar to what some of the core regression tests do. >> >> I've not followed this thread but based on Tom's response, I agree with >> his suggestion of what to do here. > > Not as far as I can see.. Please note that using installcheck on the > main regression test suite does not leave around any extra roles. I can > understand the role of having a DROP ROLE IF EXISTS though: if you get a > crash while testing, then the beginning of the tests are repeatable, so > independently of the root issue Tom's suggestion makes sense to me. Attached is a patch with more comments about the intents of the test suite, and the separate issue pointed out by Tom fixed. It seems to me that actually checking the contents of pg_init_privs would improve the reason why the test exists.. I would welcome input about this last point. -- Michael
Attachment
On Thu, Sep 06, 2018 at 09:20:15AM -0700, Michael Paquier wrote: > Attached is a patch with more comments about the intents of the test > suite, and the separate issue pointed out by Tom fixed. It seems to me > that actually checking the contents of pg_init_privs would improve the > reason why the test exists.. I would welcome input about this last > point. So, Stephen, any input to offer? This has been around for the last three weeks. I am tracking this thread in the section for older bugs in v11 open items. -- Michael
Attachment
Michael, * Michael Paquier (michael@paquier.xyz) wrote: > On Thu, Sep 06, 2018 at 09:20:15AM -0700, Michael Paquier wrote: > > Attached is a patch with more comments about the intents of the test > > suite, and the separate issue pointed out by Tom fixed. It seems to me > > that actually checking the contents of pg_init_privs would improve the > > reason why the test exists.. I would welcome input about this last > > point. > > So, Stephen, any input to offer? This has been around for the last > three weeks. I am tracking this thread in the section for older bugs in > v11 open items. It's on my list of things to look at and hope to do so soon. Thanks! Stephen
Attachment
Hi Stephen, On Thu, Sep 27, 2018 at 06:49:26PM -0400, Stephen Frost wrote: > * Michael Paquier (michael@paquier.xyz) wrote: >> On Thu, Sep 06, 2018 at 09:20:15AM -0700, Michael Paquier wrote: >>> Attached is a patch with more comments about the intents of the test >>> suite, and the separate issue pointed out by Tom fixed. It seems to me >>> that actually checking the contents of pg_init_privs would improve the >>> reason why the test exists.. I would welcome input about this last >>> point. >> >> So, Stephen, any input to offer? This has been around for the last >> three weeks. I am tracking this thread in the section for older bugs in >> v11 open items. > > It's on my list of things to look at and hope to do so soon. So it has already been two months. Do you have any input to offer or should I do the cleanup myself? If none, then I still propose my cleanup patch from upthread, and I propose to apply it within the next couple of days if there are no objections. The annoying part about this stuff is that when running tests on Windows we get failures after multiple runs on the same installed instance, and the module is not marked with NO_INSTALLCHECK. I just got annoyed by that test a portion of this week to test some other patch with MSVC as well :( -- Michael
Attachment
Michael, * Michael Paquier (michael@paquier.xyz) wrote: > On Tue, Sep 04, 2018 at 04:14:15PM -0700, Michael Paquier wrote: > > On Tue, Sep 04, 2018 at 06:02:51PM -0400, Stephen Frost wrote: > >> * Tom Lane (tgl@sss.pgh.pa.us) wrote: > >>> I'm confused. Isn't the point of that script exactly to create a modified > >>> extension for testing pg_dump with? > > > > Not really, the regression tests run for pg_regress and the TAP test > > suite are two completely isolated things and share no dependencies. > > e54f757 has actually changed test_pg_dump.sql so as it adds regression > > tests for pg_init_privs for ALTER EXTENSION ADD/DROP, and visibly those > > have been added to test_pg_dump because they were easier to add there, > > and this has no interactions with pg_dump. What I think should have > > been done initially is to add those new tests in test_extensions > > instead. > > I am able to come back to this thread, and I still don't grep from where > sql/test_pg_dump.sql is called. Stephen, if the test suite is aiming at > tracking that pg_init_privs is correctly set up with ALTER EXTENSION > ADD/DROP, shouldn't it also query the catalog to make sure that the > correct entries are added and removed after running the so-said command? > At least that's one way I could see to perform the necessary sanity > checks without running directly pg_dump. Perhaps I am missing > something? What is perhaps not clear is that the pg_init_privs work was all entirely, specifically, for pg_dump to then work with and leverage to only export out the correct set of privileges. Testing *just* pg_init_privs without also seeing what pg_dump does with values that end up in that table wouldn't really be a complete test that things are actually working. Now, that test_pg_dump script with all of the various ALTER EXTENSION ADD things that it's doing with access methods and aggregates and such, which were quite intentionally left behind, were for testing *pg_upgrade* as well and that happens to also test pg_dump since pg_upgrade mostly just wraps pg_dump. Note that the pg_upgrade test runs installcheck which is what's installing all of those various things into the database to test pg_upgrade/pg_dump with. > >>> What I'd do is leave the final state as-is and add a "drop role if exists" > >>> at the start, similar to what some of the core regression tests do. > >> > >> I've not followed this thread but based on Tom's response, I agree with > >> his suggestion of what to do here. > > > > Not as far as I can see.. Please note that using installcheck on the > > main regression test suite does not leave around any extra roles. I can > > understand the role of having a DROP ROLE IF EXISTS though: if you get a > > crash while testing, then the beginning of the tests are repeatable, so > > independently of the root issue Tom's suggestion makes sense to me. > > Attached is a patch with more comments about the intents of the test > suite, and the separate issue pointed out by Tom fixed. It seems to me > that actually checking the contents of pg_init_privs would improve the > reason why the test exists.. I would welcome input about this last > point. Now, all that said, the TAP tests for test_pg_dump also create the extension and then perform further GRANTs and such there and then check the results of doing a pg_dump. Hopefully the above helps explain where these are used and why we do *not* want to have all of those DROP commands at the end to 'clean up'- we'd be removing a bunch of important testing that we're getting thanks to pg_upgrade. In short, let's please not apply this patch. To make it repeatable, we could have DROP EXTENSION (et al) IF EXISTS at the top of the script, similar to the DROP ROLE IF EXISTS that was suggested. I'm not sure if we have all the necessary DROP .. IF EXISTS options though today.. :/ Thanks! Stephen