Re: pgsql: Add TAP tests for pg_verify_checksums - Mailing list pgsql-hackers

From Stephen Frost
Subject Re: pgsql: Add TAP tests for pg_verify_checksums
Date
Msg-id CAOuzzgr=3Y1NKfBdGUnqt7V5cKNeB5HvquZBWXawRe88CdZDLQ@mail.gmail.com
Whole thread Raw
In response to Re: pgsql: Add TAP tests for pg_verify_checksums  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
Greetings,

On Sat, Oct 20, 2018 at 01:16 Andres Freund <andres@anarazel.de> wrote:
Hi,

On 2018-10-20 00:25:19 -0400, Stephen Frost wrote:
> If we are going to decide that we only deal with files in our directories
> matching these whitelisted patterns, then shouldn’t we apply similar logic
> in things like DROP DATABASE and any other cases where we perform actions
> in a recursive manner across our database and table space directories?

I'm honestly not sure if you're just trolling at this point.  Why would
anybody reasonable be concerned about DROP DATABASE dropping the
directory for the database? You're not honestly suggesting that anybody
would write an extension or anything like that that stores data in the
wrong database's directory, right?  Other iterations like fsyncing
files, copying the entire template database directory, etc are similarly
harmless or positive.

No, I’m not trolling, what I was trying to do is make the point that this is moving us away from having a very clear idea of what’s PG’s responsibility and what we feel comfortable operating on and this new half-and-half stance where we’ll happily nuke files in a directory that we didn’t create, and back them up even if we have no idea if they’ll be consistent at all or not when restored, but we won’t try to check the checksums on them or do some other set of operations on them.

I suspect it’s pretty clear already, but just to make it plain, I really don’t like the half-and-half approach and it seems we’re being backed into this because it happens to fit some specific cases and not because there was any real thought or design put into supporting these use cases or being able to extend PG in this way.  I do also see real risks with a whitelisting kind of approach that we end up missing things, and I get that you don’t see that risk but just stating that doesn’t change my opinion on it.  Based on what you said up thread this whole thing also seems like it may be just going away anyway- because there’s real design being done to do this properly and allow PG to be extended in a way that we will know about what files are associated with what extensions or storage mechanisms and that seems like just another reason why we shouldn’t be moving to explicitly support random files being dropped into PG directories.

Thanks,

Stephen

pgsql-hackers by date:

Previous
From: David Rowley
Date:
Subject: Re: Multi-insert into a partitioned table with before insert rowtrigger causes server crash on latest HEAD
Next
From: Stephen Frost
Date:
Subject: Re: pgsql: Add TAP tests for pg_verify_checksums