Re: Patch: Global Unique Index - Mailing list pgsql-hackers
From | Cary Huang |
---|---|
Subject | Re: Patch: Global Unique Index |
Date | |
Msg-id | 39cb0eaa-47f6-1a61-e023-8ba971c2fcba@highgo.ca Whole thread Raw |
In response to | Re: Patch: Global Unique Index (Greg Stark <stark@mit.edu>) |
Responses |
Re: Patch: Global Unique Index
|
List | pgsql-hackers |
On 2022-11-30 2:30 p.m., Greg Stark wrote: > On Tue, 29 Nov 2022 at 21:16, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> I actually think that that problem should be soluble with a >> slightly different approach. The thing that feels insoluble >> is that you can't do this without acquiring sufficient locks >> to prevent addition of new partitions while the insertion is >> in progress. That will be expensive in itself, and it will >> turn ATTACH PARTITION into a performance disaster. > I think there`s a lot of room to manoeuvre here. This is a new feature > that doesn't need to be 100% complete or satisfy any existing > standard. There are lots of options for compromises that leave room > for future improvements. > > 1) We could just say sure ATTACH is slow if you're attaching an > non-empty partition > 2) We could invent a concept like convalidated and let people attach a > partition without validating the uniqueness and then validate it later > concurrently > 3) We could say ATTACH doesn't work now and come up with a better > strategy in the future > > Also, don't I vaguely recall something in exclusion constraints about > having some kind of in-memory "intent" list where you declared that > you're about to insert a value, you validate it doesn't violate the > constraint and then you're free to insert it because anyone else will > see your intent in memory? There might be a need for some kind of > global object that only holds inserted keys long enough that other > sessions are guaranteed to see the key in the correct index. And that > could maybe even be in memory rather than on disk. > > This isn't a simple project but I don't think it's impossible as long > as we keep an open mind about the requirements. In the current global unique index implementation, ATTACH can be slow if there are concurrent inserts happening. ATTACH tries to acquire shareLock on all existing partitions and partition-to-be before it scans and sorts them for uniqueness check. It will release them only after all partitions have been checked. If there are concurrent inserts, ATTACH has to wait for all inserts complete. Likewise, if ATTACH is in progress, inserts have to wait as well. This is an issue now. If we were to make ATTACH acquire a lower level lock (AccessShareLock), scans a partition, and then release it. there is nothing stopping any concurrent inserts from inserting a conflict right after it finishes checking. This is another issue. There is no transaction level lock being triggered here like in multiple concurent inserts case Another email thread called "create index concurrently on partitioned index" discuss some approaches that may be used to solve the attach issue here, basically to allow ATTACH PARTITION CONCURRENTLY... regards Cary Huang --------------------------------- HighGo Software Canada
pgsql-hackers by date: