Thread: 2019-03 CF Summary / Review - Tranche #1
Hi, As last year [1], I'll try to summarize all commitfest items in 2019-03 to see which I think could realistically be put into 12. Going through all non bugfix CF entries. Here's the summary for the entries I could stomach today: RFC: ready for committer NR: needs review WOA: waiting on author. - pgbench - another attempt at tap test for time-related options NR. This was already around last year. I think it'd be fair to argue that there's not been a ton of push to get this committed. - Psql patch to show access methods info This adds \ commands to display many properties of [index ]access methods. NR. This patch has gotten a fair bit of committer feedback via Alvaro. I personally am not particularly convinced this is functionally that many people are going to use. - Show size of partitioned table NR. There seems to have been plenty discussion over details. Feels like this ought to be committable for v12? - pgbench - add pseudo-random permutation function WOA. I'm not clear as to why we'd want to add this to pgbench. To revive a discussion from last year's thread, I feel like we're adding more code to pgbench than we can actually usefully use. - libpq host/hostaddr consistency NR. I think the patches in this needs a few more committer eyes. It's possible we just should fix the documentation, or go further and change the behaviour. Feels like without more senior attention, this'll not be resolved. - pg_dump multi VALUES INSERT NR. There seems to be some tentative agreement, excepting Tom, that we probably want this feature. There has been plenty review / improvements. Seems like it ought to be possible to get this into v12. - libpq trace log NR. There seems to be considerable debate about what exactly this feature should do, and the code hasn't yet seen a lot of review. I think we ought to just target v13 here, there seems to be some agreement that there's a problem, just not exactly what the solution is. Andres: punt to v13. - pg_dumpall --exclude-database option RFC, and author is committer. - Add sqlstate output mode to VERBOSITY RFC, there seems to be agreement. - DECLARE STATEMENT syntax support in ECPG NR. There seems to be some tentative agreement that this is desirable. But the patch was only recently (2018-12-16) revived, and hasn't yet gotten a ton of review. I pinged Michael Meskes, to make him aware of this work. Andres: punt to v13. - A new data type 'bytea' for ECPG host variable NR: This seems to be much closer to being ready than the above. Michael has done a few review cycles. Not sure if it'll be done by 12, but it seems to have a good chance. - \describe: verbose commands in psql NR: This seems like a relatively large change, and only submitted 2019-01-24. Per our policy to not add nontrivial work to the last CF, I think we should not consider this a candidate for v12. Andres: punt to v13. - documenting signal handling with readme WOA: I'm very unclear as to why this is something worth documenting in this manner. Right now I'm not clear what the audience of this is supposed to be. - First SVG graphic NR: My reading of the discussion makes this look like we'll probably have graphics in v12's docs. Neat. - Update INSTALL file WOA: It seems we've not really come to much of a conclusion what this ought to contain. - Make installcheck-world in a clean environment NR: I don't feel we really have agreement on what we want here. Thus I'm doubtful this is likely to be resolvable in short order. Andres: lightly in favor of punting to v13 - Avoid creation of the free space map for small tables NR: the majority of this patch has been committed, I assume the remaining pieces will too. - Adding a TAP test checking data consistency on standby with minRecoveryPoint NR: Seems like it ought to be committable, Andrew Gierth did provide some feedback. - idle-in-transaction timeout error does not give a hint NR: Seems trivial enough. - commontype and commontypearray polymorphic types NR: To me this seems like a potentially large change, with user visible impact. As it was only submitted in December, and is clearly not yet reviewed much, I think we ought to punt on this for v12. Andres: punt to v13 - EXPLAIN with information about modified GUC values NR: The patch seems to be getting closer to completion, but I'm not sure how many actually agree that we want this. Andres: aim for v12, but we probably should discuss soon whether we actually want this. - Include all columns in default names for foreign key constraints. WOA: This patch has been submitted to the last CF first. I think it's straddling the line after which we should just refuse that pretty closely. Not sure. - Shared-memory based stats collector WOA: I think this patch has quite some promise and solve quite a few issues, and it has been worked on for a while. But at the same time the code isn't that close to being committable. Andres: unless there's a new version cleaning up review comments PDQ, I think we're unfortunately have to punt this to v13. - timeout parameters in libpq NR: This doesn't yet seem terribly well reviewed and designed. Andres: +0.5 for punting to v13 - Log a sample of transactions WOA: Seems like a useful enough feature to me, but there were a few other senior community members that didn't quite agree. Issues in the patch seem resolvable in time for v12. I'm wondering if this doesn't need a more radical approach to avoid the overhead. - monitoring CREATE INDEX [CONCURRENTLY] NR: I think there's agreement on the desirability of the feature. But while the patch has been submitted to CF 2019-01 that seems to have been somewhat of a placeholder entry. OTOH, it's a committer's project, so we can give more leeway there. - pg_stat_statements should notice FOR UPDATE clauses NR: This seems like to get in, given how sipmle it is. Some quibbles about the appropriate approach aside. Ok, my flight's about to land. So that's it for this round. Greetings, Andres Freund [1] https://postgr.es/m/20180301110344.kyp3wejoxp2ipler%40alap3.anarazel.de
On Thu, Feb 14, 2019 at 12:37:52PM -0800, Andres Freund wrote: > - Adding a TAP test checking data consistency on standby with > minRecoveryPoint > > NR: Seems like it ought to be committable, Andrew Gierth did provide > some feedback. I am rather happy of the shape of the test, and was just waiting from Andrew for a last confirmation. If there are no objections, I could commit it as well. -- Michael
Attachment
On Fri, Feb 15, 2019 at 2:08 AM Andres Freund <andres@anarazel.de> wrote: > > Hi, > > - Avoid creation of the free space map for small tables > > NR: the majority of this patch has been committed, > This is a correct assessment. > I assume the > remaining pieces will too. > Yes, I am busy these days with something else, but I will surely get back to reviewing/committing the remaining stuff for PG12. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
Hi, On 2019-02-14 12:37:52 -0800, Andres Freund wrote: > - pg_stat_statements should notice FOR UPDATE clauses > > NR: This seems like to get in, given how sipmle it is. Some quibbles > about the appropriate approach aside. > > > Ok, my flight's about to land. So that's it for this round. - Protect syscache from bloating with negative cache entries WOA: I think unless the feature is drastically scaled down in scope, there's no chance to get anything into v12. Andres: punt to v13, unless a smaller chunk can be split off - SERIALIZABLE with parallel query NR: This seems like it's pretty much committable, and the author is a committer these days. - Removing [Merge]Append nodes which contain a single subpath NR: I can't quite tell the state of this patch just by reading the thread. It's longstanding, and the code doesn't look terrible, but Tom appears to still be a bit unhappy. Andres: ??? - verify ALTER TABLE SET NOT NULL by valid constraints NR: Seems like a pretty useful feature. The code isn't very invasive. The patch has been lingering around for a while. We should merge this. - Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist NR: This unfortunately has barely gotten any review so far, and has had a number of issues authors have discovered themselves. It's a bit sad that a useful patch has gotten this little review, but I think it's probably a stretch to get it into v12 unless some senior reviewers show up. - GiST VACUUM NR: Has gotten a fair bit of review by Heikki, but there still seems to be a number of unresolved issues. Not sure if there's cycles to get this done unless Heikki has time. - libpq compression NR: While a very useful feature, the patch seems pretty raw, there's disagreements about code structure, and some cryptographic risks need to be discussed or at least better documented. Andres: punt to v13 - Evaluate immutable functions during planning (in FROM clause) NR: As far as I can tell this CF entry should have been either WO or even rejected for the last two CFs. Even if the review feedback had been addressed, it seems there's very substantial architecture concerns that haven't been responded to. Andres: no chance for v12, CF entry should probably be closed. - Global shared meta cache NR: This is extremely invasive, and is more PoC work than anything else. Andres: punt to v13 - Remove self join on a unique column NR: This still seems pretty raw, and there's not been a ton of detailed review (although plenty of more general discussion). I don't quite see how we can get this into shape for v12, although some review would be good to allow the feature to progress. Andres: punt to v13 - Inline Common Table Expressions NR: I think it's likely that Tom will commit this soon, we're mostly debating syntax and similar things at this point (and man, I'm looking forward to this). - Autoprepare: implicitly replace literals with parameters and store generalized plan NR: I think there's no chance to get this into v12, given the state of the patch. There's not even agreement that we want this feature (although I think we can't avoid it for much longer), not to speak of agreement on the architecture. I think this needs a lot more attention to ever get anywhere. Andres: punt to v13 - Tid scan improvements (ordering and range scan) NR: The patch has been through recent significant architectural changes, albeit to an architecture more similar to an earlier approach. There's not been meaningful review since. On the other hand, the patch isn't actually all that complex. Seems like a stretch to get into v12, but possible if e.g. Tom wants to pick it up. Andres: +0.5 for punting to v13 - Block level parallel vacuum NR: Cool feature, but I don't think this has gotten even remotely enough review to be mergable into v12. Andres: punt to v13 - Speed up planning with partitions NR: Important feature, but based on a skim of the discussion this doesn't seem ready for v12. Andres: punt to v13 - Make nbtree keys unique by appending heap TID, suffix truncation NR: Important, seemingly carefully worked on feature. But it's *complicated* stuff, and there's only been a bit of design review by Heikki. The author's a committer. I don't feel qualified to judge this. - KNN for B-tree NR: The patch still seems to lack a bit of review. Possible but a stretch for v12. (While the thread is quite old, there've been substantial gaps where it wasn't worked on, so I don't think it's one of the really bad cases of not getting review.) Andres: +0.5 for punting to v13 - New vacuum option to do only freezing NR: Seems simple enough. We probably can merge this. - Speed up transaction completion faster after many relations are accessed in a transaction NR: This has only been submitted 2019-02-12. While not a really complicated patch, it's also not trivial. Therefore I'd say this falls under our policy of not merging nontrivial patches first submitted to the last CF. Andres: punt to v13 - SortSupport implementation on inet/cdir WOA: This is a substantial amount of code submitted first for the last CF. Andres: punt to v13 - Referential Integrity Checks with Statement Level Triggers NR: This has only been submitted to this CF, and is a very substantial change. There has been no review as of yet, and the author acknowledges several shortcomings in the patch. Andres: punt to v13 - postgres_fdw: Perform UPPERREL_ORDERED and UPPERREL_FINAL steps remotely WOA: This is a nontrivial change, and the design and review only started in late December. It's probably not realistic to target v12. Andres: punt to v13 - Delay locking partitions during query execution NR: Important patch. Development has only started in December. Not a ton of code though. Andres: ??? - Delay locking partitions during INSERT and UPDATE RFC: Looks reasonable enough, although there's some discussion related to increased deadlock risk. Re-raised that on the thread. - Prove IS NOT NULL inference for large arrays NR: No idea. - Detoast Compressed Datum Slice NR: Probably just can get committed close to as-is. Pinged Stephen, who mentioned he's interested in committing it. - Ordered Partitioned Table Scans NR: Patch has been through a few rounds, but probably needs a look soon by somebody else with a fair bit of planner experience. Might be doable. - schema variables, LET command NR: For a feature that's as user exposed as this, I don't think this has gotten even remotely enough review. Andres: punt to v13 - block level PRAGMA NR: My reading of this thread is that the proposal is closer to being rejected than merged. Andres: reject or punt? - get rid of StdRdOptions, use individual binary reloptions representation for each relation kind instead NR: I'm not sure I understand what this going to buy us. Andres: ??? - Track the next xid using 64 bits WOA: Can probably be merged, I posted a few relatively minor review comments, but I assume Thomas is going to merge an updated version. - Refactoring the checkpointer's fsync request queue NR: There's been some design issues raised (by your's truly, in person). And there's very likely not going to be an in-core user for this in v12 (neither slru-via-bufmgr or undo seem likely to get merged). So I'm not sure it's realistic to get this into v12, although it certainly seems doable if a bit of elbow grease is put into it. - Making WAL receiver startup rely on GUC context for primary_conninfo and primary_slot_name NR: I think this should be rejected. As noted in the thread, this doesn't actually buy us much, and has some potential costs once we make primary_conninfo PGC_SIGHUP. Andres: Reject - Respect client-initiated CopyDone during logical streaming replication NR: I don't think this is ready. Andres: punt to v13 - logical decoding of two-phase transactions WOA: This is probably not going to be ready by v13, although I think it could become so if somebody senior really started working on it. Andres: punt to v13 :( - logical streaming for large in-progress transactions WOA: Tomas, at the dev meeting in Brussels, said he doesn't believe this is going to be ready for v12. Andres: punt to v13 :( - Restricting maximum keep segments by repslots WOA: This seems to need substantial polishing before being ready. Might be doable before v13, but the author also is involved in numerous other patches needing work... - Synchronous replay mode for avoiding stale reads on hot standbys NR: There's some disagreement about the desirability of the feature, but plenty people signalled they want it. Seems like it ought to get merged at some point, there's been review (but more probably wouldn't hurt). - Copy function for logical replication slots WOA: Probably can be committed, was briefly marked RFC, but I found some issues (which should be easy enough to rectify). - pg_rewind: options to use restore_command from recovery.conf or command line WOA: Was previously marked as RFC, but I don't see how it is. Possibly can be finished, but does require a good bit more work. - create and use subscription for nonsuperuser NR: This seems to need a good bit more work. I'm a bit doubtful this can be finished for v12. Andres: +0.25 for punting to v13 - online change primary_conninfo WOA: Needs a bit more work, but probably can be finished for v12. - Remove deprecated exclusive backup mode NR: There clearly seems to be no concensus on making this change. Andres: punt to v13 or something - Add timeline to partial WAL segments WOA: Seems to need a good bit more work, and touches sensitive bits. Andres: +0.5 for punting to v13 - Synchronizing slots from primary to standby NR: This clearly is just a POC at this point. Andres: punt to v13 - pg_hba.conf : new auth option : clientcert=verify-full NR: this should probably be RFC, as it was before needing to be rebased. Looks like it should just get merged. - GSSAPI encryption support NR: Seems Stephen is pondering committing this. Not quite sure I like the way it's integreated in fe-secure/be-secure. - multivariate MCV lists and histograms WOA: Seems the MCV bits might be realistic for v12, but the histogram part not? - Push aggregation down to base relations and joins NR: Seems like this needs a few more review, and is probably not quite going to be ready for v12. But it'd probably need some attention by Tom for the author to be able to move forward. Andres: push to v13 - Pluggable storage API NR: I'm biased... I plan to push substantial portions of this feature. There's a few later features that I'm not sure are going to be ready (e.g. doing trigger lookups using a snapshot). - Custom compression methods WOA: Hm. Andres: I think we need to make a call whether we actually want this, rather than just continuing to punt this forward. - BRIN bloom and multi-minmax indexes NR: Unfortunately this doesn't seem to have gotten meaningful review in the last year :( - SQL/JSON: jsonpath WOA: This seems to need substantial further work before being committable Andres: +0.5 for punting to v13 (okay, breaking open a bottle of wine here) - Add enum relation option type RFC: I think Alvaro probably can commit this? He's edited a few versions of the patch, and set the target version to 12. - Covering GiST indexes RFC - amcheck verification for GiST WOA: Some locking changes are apparently needed, possibly a bit too much to fix up for v12? - DNS SRV support for LDAP authentication NR: Looks like Thomas should just merge this - Add Hook Functions for Disk Quota Extension NR: This is in the early design stages, rather than realistically targeting v12. Andres: punt to v13 - Implement NULL-related checks in object address functions to prevent cache lookup errors, take two NR: Seems like this should go into 12, although it'd be good if Alvaro could take another peek before Michael pushes. - Triggers on Materialized Views NR: I think we need to provide useful feedback whether we actually want this feature. But either way, it's not v12 material. Andres: punt to v13, discuss whether to reject - Ltree syntax improvement NR: Given this is a nontrivial patch, and was submitted 2019-01-29, it's clearly not v12 material. Andres: punt to v13 - Skip table truncation at VACUUM (should be: allow to disable truncations via a reloptions) NR: The feature is near trivial, and avoids significant problems in hot standby environments. It seems to need some language lawyering. - SQL/JSON: functions NR: Dependant on jsonpath which I have a hard time seeing in v12. And it's barely reviewed (and still contains exciting PG_CATCH games that I warned about at the tail end of v11...). Andres: punt to v13 - SQL/JSON: JSON_TABLE NR: Depends on previous, no review. Andres: punt to v13 - chained transactions NR: Looks like it ought to be committable - conflict handling for COPY FROM NR: Clearly not ready for v12, the whole business with requiring a log file doesn't seem acceptable. Andres: punt to v13 - FETCH FIRST clause PERCENT option WOA: Clearly not ready for v12. Andres: punt to v13 - ALTER TABLE on system catalogs NR: I still think this is the wrong approach, but I can also live with Peter hacking this up. But I think a call got to be made at some point, rather than schlepping this around continuously. - ATTACH/DETACH PARTITION CONCURRENTLY NR: Seems to be getting closer to being mergable. - FETCH FIRST clause WITH TIES option NR: Doesn't quite seem ready, insufficient tests, new code probably should be in separate function. But possibly could be fixed up for v12? - support VARIADIC arg for least/greatest functions NR: There's debate whether we want this feature. Tom argues, and I'm inclined to agree, that this should rather be separate array specific functions. Pavel's position is that that's a separate thing, but I'm not sure I agree. Andres: reject? - Temporary materialized views NR: Some infrastructure work is needed before this can go in. Not sure if that can be finished for v12? - insensitive/non-deterministic collations NR: Peter has stated that he's targeting v12, but I'm not sure this had enough review? But it's not *that* complicated... - Log10 and hyperbolic functions for SQL:2016 compliance NR: Seems simple enough, we should just merge this. - pg_upgrade: Pass -j option down to vacuumdb NR: Seems simple enough, we should just merge this. - Support huge_pages on AIX NR: Probably can be merged. There's a few patches that authors, rather than others, have market as targeting v13, or where authors consented to that. I don't see a need to go through those here. - Generic type subscripting - Transactions involving multiple postgres foreign servers - Undo logs - Undo worker and transaction rollback - Index Skip Scan - SERIALIZABLE on standby servers - Advanced partition matching for partition-wise join Comments? Greetings, Andres Freund
Hi
- block level PRAGMA
NR: My reading of this thread is that the proposal is closer to being
rejected than merged.
Andres: reject or punt?
This patch is very simple and has strong sense for users of plpgsql_checks. For first moment, only plpgsql_check's users can get some advance from it. But if we implement autonomous transactions, and I hope so this feature will be implemented, then this code can be used for Oracle's PL/SQL syntax compatible implementation. There is not any disadvantage - it is clean, and compatible with ADA, PL/SQL .. I implemented just only block level PRAGMA, and there was not any disagreement.
Regards
Pavel
On Fri, Feb 15, 2019 at 9:45 PM Andres Freund <andres@anarazel.de> wrote: > - Make nbtree keys unique by appending heap TID, suffix truncation > > NR: Important, seemingly carefully worked on feature. But it's > *complicated* stuff, and there's only been a bit of design review by > Heikki. The author's a committer. I don't feel qualified to judge > this. I think that's fair. The best way of understanding the risks as a non-expert is to think of the patch as having two distinct components: 1. Make nbtree entries unique + add suffix truncation -- the basic mechanism. 2. Make more intelligent decisions about where to split pages, to work with suffix truncation, while still mostly caring about the balance of free space among each half of the split, and caring about a number of other new concerns besides these two. You're right that some parts of the patch are very complicated, but those are all contained in the second component. That has been the main focus of Heikki's review, by far. This second component is concerned with picking a split point that is already known to be legal based on the *existing* criteria. If there were bugs here, they could not result in data corruption. The worst outcome I can conceive of is that an index would be bloated in a new and novel way. It would be possible to correct that in a point release without breaking on-disk compatibility. That would be painful, certainly, but it's far from the worst outcome. Granted, there are also one or two subtle things in the first, more critical component, but these are also the things that were established earliest and have received the most testing. And, amcheck is now capable of doing point lookups using the same code paths as index scans (calling _bt_search()) to relocate each and every tuple on the leaf level, starting from the root. The first component does not change anything about how crash recovery or VACUUM works, either. It's all about how keys compare, and how new pivot tuples are generated -- it's mostly about the key space, while changing very little about the physical on-disk representation. (It builds on the on-disk representation changes added in Postgres 11, for INCLUDE indexes.) > - SortSupport implementation on inet/cdir > > WOA: This is a substantial amount of code submitted first for the last > CF. > > Andres: punt to v13 I was kind of involved here. I think that it's fair to punt, based on the rule about submitting a big patch to the last CF. > - amcheck verification for GiST > > WOA: Some locking changes are apparently needed, possibly a bit too > much to fix up for v12? I had hoped that Andrey Borodin would get back to me on this soon. It does still seem unsettled. > - insensitive/non-deterministic collations > > NR: Peter has stated that he's targeting v12, but I'm not sure this > had enough review? But it's not *that* complicated... I could help out here. -- Peter Geoghegan
Andres Freund <andres@anarazel.de> writes:
> - Inline Common Table Expressions
> NR: I think it's likely that Tom will commit this soon, we're mostly
> debating syntax and similar things at this point (and man, I'm looking
> forward to this).
Unless there are a bunch of votes real soon against the [NOT] MATERIALIZED
syntax, I'm going to commit it that way. "Real soon" means "probably
tomorrow".
regards, tom lane
On Fri, Feb 15, 2019 at 09:45:26PM -0800, Andres Freund wrote: > - Making WAL receiver startup rely on GUC context for primary_conninfo > and primary_slot_name > > NR: I think this should be rejected. As noted in the thread, this > doesn't actually buy us much, and has some potential costs once we > make primary_conninfo PGC_SIGHUP. > > Andres: Reject I am not surprised by your opinion here, you stated it clearly on the thread :) I have just marked it as rejected, as I don't have the energy to fight for it. > - online change primary_conninfo > > WOA: Needs a bit more work, but probably can be finished for v12. Yep, agreed. > - Add timeline to partial WAL segments > > WOA: Seems to need a good bit more work, and touches sensitive bits. > > Andres: +0.5 for punting to v13 The problem is not that easy, particularly to make things consistent between the backend and pg_receivewal. > - Implement NULL-related checks in object address functions to prevent cache lookup errors, take two > > NR: Seems like this should go into 12, although it'd be good if Alvaro > could take another peek before Michael pushes. I would prefer if Alvaro has an extra look at what I am proposing as most of the stuff in objectaddress.c is originally his. > - Temporary materialized views > > NR: Some infrastructure work is needed before this can go in. Not sure > if that can be finished for v12? I would suggest to move that to v13. Relation creation done by CTAS needs to be refactored first, and my take is that we could have this refactoring part for v12, moving the core portion of the proposal to the next dev cycle. -- Michael
Attachment
On Sat, Feb 16, 2019 at 8:45 AM Andres Freund <andres@anarazel.de> wrote: > - SQL/JSON: jsonpath > > WOA: This seems to need substantial further work before being > committable > > Andres: +0.5 for punting to v13 I'm going to post updated patchset next week. All the issues highlighted will be resolved there. So, let's don't decide this too early. ------ Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
Hi, On February 16, 2019 11:22:32 AM PST, Alexander Korotkov <a.korotkov@postgrespro.ru> wrote: >On Sat, Feb 16, 2019 at 8:45 AM Andres Freund <andres@anarazel.de> >wrote: >> - SQL/JSON: jsonpath >> >> WOA: This seems to need substantial further work before being >> committable >> >> Andres: +0.5 for punting to v13 > >I'm going to post updated patchset next week. All the issues >highlighted will be resolved there. So, let's don't decide this too >early. Well, given that the patches still have a lot of the same issues complained about a year ago, where people said we shouldtry to get them into v11, I'm not sure that that's a realistic goal. Jsonb was a success, but also held up the releaseby several months. Andres -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
On Sat, Feb 16, 2019 at 11:31 PM Andres Freund <andres@anarazel.de> wrote: > On February 16, 2019 11:22:32 AM PST, Alexander Korotkov <a.korotkov@postgrespro.ru> wrote: > >On Sat, Feb 16, 2019 at 8:45 AM Andres Freund <andres@anarazel.de> > >wrote: > >> - SQL/JSON: jsonpath > >> > >> WOA: This seems to need substantial further work before being > >> committable > >> > >> Andres: +0.5 for punting to v13 > > > >I'm going to post updated patchset next week. All the issues > >highlighted will be resolved there. So, let's don't decide this too > >early. > > Well, given that the patches still have a lot of the same issues complained about a year ago, where people said we shouldtry to get them into v11, I'm not sure that that's a realistic goal. I'm sorry, a year ago I didn't understand this issue correctly. Otherwise, I would push people to do something more productive during this year. If solution I'm going to post next week wouldn't be good enough, there is a backup plan. We can wipe out error suppression completely. Then we implement less part of standard, but still can get something very useful into core. > Jsonb was a success, but also held up the release by several months. I don't ask to commit patchset in current shape and help up the release because of it. I just say there is still time before FF. So, let's not doom patchset too early. ------ Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
On Sun, Feb 17, 2019 at 1:09 AM Alexander Korotkov <a.korotkov@postgrespro.ru> wrote: > > On Sat, Feb 16, 2019 at 11:31 PM Andres Freund <andres@anarazel.de> wrote: > > On February 16, 2019 11:22:32 AM PST, Alexander Korotkov <a.korotkov@postgrespro.ru> wrote: > > >On Sat, Feb 16, 2019 at 8:45 AM Andres Freund <andres@anarazel.de> > > >wrote: > > >> - SQL/JSON: jsonpath > > >> > > >> WOA: This seems to need substantial further work before being > > >> committable > > >> > > >> Andres: +0.5 for punting to v13 > > > > > >I'm going to post updated patchset next week. All the issues > > >highlighted will be resolved there. So, let's don't decide this too > > >early. > > > > Well, given that the patches still have a lot of the same issues complained about a year ago, where people said we shouldtry to get them into v11, I'm not sure that that's a realistic goal. > > I'm sorry, a year ago I didn't understand this issue correctly. > Otherwise, I would push people to do something more productive during > this year. > > If solution I'm going to post next week wouldn't be good enough, there > is a backup plan. We can wipe out error suppression completely. Then > we implement less part of standard, but still can get something very > useful into core. To be more clear. In both options I'm NOT proposing to commit any PG_TRY/PG_CATCH anymore :) ------ Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
> - Prove IS NOT NULL inference for large arrays > > NR: No idea. As the fairly new author of this patch, my perspective is that this patch got quite a bit of review, albeit without a formal "yes or no" response. I'm obviously interested in getting it committed, and I believe it's a fairly simple patch to look at (though since it's in predtest probably requires extra brain cycles to be careful we don't make spurious assumptions in the optimizer). It's also an obvious performance win for queries that can use partial indexes with almost no additional optimizer overhead. But I'm also interested in feedback on how patches like this work in the review process -- particularly when given the fair amount of discussion/cleanup but without a final word. As a new patch author a lot of understanding how this works feels very much like a "learn as you go", but since there's not a lot of information directly written on it I figured asking explicitly is the best way to learn the process better. Thanks, James Coleman
James Coleman <jtc331@gmail.com> writes:
>> - Prove IS NOT NULL inference for large arrays
>>
>> NR: No idea.
> As the fairly new author of this patch, my perspective is that this
> patch got quite a bit of review, albeit without a formal "yes or no"
> response.
FWIW, I ran out of time for that patch during the January 'fest, but
I think it still has a good shot at getting committed during March.
regards, tom lane
From: Andres Freund [mailto:andres@anarazel.de] > - Protect syscache from bloating with negative cache entries > > WOA: I think unless the feature is drastically scaled down in scope, > there's no chance to get anything into v12. > > Andres: punt to v13, unless a smaller chunk can be split off At a glance, the patch set looks big divided in 5 patch files, but the actual size may get much smaller by merging thosefiles and excluding 004 (statistics views). I'm reviewing this. I wish this could not be given up yet... > - Speed up planning with partitions > > NR: Important feature, but based on a skim of the discussion this > doesn't seem ready for v12. > > Andres: punt to v13 > - Speed up transaction completion faster after many relations are > accessed in a transaction > > NR: This has only been submitted 2019-02-12. While not a really > complicated patch, it's also not trivial. Therefore I'd say this falls > under our policy of not merging nontrivial patches first submitted to > the last CF. > > Andres: punt to v13 I hope these will be continued in PG 12, because these will make PostgreSQL comparable to a commercial database in OLTP workloads. I'd like to hear from Amit and David on the feasibility. Regards Takayuki Tsunakawa
Hi, On 2019/02/18 11:13, Tsunakawa, Takayuki wrote: > From: Andres Freund [mailto:andres@anarazel.de] >> - Speed up planning with partitions >> >> NR: Important feature, but based on a skim of the discussion this >> doesn't seem ready for v12. >> >> Andres: punt to v13 > >> - Speed up transaction completion faster after many relations are >> accessed in a transaction >> >> NR: This has only been submitted 2019-02-12. While not a really >> complicated patch, it's also not trivial. Therefore I'd say this falls >> under our policy of not merging nontrivial patches first submitted to >> the last CF. >> >> Andres: punt to v13 > > I hope these will be continued in PG 12, because these will make PostgreSQL comparable to a commercial database in OLTPworkloads. I'd like to hear from Amit and David on the feasibility. As far as the first one is concerned (speed up planning with partitions), although there's no new functionality [1], there is quite a bit of code churn affecting somewhat complex logic of inheritance planning. As I mentioned in the FOSDEM developer meeting's patch triage discussion, there's a chance of moving this forward if Tom has time to look at some portions of these patches. David's and Imai-san's review over the past few months has been very helpful to get the patches to the state they are in now. As for the 2nd one, while I can say it's really helpful for workloads with many partitions, I have to admit I can't give an expert opinion on the code changes. Thank you for working on it. Thanks, Amit [1] it does enable hash partition pruning for update/delete queries though, which is a new feature
(2019/02/16 14:45), Andres Freund wrote: > - postgres_fdw: Perform UPPERREL_ORDERED and UPPERREL_FINAL steps > remotely > > WOA: This is a nontrivial change, and the design and review only > started in late December. It's probably not realistic to target v12. > > Andres: punt to v13 I also think this needs more reviews, but I don't think it's unrealistic to target v12, because 1) the patch is actually not that large (at least in the latest version, most of the changes are in regression tests), and 2) IMO the patch is rather straightforward. Best regards, Etsuro Fujita
On Mon, 18 Feb 2019 at 15:50, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote: > On 2019/02/18 11:13, Tsunakawa, Takayuki wrote: > > From: Andres Freund [mailto:andres@anarazel.de] > >> - Speed up planning with partitions > >> > >> NR: Important feature, but based on a skim of the discussion this > >> doesn't seem ready for v12. > >> > >> Andres: punt to v13 > > > >> - Speed up transaction completion faster after many relations are > >> accessed in a transaction > >> > >> NR: This has only been submitted 2019-02-12. While not a really > >> complicated patch, it's also not trivial. Therefore I'd say this falls > >> under our policy of not merging nontrivial patches first submitted to > >> the last CF. > >> > >> Andres: punt to v13 > > > > I hope these will be continued in PG 12, because these will make PostgreSQL comparable to a commercial database in OLTPworkloads. I'd like to hear from Amit and David on the feasibility. > > As far as the first one is concerned (speed up planning with partitions), > although there's no new functionality [1], there is quite a bit of code > churn affecting somewhat complex logic of inheritance planning. I think we need to treat each patch in that series individually. Last time I looked, the first 1 or 2 patches looked very close. I know there's some tricky stuff in later patches in the series, but I don't think that should stop earlier patches being committed for PG12. However, I'd say that if the entire thing is to make PG12 then we'll need to start ticking off earlier patches pretty soon, likely before March. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On 2/16/19 7:45 AM, Andres Freund wrote: > - Add timeline to partial WAL segments > > WOA: Seems to need a good bit more work, and touches sensitive bits. > > Andres: +0.5 for punting to v13 I have labelled this patch v13 and I'll push it as soon as the CF app allows me to do so. I think this is important but there is a lot of tricky work to be done in pg_receivewal and it's best not to rush that. Michael and I have an agreement in principle but ... Regards, -- -David david@pgmasters.net
Hi, On 2019-02-18 18:59:22 +0200, David Steele wrote: > On 2/16/19 7:45 AM, Andres Freund wrote: > > > - Add timeline to partial WAL segments > > > > WOA: Seems to need a good bit more work, and touches sensitive bits. > > > > Andres: +0.5 for punting to v13 > I have labelled this patch v13 and I'll push it as soon as the CF app allows > me to do so. > > I think this is important but there is a lot of tricky work to be done in > pg_receivewal and it's best not to rush that. Michael and I have an > agreement in principle but ... FWIW, given that we now can filter by targetting v12 (although it'd be nice if it were possible to filter by v12 or NULL), I don't think there's any urgency to push to the next CF immediately. Greetings, Andres Freund
On 2/18/19 7:03 PM, Andres Freund wrote: > Hi, > > On 2019-02-18 18:59:22 +0200, David Steele wrote: >> On 2/16/19 7:45 AM, Andres Freund wrote: >> >>> - Add timeline to partial WAL segments >>> >>> WOA: Seems to need a good bit more work, and touches sensitive bits. >>> >>> Andres: +0.5 for punting to v13 >> I have labelled this patch v13 and I'll push it as soon as the CF app allows >> me to do so. >> >> I think this is important but there is a lot of tricky work to be done in >> pg_receivewal and it's best not to rush that. Michael and I have an >> agreement in principle but ... > > FWIW, given that we now can filter by targetting v12 (although it'd be > nice if it were possible to filter by v12 or NULL), I don't think > there's any urgency to push to the next CF immediately. Agreed. This new feature helps a lot. Even so, I'll push it when I can to get it out of my hair, as it were. I'll be spending a lot of time look at that list next month. -- -David david@pgmasters.net
David Steele <david@pgmasters.net> writes:
> Even so, I'll push it when I can to get it out of my hair, as it were.
> I'll be spending a lot of time look at that list next month.
Can't you do it now? The status summary line already shows one
patch as having been pushed to the next CF.
regards, tom lane
On 2019-02-18 12:37:27 -0500, Tom Lane wrote: > David Steele <david@pgmasters.net> writes: > > Even so, I'll push it when I can to get it out of my hair, as it were. > > I'll be spending a lot of time look at that list next month. > > Can't you do it now? The status summary line already shows one > patch as having been pushed to the next CF. It's CF app nannyism. One can't move a patch to the next CF that's waiting-on-author. I've complained about that a number of times, but...
Andres Freund <andres@anarazel.de> writes:
> On 2019-02-18 12:37:27 -0500, Tom Lane wrote:
>> Can't you do it now? The status summary line already shows one
>> patch as having been pushed to the next CF.
> It's CF app nannyism. One can't move a patch to the next CF that's
> waiting-on-author. I've complained about that a number of times, but...
So change it to another state, push it, change it again.
regards, tom lane
On 2/18/19 12:43 PM, Tom Lane wrote: > Andres Freund <andres@anarazel.de> writes: >> On 2019-02-18 12:37:27 -0500, Tom Lane wrote: >>> Can't you do it now? The status summary line already shows one >>> patch as having been pushed to the next CF. >> It's CF app nannyism. One can't move a patch to the next CF that's >> waiting-on-author. I've complained about that a number of times, but... > So change it to another state, push it, change it again. > > I'm with Andres. I found this annoying six months ago. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
> On 2/18/19 12:43 PM, Tom Lane wrote:
>> Andres Freund <andres@anarazel.de> writes:
>>> It's CF app nannyism. One can't move a patch to the next CF that's
>>> waiting-on-author. I've complained about that a number of times, but...
>> So change it to another state, push it, change it again.
> I'm with Andres. I found this annoying six months ago.
Oh, I agree the restriction is stupid. I'm just pointing out that
it can be gotten around.
regards, tom lane
On 2/19/19 1:23 AM, Tom Lane wrote: > Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes: >> On 2/18/19 12:43 PM, Tom Lane wrote: >>> Andres Freund <andres@anarazel.de> writes: >>>> It's CF app nannyism. One can't move a patch to the next CF that's >>>> waiting-on-author. I've complained about that a number of times, but... > >>> So change it to another state, push it, change it again. > >> I'm with Andres. I found this annoying six months ago. > > Oh, I agree the restriction is stupid. I'm just pointing out that > it can be gotten around. OK, yeah, that worked. For some reason I was having trouble moving things out of the 2019-01 CF last month but this time it worked just fine. I think more than one was marked as open then, not sure. -- -David david@pgmasters.net
(2019/02/18 12:40), Etsuro Fujita wrote: > (2019/02/16 14:45), Andres Freund wrote: >> - postgres_fdw: Perform UPPERREL_ORDERED and UPPERREL_FINAL steps >> remotely >> >> WOA: This is a nontrivial change, and the design and review only >> started in late December. It's probably not realistic to target v12. >> >> Andres: punt to v13 > > I also think this needs more reviews, but I don't think it's unrealistic > to target v12, because 1) the patch is actually not that large (at least > in the latest version, most of the changes are in regression tests), and > 2) IMO the patch is rather straightforward. There seems to be no objections, so I marked this as targeting v12. Best regards, Etsuro Fujita
On 16.02.2019 8:45, Andres Freund wrote: > - pg_rewind: options to use restore_command from recovery.conf or > command line > > WOA: Was previously marked as RFC, but I don't see how it is. Possibly > can be finished, but does require a good bit more work. Just sent new version of the patch to the thread [1], which removes all unnecessary complexity. I am willing to address all new issues during 2019-03 CF if any. [1] https://www.postgresql.org/message-id/c9cfabce-8fb6-493f-68ec-e0a72d957bf4%40postgrespro.ru Thanks -- Alexey Kondratov