== PostgreSQL Weekly News - November 11, 2018 == - Mailing list pgsql-announce
From | David Fetter |
---|---|
Subject | == PostgreSQL Weekly News - November 11, 2018 == |
Date | |
Msg-id | 20181112163538.GA503@fetter.org Whole thread Raw |
List | pgsql-announce |
== PostgreSQL Weekly News - November 11, 2018 == PostgreSQL security releases 11.1, 10.6, 9.6.11, 9.5.15, 9.4.20, and 9.3.25 released. This is the final release of the 9.3 series. Please read the announcement below and upgrade as soon as possible. https://www.postgresql.org/about/news/1905/ == PostgreSQL Product News == pg_back 1.7, a backup script for PostgreSQL, released. https://github.com/orgrim/pg_back Bucardo 5.5.0, a replication system with dual-master capability, released. http://bucardo.org/wiki/Bucardo PipelineDB v1.0.0 released as an extension for PostgreSQL 11 https://www.pipelinedb.com/blog/high-performance-time-series-aggregation-for-postgresql-11 == PostgreSQL Jobs for November == http://archives.postgresql.org/pgsql-jobs/2018-11/ == PostgreSQL Local == pgday.Seoul 2018 will be held in Seoul, South Korea on November 3, 2018 in Seoul. Korean language information is here: http://pgday.postgresql.kr/ PGDay Argentina 2018 will be held in Buenos Aires on November 21, 2018. http://www.pgday.com.ar/buenosaires2018/?lang=en 2Q PGConf will be on December 4-5, 2018 in Chicago, IL. http://www.2qpgconf.com/ PGConf.ASIA 2018 will take place on December 10-12, 2018 in Akihabara, Tokyo, Japan. http://www.pgconf.asia/EN/2018/ FOSDEM PGDay 2019, a one day conference held before the main FOSDEM event will be held in Brussels, Belgium, on Feb 1st, 2019. The CfP is open until November 9th, 2018 at https://2019.fosdempgday.org/callforpapers/ . https://2019.fosdempgday.org/ Prague PostgreSQL Developer Day 2019 (P2D2 2019) is a two-day conference that will be held on February 13-14, 2019 in Prague, Czech Republic. The CfP is open until January 4, 2018 at https://p2d2.cz/callforpapers http://www.p2d2.cz/ PGConf India 2019 will be on February 13-15, 2019 in Bengaluru, Karnataka. Proposals are due via https://goo.gl/forms/F9hRjOIsaNasVOAz2 by October 31st, 2017. http://pgconf.in/ pgDay Paris 2019 will be held in Paris, France on March 12, 2019 at 199bis rue Saint-Martin. The CfP is open until November 30, 2018. http://2019.pgday.paris/callforpapers/ PGConf APAC 2019 will be held in Singapore March 19-21, 2019. The CfP is open at http://2019.pgconfapac.org/cfp through November 16, 2018. http://2019.pgconfapac.org/ PGDay.IT 2019 will take place May 16th and May 17th in Bologna, Italy. The CfP is open at https://2019.pgday.it/en/blog/cfp and the Call for Workshops is at https://2019.pgday.it/en/blog/cfw until January 15, 2019. https://2019.pgday.it/en/ == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm PST8PDT to david@fetter.org. == Applied Patches == Michaël Paquier pushed: - Ignore partitioned tables when processing ON COMMIT DELETE ROWS. Those tables have no physical storage, making this option unusable with partition trees as at commit time an actual truncation was attempted. There are still issues with the way ON COMMIT actions are done when mixing several action types, however this impacts as well inheritance trees, so this issue will be dealt with later. Reported-by: Rajkumar Raghuwanshi Author: Amit Langote Reviewed-by: Michael Paquier, Tom Lane Discussion: https://postgr.es/m/CAKcux6mhgcjSiB_egqEAEFgX462QZtncU8QCAJ2HZwM-wWGVew@mail.gmail.com https://git.postgresql.org/pg/commitdiff/4bc772e2afa55f26734ff3fbdf27601db030b7e5 - Block creation of partitions with open references to its parent. When a partition is created as part of a trigger processing, it is possible that the partition which just gets created changes the properties of the table the executor of the ongoing command relies on, causing a subsequent crash. This has been found possible when for example using a BEFORE INSERT which creates a new partition for a partitioned table being inserted to. Any attempt to do so is blocked when working on a partition, with regression tests added for both CREATE TABLE PARTITION OF and ALTER TABLE ATTACH PARTITION. Reported-by: Dmitry Shalashov Author: Amit Langote Reviewed-by: Michael Paquier, Tom Lane Discussion: https://postgr.es/m/15437-3fe01ee66bd1bae1@postgresql.org Backpatch-through: 10 https://git.postgresql.org/pg/commitdiff/dc3e436b191a8f8d6f35fad952dd3dc314ccabf9 - Switch pg_promote to be parallel-safe. pg_promote uses nothing relying on a global state, so it is fine to mark it as parallel-safe, conclusion based on a detailed analysis from Robert Haas. This also fixes an inconsistency where pg_proc.dat missed to mark the function with its previous value for proparallel, update which does not matter now as the default is used. Based on a discussion between multiple folks: Laurenz Albe, Robert Haas, Amit Kapila, Tom Lane and myself. Discussion: https://postgr.es/m/20181029082530.GL14242@paquier.xyz https://git.postgresql.org/pg/commitdiff/8f045e242b2308473f5de532590454294e3d5683 - Reorganize format options of psql in alphabetical order. This makes the addition of new formats easier, and documentation lookups easier. Author: Daniel Vérité Reviewed-by: Fabien Coelho Discussion: https://postgr.es/m/alpine.DEB.2.20.1803081004241.2916@lancre https://git.postgresql.org/pg/commitdiff/add9182e59084faf57bb264fefcd39fbc93fb116 - Fix incorrect routine name reference in partprune.c. Author: Yuzuko Hosoya Discussion: https://postgr.es/m/00ac01d4774c$7feac860$7fc05920$@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/170dccc69ddb5eb379ebb5282d699738177d4543 - Fix dependency handling of partitions and inheritance for ON COMMIT This commit fixes a set of issues with ON COMMIT actions when used on partitioned tables and tables with inheritance children: - Applying ON COMMIT DROP on a partitioned table with partitions or on a table with inheritance children caused a failure at commit time, with complains about the children being already dropped as all relations are dropped one at the same time. - Applying ON COMMIT DELETE on a partition relying on a partitioned. table which uses ON COMMIT DROP would cause the partition truncation to fail as the parent is removed first. The solution to the first problem is to handle the removal of all the dependencies in one go instead of dropping relations one-by-one, based on a suggestion from Álvaro Herrera. So instead all the relation OIDs to remove are gathered and then processed in one round of multiple deletions. The solution to the second problem is to reorder the actions, with truncation happening first and relation drop done after. Even if it means that a partition could be first truncated, then immediately dropped if its partitioned table is dropped, this has the merit to keep the code simple as there is no need to do existence checks on the relations to drop. Contrary to a manual TRUNCATE on a partitioned table, ON COMMIT DELETE does not cascade to its partitions. The ON COMMIT action defined on each partition gets the priority. Author: Michael Paquier Reviewed-by: Amit Langote, Álvaro Herrera, Robert Haas Discussion: https://postgr.es/m/68f17907-ec98-1192-f99f-8011400517f5@lab.ntt.co.jp Backpatch-through: 10 https://git.postgresql.org/pg/commitdiff/319a8101804f3b62512fdce1a3af1c839344b593 - Fix incorrect routine name in xlog_heapam.h. s/xl_heap_delete/xl_heap_truncate/ in a comment block referring to flags for truncation. Discussion: https://postgr.es/m/20180413034734.GE1552@paquier.xyz https://git.postgresql.org/pg/commitdiff/3ce12018948b2c4ead9a47df7ae0b5ffa2fe06d0 Tom Lane pushed: - Remove unreferenced pg_opfamily entry. The entry with OID 4035, for GIST jsonb_ops, is unused; apparently it was added in preparation for index support that never materialized. Remove it, and add a regression test case to detect future mistakes of the same kind. Discussion: https://postgr.es/m/17188.1541379745@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/55f3d1029676ad53621716fd15faed5848c9f4cd - Last-minute updates for release notes. I removed the item about the pg_stat_statements change from release-11.sgml, as part of a sweep to delete items already committed in 11.0; but actually we'd best keep it to ensure that people who've pg_upgraded their databases will take the requisite action. Also make said action more visible by making it into its own para. Noted by Jonathan Katz. https://git.postgresql.org/pg/commitdiff/b0a1ff8a6a01163f830490e6c542ff852d240690 - Remove useless symbol from Makefile.global. I added HAVE_IPV6 to Makefile.global way back in commit 7703e55c3 so that we could transmit its value to the shell-script version of initdb. Since initdb was rewritten in C, it's been finding that out from pg_config.h instead, so this is useless. Keeping it here just wastes configure and make cycles, plus it's a potential two-sources-of-truth problem. https://git.postgresql.org/pg/commitdiff/8f623bedfb4fee5a125b25720e5451379cf26ff9 - Rename rbtree.c functions to use "rbt" prefix not "rb" prefix. The "rb" prefix is used by Ruby, so that our existing code results in name collisions that break plruby. We discussed ways to prevent that by adjusting dynamic linker options, but it seems that at best we'd move the pain to other cases. Renaming to avoid the collision is the only portable fix anyway. Fortunately, our rbtree code is not (yet?) widely used --- in core, there's only a single usage in GIN --- so it seems likely that we can get away with a rename. I chose to do this basically as s/rb/rbt/g, except for places where there already was a "t" after "rb". The patch could have been made smaller by only touching linker-visible symbols, but it would have resulted in oddly inconsistent-looking code. Better to make it look like "rbt" was the plan all along. Back-patch to v10. The rbtree.c code exists back to 9.5, but rb_iterate() which is the actual immediate source of pain was added in v10, so it seems like changing the names before that would have more risk than benefit. Per report from Pavel Raiskup. Discussion: https://postgr.es/m/4738198.8KVIIDhgEB@nb.usersys.redhat.com https://git.postgresql.org/pg/commitdiff/003c68a3b45d0d135b874acfe04cf3fb79a6f172 - Disable recheck_on_update optimization to avoid crashes. The code added by commit c203d6cf8 causes a crash in at least one case, where a potentially-optimizable expression index has a storage type different from the input data type. A cursory code review turned up numerous other problems that seem impractical to fix on short notice. Andres argued for revert of that patch some time ago, and if additional senior committers had been paying attention, that's likely what would have happened, but we were not :-( At this point we can't just revert, at least not in v11, because that would mean an ABI break for code touching relcache entries. And we should not remove the (also buggy) support for the recheck_on_update index reloption, since it might already be used in some databases in the field. So this patch just does the as-little-invasive-as-possible measure of disabling the feature as though recheck_on_update were forced off for all indexes. I also removed the related regression tests (which would otherwise fail) and the user-facing documentation of the reloption. We should undertake a more thorough code cleanup if the patch can't be fixed, but not under the extreme time pressure of being already overdue for 11.1 release. Per report from Ondřej Bouda and subsequent private discussion among pgsql-release. Discussion: https://postgr.es/m/20181106185255.776mstcyehnc63ty@alvherre.pgsql https://git.postgresql.org/pg/commitdiff/5d28c9bd73e29890cccd3f6b188b86f81031f671 - Last-minute updates for release notes. Add entries for v11 changes that went in post-stamping, but before the final wrap. https://git.postgresql.org/pg/commitdiff/77366d90f44a4e1c15d36e51061ffbc17f0dd12d - Postpone calculating total_table_pages until after pruning/exclusion. The planner doesn't make any use of root->total_table_pages until it estimates costs of indexscans, so we don't need to compute it as early as that's currently done. By doing the calculation between set_base_rel_sizes and set_base_rel_pathlists, we can omit relations that get removed from the query by partition pruning or constraint exclusion, which seems like a more accurate basis for costing. (Historical note: I think at the time this code was written, there was not a separation between the "set sizes" and "set pathlists" steps, so that this approach would have been impossible at the time. But now that we have that separation, this is clearly the better way to do things.) David Rowley, reviewed by Edmund Horner Discussion: https://postgr.es/m/CAKJS1f-NG1mRM0VOtkAG7=ZLQWihoqees9R4ki3CKBB0-fRfCA@mail.gmail.com https://git.postgresql.org/pg/commitdiff/c6e4133fae1fde93769197379ffcc2b379845113 - Fix inadequate autoconfiscation of copyfile() usage. Per buildfarm, HAVE_COPYFILE is not the same thing as HAVE_COPYFILE_H. Add the extra configure test. https://git.postgresql.org/pg/commitdiff/c3e6d5d38699812e047270a711be1926b7f6f396 - Last-minute updates for release notes. Security: CVE-2018-16850 https://git.postgresql.org/pg/commitdiff/1eaeb02a30b307bad4d6a9cc2d596273a5a38a7a - Disallow setting client_min_messages higher than ERROR. Previously it was possible to set client_min_messages to FATAL or PANIC, which had the effect of suppressing transmission of regular ERROR messages to the client. Perhaps that seemed like a useful option in the past, but the trouble with it is that it breaks guarantees that are explicitly made in our FE/BE protocol spec about how a query cycle can end. While libpq and psql manage to cope with the omission, that's mostly because they are not very bright; client libraries that have more semantic knowledge are likely to get confused. Notably, pgODBC doesn't behave very sanely. Let's fix this by getting rid of the ability to set client_min_messages above ERROR. In HEAD, just remove the FATAL and PANIC options from the set of allowed enum values for client_min_messages. (This change also affects trace_recovery_messages, but that's OK since these aren't useful values for that variable either.) In the back branches, there was concern that rejecting these values might break applications that are explicitly setting things that way. I'm pretty skeptical of that argument, but accommodate it by accepting these values and then internally setting the variable to ERROR anyway. In all branches, this allows a couple of tiny simplifications in the logic in elog.c, so do that. Also respond to the point that was made that client_min_messages has exactly nothing to do with the server's logging behavior, and therefore does not belong in the "When To Log" subsection of the documentation. The "Statement Behavior" subsection is a better match, so move it there. Jonah Harris and Tom Lane Discussion: https://postgr.es/m/7809.1541521180@sss.pgh.pa.us Discussion: https://postgr.es/m/15479-ef0f4cc2fd995ca2@postgresql.org https://git.postgresql.org/pg/commitdiff/3d360e20c9ae48a8139b4dd84f5a8bacc392d8e9 - Fix missing role dependencies for some schema and type ACLs. This patch fixes several related cases in which pg_shdepend entries were never made, or were lost, for references to roles appearing in the ACLs of schemas and/or types. While that did no immediate harm, if a referenced role were later dropped, the drop would be allowed and would leave a dangling reference in the object's ACL. That still wasn't a big problem for normal database usage, but it would cause obscure failures in subsequent dump/reload or pg_upgrade attempts, taking the form of attempts to grant privileges to all-numeric role names. (I think I've seen field reports matching that symptom, but can't find any right now.) Several cases are fixed here: 1. ALTER DOMAIN SET/DROP DEFAULT would lose the dependencies for any existing ACL entries for the domain. This case is ancient, dating back as far as we've had pg_shdepend tracking at all. 2. If a default type privilege applies, CREATE TYPE recorded the ACL properly but forgot to install dependency entries for it. This dates to the addition of default privileges for types in 9.2. 3. If a default schema privilege applies, CREATE SCHEMA recorded the ACL properly but forgot to install dependency entries for it. This dates to the addition of default privileges for schemas in v10 (commit ab89e465c). Another somewhat-related problem is that when creating a relation rowtype or implicit array type, TypeCreate would apply any available default type privileges to that type, which we don't really want since such an object isn't supposed to have privileges of its own. (You can't, for example, drop such privileges once they've been added to an array type.) ab89e465c is also to blame for a race condition in the regression tests: privileges.sql transiently installed globally-applicable default privileges on schemas, which sometimes got absorbed into the ACLs of schemas created by concurrent test scripts. This should have resulted in failures when privileges.sql tried to drop the role holding such privileges; but thanks to the bug fixed here, it instead led to dangling ACLs in the final state of the regression database. We'd managed not to notice that, but it became obvious in the wake of commit da906766c, which allowed the race condition to occur in pg_upgrade tests. To fix, add a function recordDependencyOnNewAcl to encapsulate what callers of get_user_default_acl need to do; while the original call sites got that right via ad-hoc code, none of the later-added ones have. Also change GenerateTypeDependencies to generate these dependencies, which requires adding the typacl to its parameter list. (That might be annoying if there are any extensions calling that function directly; but if there are, they're most likely buggy in the same way as the core callers were, so they need work anyway.) While I was at it, I changed GenerateTypeDependencies to accept most of its parameters in the form of a Form_pg_type pointer, making its parameter list a bit less unwieldy and mistake-prone. The test race condition is fixed just by wrapping the addition and removal of default privileges into a single transaction, so that that state is never visible externally. We might eventually prefer to separate out tests of default privileges into a script that runs by itself, but that would be a bigger change and would make the tests run slower overall. Back-patch relevant parts to all supported branches. Discussion: https://postgr.es/m/15719.1541725287@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/fa2952d8eb0423a02fde7b36f748f8de11aec8db - Fix error-cleanup mistakes in exec_stmt_call(). Commit 15c729347 was a couple bricks shy of a load: we need to ensure that expr->plan gets reset to NULL on any error exit, if it's not supposed to be saved. Also ensure that the stmt->target calculation gets redone if needed. The easy way to exhibit a problem is to set up code that violates the writable-argument restriction and then execute it twice. But error exits out of, eg, setup_param_list() could also break it. Make the existing PG_TRY block cover all of that code to be sure. Per report from Pavel Stehule. Discussion: https://postgr.es/m/CAFj8pRAeXNTO43W2Y0Cn0YOVFPv1WpYyOqQrrzUiN6s=dn7gCg@mail.gmail.com https://git.postgresql.org/pg/commitdiff/f26c06a4046b62c04ab4a8ef8632a5f705b6dd2d Thomas Munro pushed: - Remove dsm_resize() and dsm_remap(). These interfaces were never used in core, didn't handle failure of posix_fallocate() correctly and weren't supported on all platforms. We agreed to remove them in 12. Author: Thomas Munro Reported-by: Andres Freund Discussion: https://postgr.es/m/CAA4eK1%2B%3DyAFUvpFoHXFi_gm8YqmXN-TtkFH%2BVYjvDLS6-SFq-Q%40mail.gmail.com https://git.postgresql.org/pg/commitdiff/3c60d0fa231f7699af5dd83345919d0423305973 - Remove some remaining traces of dsm_resize(). A couple of obsolete comments and unreachable blocks remained after commit 3c60d0fa. Discussion: https://postgr.es/m/CAA4eK1%2B%3DyAFUvpFoHXFi_gm8YqmXN-TtkFH%2BVYjvDLS6-SFq-Q%40mail.gmail.com https://git.postgresql.org/pg/commitdiff/9e12fb02b7ec27ff3707a0a7c31f2421ba62c9df - Provide pg_pread() and pg_pwrite() for random I/O. Forward to POSIX pread() and pwrite(), or emulate them if unavailable. The emulation is not perfect as the file position is changed, so we'll put pg_ prefixes on the names to minimize the risk of confusion in future patches that might inadvertently try to mix pread() and read() on the same file descriptor. Author: Thomas Munro Reviewed-by: Tom Lane, Jesper Pedersen Discussion: https://postgr.es/m/CAEepm=02rapCpPR3ZGF2vW=SBHSdFYO_bz_f-wwWJonmA3APgw@mail.gmail.com https://git.postgresql.org/pg/commitdiff/3fd2a7932ef0708dda57369bb20c0499d905cc82 - Use pg_pread() and pg_pwrite() for data files and WAL. Cut down on system calls by doing random I/O using offset-based OS routines where available. Remove the code for tracking the 'virtual' seek position. The only reason left to call FileSeek() was to get the file's size, so provide a new function FileSize() instead. Author: Oskari Saarenmaa, Thomas Munro Reviewed-by: Thomas Munro, Jesper Pedersen, Tom Lane, Alvaro Herrera Discussion: https://postgr.es/m/CAEepm=02rapCpPR3ZGF2vW=SBHSdFYO_bz_f-wwWJonmA3APgw@mail.gmail.com Discussion: https://postgr.es/m/b8748d39-0b19-0514-a1b9-4e5a28e6a208%40gmail.com Discussion: https://postgr.es/m/a86bd200-ebbe-d829-e3ca-0c4474b2fcb7%40ohmu.fi https://git.postgresql.org/pg/commitdiff/c24dcd0cfd949bdf245814c4c2b3df828ee7db36 - Remove set-but-unused variable. Clean-up for commit c24dcd0c. Reported-by: Andrew Dunstan Discussion: https://postgr.es/m/2d52ff4a-5440-f6f1-7806-423b0e6370cb%402ndQuadrant.com https://git.postgresql.org/pg/commitdiff/c4f0876fb8d8f84fc0a6fb6aee91de120718664c Bruce Momjian pushed: - GUC: adjust effective_cache_size SQL descriptions. Follow on patch for commit 3e0f1a4741f564c1a2fa6e944729d6967355d8c7. Reported-by: Peter Eisentraut Discussion: https://postgr.es/m/369ec766-b947-51bd-4dad-6fb9e026439f@2ndquadrant.com Backpatch-through: 9.4 https://git.postgresql.org/pg/commitdiff/b43df566b372650a9b9e2a0dd9e695c1f16da14f Andrew Gierth pushed: - Optimize nested ConvertRowtypeExpr nodes. A ConvertRowtypeExpr is used to translate a whole-row reference of a child to that of a parent. The planner produces nested ConvertRowtypeExpr while translating whole-row reference of a leaf partition in a multi-level partition hierarchy. Executor then translates the whole-row reference from the leaf partition into all the intermediate parent's whole-row references before arriving at the final whole-row reference. It could instead translate the whole-row reference from the leaf partition directly to the top-most parent's whole-row reference skipping any intermediate translations. Ashutosh Bapat, with tests by Kyotaro Horiguchi and some editorialization by me. Reviewed by Andres Freund, Pavel Stehule, Kyotaro Horiguchi, Dmitry Dolgov, Tom Lane. https://git.postgresql.org/pg/commitdiff/5613da4cc7910f4b686b818dd1ee68703d62192f Andres Freund pushed: - Use installcheck-parallel in pg_upgrade's testsuite. The installcheck run takes a sizable fraction of test.sh to run. Using a parallel schedule reduces that noticably. It's possible that we want to backpatch this at some point, to reduce buildfarm times, but for now lets just see if this upsets the buildfarm. Author: Andres Freund Discussion: https://postgr.es/m/20181105210039.hh4vvi4vwoq5ba2q@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/da906766cd3bef8290db25bd5c8776c03689f3d5 - Build HashState's hashkeys expression with the correct parent. Previously the expressions were built with the HashJoinState as a parent. That's incorrect. Currently this does not appear to be harmful, but for the upcoming 'slot abstraction' work this proves to be problematic, as the underlying slot types can differ between Hash and HashJoin. It's possible that this already causes a problem, but I've not been able to come up with a scenario. Therefore don't backpatch at this point. Author: Andres Freund Discussion: https://postgr.es/m/20180220224318.gw4oe5jadhpmcdnm@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/5f32b29c18195299e90c1fb6c8945e9a46d772d2 - Fixup for b84a6dafbf triggering assert failure in LLVM debug builds. Author: Andres Freund https://git.postgresql.org/pg/commitdiff/c14a8ff27e26155345e554a9636bab8236500bb9 - Fix copy-paste error in errhint() introduced in 691d79a07933. Reported-By: Petr Jelinek Discussion: https://postgr.es/m/c95a620b-34f0-7930-aeb5-f7ab804f26cb@2ndquadrant.com Backpatch: 9.4-, like the previous commit https://git.postgresql.org/pg/commitdiff/a3fb382e9c166e398763e3bbf706b6c92a647fe1 - Move EEOP_*_SYSVAR evaluation out of line. This mainly de-duplicates code. As evaluating a system variable isn't the hottest path and the current inline implementation ends up calling out to an external function anyway, this is OK from a performance POV. The main motivation for de-duplicating is the upcoming slot abstraction work, after which there's not guaranteed to be a HeapTuple backing the slot. Author: Andres Freund, Amit Khandekar Discussion: https://postgr.es/m/20181105210039.hh4vvi4vwoq5ba2q@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/b84a6dafbf2bb921baee53c0c1aba7719ee38817 - Don't require return slots for nodes without projection. In a lot of nodes the return slot is not required. That can either be because the node doesn't do any projection (say an Append node), or because the node does perform projections but the projection is optimized away because the projection would yield an identical row. Slots aren't that small, especially for wide rows, so it's worthwhile to avoid creating them. It's not possible to just skip creating the slot - it's currently used to determine the tuple descriptor returned by ExecGetResultType(). So separate the determination of the result type from the slot creation. The work previously done internally ExecInitResultTupleSlotTL() can now also be done separately with ExecInitResultTypeTL() and ExecInitResultSlot(). That way nodes that aren't guaranteed to need a result slot, can use ExecInitResultTypeTL() to determine the result type of the node, and ExecAssignScanProjectionInfo() (via ExecConditionalAssignProjectionInfo()) determines that a result slot is needed, it is created with ExecInitResultSlot(). Besides the advantage of avoiding to create slots that then are unused, this is necessary preparation for later patches around tuple table slot abstraction. In particular separating the return descriptor and slot is a prerequisite to allow JITing of tuple deforming with knowledge of the underlying tuple format, and to avoid unnecessarily creating JITed tuple deforming for virtual slots. This commit removes a redundant argument from ExecInitResultTupleSlotTL(). While this commit touches a lot of the relevant lines anyway, it'd normally still not worthwhile to cause breakage, except that aforementioned later commits will touch *all* ExecInitResultTupleSlotTL() callers anyway (but fits worse thematically). Author: Andres Freund Discussion: https://postgr.es/m/20181105210039.hh4vvi4vwoq5ba2q@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/1ef6bd2954c4ec63ff8a2c9c4ebc38251d7ef5c5 - docs: Adapt wal_segment_size docs to fc49e24fa69. Before this change the docs weren't adapted to the fact that wal_segment_size is now measured in bytes, rather than multiples of wal_block_size. Author: David Steele Discussion: https://postgr.es/m/68ea97d6-2ed9-f339-e57d-ab3a33caf3b1@pgmasters.net Backpatch: 11-, like fc49e24fa69 itself. https://git.postgresql.org/pg/commitdiff/5fc1670bad9bfa138be1c790d529249e0706d38e - Combine two flag tests in GetSnapshotData(). Previously the code checked PROC_IN_LOGICAL_DECODING and PROC_IN_VACUUM separately. As the relevant variable is marked as volatile, the compiler cannot combine the two tests. As GetSnapshotData() is pretty hot in a number of workloads, it's worthwhile to fix that. It'd also be a good idea to get rid of the volatiles altogether. But for one that's a larger patch, and for another, the code after this change still seems at least as easy to read as before. Author: Andres Freund Discussion: https://postgr.es/m/20181005172955.wyjb4fzcdzqtaxjq@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/5fde047f2b41e71706d4ea9e8eb0a401f64192c1 - Remove ineffective check against dropped columns from slot_getattr(). Before this commit slot_getattr() checked for dropped columns (returning NULL in that case), but only after checking for previously deformed columns. As slot_deform_tuple() does not contain such a check, the check in slot_getattr() would often not have been reached, depending on previous use of the slot. These days locking and plan invalidation ought to ensure that dropped columns are not accessed in query plans. Therefore this commit just drops the insufficient check in slot_getattr(). It's possible that we'll find some holes againt use of dropped columns, but if so, those need to be addressed independent of slot_getattr(), as most accesses don't go through that function anyway. Author: Andres Freund Discussion: https://postgr.es/m/20181107174403.zai7fedgcjoqx44p@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/c670d0faace6184216c349a4cf830aa415c58068 - Remove volatiles from {procarray,volatile}.c and fix memory ordering issue. The use of volatiles in procarray.c largely originated from the time when postgres did not have reliable compiler and memory barriers. That's not the case anymore, so we can do better. Several of the functions in procarray.c can be bottlenecks, and removal of volatile yields mildly better code. The new state, with explicit memory barriers, is also more correct. The previous use of volatile did not actually deliver sufficient guarantees on weakly ordered machines, in particular the logic in GetNewTransactionId() does not look safe. It seems unlikely to be a problem in practice, but worth fixing. Thomas and I independently wrote a patch for this. Reported-By: Andres Freund and Thomas Munro Author: Andres Freund, with cherrypicked changes from a patch by Thomas Munro Discussion: https://postgr.es/m/20181005172955.wyjb4fzcdzqtaxjq@alap3.anarazel.de https://postgr.es/m/CAEepm=1nff0x=7i3YQO16jLA2qw-F9O39YmUew4oq-xcBQBs0g@mail.gmail.com https://git.postgresql.org/pg/commitdiff/450c7defa6d5f3c9fdb6efc92cca106c7b582cb3 Peter Eisentraut pushed: - pg_upgrade: Allow use of file cloning. Add another transfer mode --clone to pg_upgrade (besides the existing --link and the default copy), using special file cloning calls. This makes the file transfer faster and more space efficient, achieving speed similar to --link mode without the associated drawbacks. On Linux, file cloning is supported on Btrfs and XFS (if formatted with reflink support). On macOS, file cloning is supported on APFS. Reviewed-by: Michael Paquier <michael@paquier.xyz> https://git.postgresql.org/pg/commitdiff/3a769d8239afdc003c91a56d2d8d5adfadacda5d - Apply RI trigger skipping tests also for DELETE. The tests added in cfa0f4255bb0f5550d37a01c4d8fe2966d20040c to skip firing an RI trigger if any old key value is NULL can also be applied for DELETE. This should give a performance gain in those cases, and it also saves a lot of duplicate code in the actual RI triggers. (That code was already dead code for the UPDATE cases.) Reviewed-by: Daniel Gustafsson <daniel@yesql.se> https://git.postgresql.org/pg/commitdiff/69ee2ff9308bbfe0e0a89667555e901fec5e154b - Remove dead foreign key optimization code. The ri_KeysEqual() calls in the foreign-key trigger functions to optimize away some updates are useless because since adfeef55cbcc5dc72a772777f88c1be05a70dfee those triggers are not enqueued at all. (It's also not useful to keep these checks as some kind of backstop, since it's also semantically correct to just run the full check even with equal keys.) Reviewed-by: Daniel Gustafsson <daniel@yesql.se> https://git.postgresql.org/pg/commitdiff/34479d9a36a280afe1387146871d4c1528cf2cbf Andrew Dunstan pushed: - Quiet valgrind complaints following pread/pwrite changes. Per complaints from buildfarm and elsewhere Patch from Jasper Pedersen Discussion: https://postgr.es/m/0f419c91-49ab-2399-0143-13063bd97c46@redhat.com https://git.postgresql.org/pg/commitdiff/517b0d0b5f38c92c1807a40d71ca4bf6428ca7d5 - Use parallel installcheck in vcregress.pl's upgrade test. This is to keep the test in sync with what's done in test.sh, which acquired this change in commit da906766c. https://git.postgresql.org/pg/commitdiff/54ad7282fe7ad80a15533546d3b0a2be2e19e3f6 - Adjust valgrind fix in commit 517b0d0b5f. lousyjack still wasn't happy. I have tested this modification and it worked. https://git.postgresql.org/pg/commitdiff/12d5f39b15c31c0adc6c7ed06979af340e494fb9 - Disable MSVC warning caused by recent snprintf.c changes. Discussion: https://postgr.es/m/05f348de-0c79-d88d-69b7-434ef828bd4d@2ndQuadrant.com https://git.postgresql.org/pg/commitdiff/7f284debaf1933b322ff54a643c6785362a034d3 Álvaro Herrera pushed: - Revise attribute handling code on partition creation. The original code to propagate NOT NULL and default expressions specified when creating a partition was mostly copy-pasted from typed-tables creation, but not being a great match it contained some duplicity, inefficiency and bugs. This commit fixes the bug that NOT NULL constraints declared in the parent table would not be honored in the partition. One reported issue that is not fixed is that a DEFAULT declared in the child is not used when inserting through the parent. That would amount to a behavioral change that's better not back-patched. This rewrite makes the code simpler: 1. instead of checking for duplicate column names in its own block, reuse the original one that already did that; 2. instead of concatenating the list of columns from parent and the one declared in the partition and scanning the result to (incorrectly) propagate defaults and not-null constraints, just scan the latter searching the former for a match, and merging sensibly. This works because we know the list in the parent is already correct and there can only be one parent. This rewrite makes ColumnDef->is_from_parent unused, so it's removed on branch master; on released branches, it's kept as an unused field in order not to cause ABI incompatibilities. This commit also adds a test case for creating partitions with collations mismatching that on the parent table, something that is closely related to the code being patched. No code change is introduced though, since that'd be a behavior change that could break some (broken) working applications. Amit Langote wrote a less invasive fix for the original NOT NULL/defaults bug, but while I kept the tests he added, I ended up not using his original code. Ashutosh Bapat reviewed Amit's fix. Amit reviewed mine. Author: Álvaro Herrera, Amit Langote Reviewed-by: Ashutosh Bapat, Amit Langote Reported-by: Jürgen Strobel (bug #15212) Discussion: https://postgr.es/m/152746742177.1291.9847032632907407358@wrigleys.postgresql.org https://git.postgresql.org/pg/commitdiff/705d433fd5e29ac173ef46a5acd4a2d7336f5573 - Indicate session name in isolationtester notices. When a session under isolationtester produces printable notices (NOTICE, WARNING) we were just printing them unadorned, which can be confusing when debugging. Prefix them with the session name, which makes things clearer. Author: Álvaro Herrera Reviewed-by: Hari Babu Kommi Discussion: https://postgr.es/m/20181024213451.75nh3f3dctmcdbfq@alvherre.pgsql https://git.postgresql.org/pg/commitdiff/a28e10e82e540d45e448bc404af76df8a54008f1 == Pending Patches == Pavel Stěhule sent in another revision of a patch to implement \dP (show partition information) in psql. Dilip Kumar sent in two more revisions of a patch to help with the UNDO system. Lam Jun Rong sent in two more revisions of a patch to update and improve the "Getting Started" documentation. Kuntal Ghosh sent in another revision of a patch to fix shared memory size for rollback hash table. Dilip Kumar sent in another revision of a patch to implement UNDO. John Dent sent in two revisions of a patch to implement delta refreshes for materialized views using a WHERE clause. Peter Geoghegan sent in a patch to implement some questionable/tentative ASC regress output fixes. Emre Hasegeli and Álvaro Herrera traded patches to quiet Coverity with respect to the improved geometry types. Daisuke Higuchi sent in another revision of a patch to add tests for a fixed bug in ECPG numerics. Surafel Temesgen sent in two more revisions of a patch to add an option to pg_dump to output multi-row INSERTs. Kyotaro HORIGUCHI sent in a patch to fix an infelicity between slot allocation and replication_reserved_connections. Pavel Stěhule sent in two revisions of a patch to fix a bug in xmltable xpath result processing. Kyotaro HORIGUCHI sent in two revisions of a patch to fix an issue which manifested as file_fdw using a program could cause an exit code error when using LIMIT. Nikita Glukhov sent in two more revisions of a patch to implement JSONPATH. Daniel Vérité and Michaël Paquier traded patches to add a csv output format to psql. Aleksey Kondratov sent in another revision of a patch to fix some issues in pg_rewind. Daniel Gustafsson sent in another revision of a patch to support a custom socket directory during upgrades. Amit Langote sent in two more revisions of a patch to move the PartitionBoundInfo creation code to partbounds.c. Pavel Stěhule sent in another revision of a patch to fix psql's \conninfo and \connect when using hostaddr. Edmund Horner sent in a patch to ExecStoreTuple notes from nodeSeqscan.c and nodeIndexscan.c. David Rowley sent in two more revisions of a patch to speed up INSERTs and UPDATEs to partitioned tables Yoshikazu Imai sent in a patch to rearrange the order in which inheritance_make_rel_from_joinlist() translates join_list and join_info_list. David Rowley sent in two more revisions of a patch to better document PostgreSQL's limits. Daniel Gustafsson sent in a patch to avoid requesting unused NULLs from extconfig. Haribabu Kommi sent in another revision of a patch to implement pg_stat_statements_reset() to reset specific query/user/db statistics. Amit Kapila sent in two more revisions of a patch to fix the initialization of atomic variables introduced by the group clearing mechanism. David Rowley sent in a patch, for performance demonstration purposes only, which does unsafe locking reduction for partitioned INSERT/UPDATEs. Michaël Paquier sent in a patch to add a TAP test checking data consistency on standby with minRecoveryPoint. Kyotaro HORIGUCHI sent in two more revisions of a patch to implement the stats collector in shared memory rather than on disk. Konstantin Knizhnik sent in two more revisions of a patch to refactor projection indexes. Pavel Stěhule sent in a patch to make LEAST and GREATEST take variadic arrays, as other variadic functions do. Michaël Paquier sent in a patch to remove an unnecessary CommandCounterIncrement() call when doing ON COMMIT DELETE ROWS. Amit Langote sent in another revision of a patch to allow generalized expression syntax for partition bounds. Haribabu Kommi sent in another revision of a patch to allow target session attrs to accept prefer-read option. Dmitry Dolgov sent in another revision of a patch to implement generic type subscripting. Alexander Kuzmenkov sent in a patch to fix a problem that manifested as uninterruptible long planning of a query with too many WHERE clauses by limiting the number of clauses the code attempts to identify uniquely. Daniel Vérité sent in a patch to implement \copyfrom and \copyto in psql. Amit Langote and David Rowley traded patches to speed up planning with partitions. Michaël Paquier sent in a patch to mark flag output from wal_dump with 0x. James Coleman sent in a patch to ensure that ScalarArrayOpExpr is not NULL.
pgsql-announce by date: