== PostgreSQL Weekly News - April 14, 2019 == - Mailing list pgsql-announce
| From | David Fetter |
|---|---|
| Subject | == PostgreSQL Weekly News - April 14, 2019 == |
| Date | |
| Msg-id | 20190414181649.GA12174@fetter.org Whole thread Raw |
| List | pgsql-announce |
== PostgreSQL Weekly News - April 14, 2019 == PostgresConf South Africa 2019 will take place in Johannesburg on October 8-9, 2019 The Call for Papers is open through June 30, 2019. https://postgresconf.org/conferences/SouthAfrica2019 == PostgreSQL Product News == pgMustard, a user interface for 'explain analyze' which provides performance tips, released. https://www.pgmustard.com/ repods, a cloud data platform based on PostgreSQL, released. https://repods.io pgAdmin4 4.5, a web- and native GUI control center for PostgreSQL, released. https://www.pgadmin.org/docs/pgadmin4/dev/release_notes_4_5.html == PostgreSQL Jobs for April == http://archives.postgresql.org/pgsql-jobs/2019-04/ == PostgreSQL Local == The German-speaking PostgreSQL Conference 2019 will take place on May 10, 2019 in Leipzig. http://2019.pgconf.de/ PGDay.IT 2019 will take place May 16th and May 17th in Bologna, Italy. https://2019.pgday.it/en/ PGCon 2019 will take place in Ottawa on May 28-31, 2019. Registration is open. https://www.pgcon.org/2019 Swiss PGDay 2019 will take place in Rapperswil (near Zurich) on June 28, 2019. The CfP is open through April 18, 2019, and registration is open. http://www.pgday.ch/2019/ PostgresLondon 2019 will be July 2-3, 2019 with an optional training day on July 1. http://postgreslondon.org PGConf.Brazil 2019 will take place August 1-3, 2019 in São Paulo. http://pgconf.com.br The first Austrian pgDay, will take place September 6, 2019 at the Hilton Garden Inn in Wiener Neustadt. https://pgday.at/en/ == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm PST8PDT to david@fetter.org. == Applied Patches == Tom Lane pushed: - Avoid fetching past the end of the indoption array. pg_get_indexdef_worker carelessly fetched indoption entries even for non-key index columns that don't have one. 99.999% of the time this would be harmless, since the code wouldn't examine the value ... but some fine day this will be a fetch off the end of memory, resulting in SIGSEGV. Detected through valgrind testing. Odd that the buildfarm's valgrind critters haven't noticed. https://git.postgresql.org/pg/commitdiff/80a96e066eecb6bd1788964b5911a405d932a784 - Fix EvalPlanQualStart to handle partitioned result rels correctly. The es_root_result_relations array needs to be shallow-copied in the same way as the main es_result_relations array, else EPQ rechecks on partitioned result relations fail, as seen in bug #15677 from Norbert Benkocs. Amit Langote, isolation test case added by me Discussion: https://postgr.es/m/15677-0bf089579b4cd02d@postgresql.org Discussion: https://postgr.es/m/19321.1554567786@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/a8cb8f124679e0c373fdd07108b136e1cf1ee14a - Fix improper interaction of FULL JOINs with lateral references. join_is_legal() needs to reject forming certain outer joins in cases where that would lead the planner down a blind alley. However, it mistakenly supposed that the way to handle full joins was to treat them as applying the same constraints as for left joins, only to both sides. That doesn't work, as shown in bug #15741 from Anthony Skorski: given a lateral reference out of a join that's fully enclosed by a full join, the code would fail to believe that any join ordering is legal, resulting in errors like "failed to build any N-way joins". However, we don't really need to consider full joins at all for this purpose, because we effectively force them to be evaluated in syntactic order, and that order is always legal for lateral references. Hence, get rid of this broken logic for full joins and just ignore them instead. This seems to have been an oversight in commit 7e19db0c0. Back-patch to all supported branches, as that was. Discussion: https://postgr.es/m/15741-276f1f464b3f40eb@postgresql.org https://git.postgresql.org/pg/commitdiff/45f8eaa8e3031c9cb12deb1b5e294bc052b378f2 - Test some more cases with partitioned tables in EvalPlanQual. We weren't testing anything involving EPQ on UPDATEs that move tuples into different partitions. Depending on the implementation, it might be that these cases aren't actually very interesting ... but given our thin coverage of EPQ in general, I think it's a good idea to have a test case. Amit Langote, minor tweak by me Discussion: https://postgr.es/m/7889df35-ad1a-691a-00e3-4d4b18f364e3@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/a2418f9e238794fcaaf00bbd5b8f953ca2856aa0 - Prevent inlining of multiply-referenced CTEs with outer recursive refs. This has to be prevented because inlining would result in multiple self-references, which we don't support (and in fact that's disallowed by the SQL spec, see statements about linearly vs. nonlinearly recursive queries). Bug fix for commit 608b167f9. Per report from Yaroslav Schekin (via Andrew Gierth) Discussion: https://postgr.es/m/87wolmg60q.fsf@news-spur.riddles.org.uk https://git.postgresql.org/pg/commitdiff/9476131278c7bfc435ad9a21fc8e981272ac0dd2 - Fix backwards test in operator_precedence_warning logic. Warnings about unary minus might have been wrong. It's a bit surprising that nobody noticed yet ... probably the precedence-warning feature hasn't really been used much in the field. Rikard Falkeborn Discussion: https://postgr.es/m/CADRDgG6fzA8A2oeygUw4=o7ywo4kvz26NxCSgpq22nMD73Bx4Q@mail.gmail.com https://git.postgresql.org/pg/commitdiff/4cae471d1b6bec7493dcb2ca156382bef738f293 - Remove redundant and ineffective test for btree insertion fast path. indexing.sql's test for this feature was added along with the feature in commit 2b2727343. However, shortly later that test was rendered ineffective by commit 074251db6, which limited when the optimization would be applied, so that the test didn't test it. Since then, commit dd299df81 added new tests (in btree_index.sql) that actually do test the feature. Code coverage comparisons confirm that this test sequence adds no meaningful coverage, and it's rather expensive, accounting for nearly half of the runtime of indexing.sql according to my measurements. So let's remove it. Per advice from Peter Geoghegan. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/f72d9a5e7dda5f89336a60e8b720ef9964a67177 - Remove duplicative polygon SP-GiST sequencing test. Code coverage comparisons confirm that the tests using quad_poly_tbl_ord_seq1/quad_poly_tbl_ord_idx1 hit no code paths not also covered by the similar tests using quad_poly_tbl_ord_seq2/quad_poly_tbl_ord_idx2. Since these test cases are pretty expensive, they need to contribute more than zero benefit. In passing, make quad_poly_tbl_ord_seq2 a temp table, since there seems little reason to keep it around after the test. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/4aaa3b5cf1c33d713b8eefa3612a9112cbcf1074 - Move plpgsql error-trapping tests to a new module-specific test file. The test for statement timeout has a 2-second timeout, which was only moderately annoying when it was written, but nowadays it contributes a pretty significant chunk of the elapsed time needed to run the core regression tests on a fast machine. We can improve this situation by pushing the test into a plpgsql-specific test file instead of having it in a core regression test. That's a clean win when considering just the core tests. Even when considering check-world or a buildfarm test run, we should come out ahead because the core tests get run more times in those sequences. Furthermore, since the plpgsql tests aren't currently parallelized, it seems likely that the timing problems reflected in commit f1e671a0b (which increased that timeout from 1 sec to 2) will be much less severe in this context. Hence, let's try cutting the timeout back to 1 second in hopes of a further win for check-world. We can undo that if buildfarm experience proves it to be a bad idea. To give the new test file some modicum of intellectual coherency, I moved the surrounding tests related to error-trapping along with the statement timeout test proper. Those other tests don't run long enough to have any particular bearing on test-runtime considerations. The tests are the same as before, except with minor adjustments to not depend on an externally-created table. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/6726d8d476b424633ebdc7068da3f5a6e6da10af - Split up a couple of long-running regression test scripts. The point of this change is to increase the potential for parallelism while running the core regression tests. Most people these days are using parallel testing modes on multi-core machines, so we might as well try a bit harder to keep multiple cores busy. Hence, a test that runs much longer than others in its parallel group is a candidate to be sub-divided. In this patch, create_index.sql and join.sql are split up. I haven't changed the content of the tests in any way, just moved them. I moved create_index.sql's SP-GiST-related tests into a new script create_index_spgist, and moved its btree multilevel page deletion test over to the existing script btree_index. (btree_index is a more natural home for that test, and it's shorter than others in its parallel group, so this doesn't hurt total runtime of that group.) There might be room for more aggressive splitting of create_index, but this is enough to improve matters considerably. Likewise, I moved join.sql's "exercises for the hash join code" into a new file join_hash. Those exercises contributed three-quarters of the script's runtime. Which might well be excessive ... but for the moment, I'm satisfied with shoving them into a different parallel group, where they can share runtime with the roughly-equally-lengthy gist test. (Note for anybody following along at home: there are interesting interactions between the runtimes of create_index and anything running in parallel with it, because the tests of CREATE INDEX CONCURRENTLY in that file will repeatedly block waiting for concurrent transactions to commit. As committed in this patch, create_index and create_index_spgist have roughly equal runtimes, but that's mostly an artifact of forced synchronization of the CONCURRENTLY tests; when run serially, create_index is much faster. A followup patch will reduce the runtime of create_index_spgist and thereby also create_index.) Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/385d396b807bdd7034ad3d0cea3c921d7cb04faa - Speed up sort-order-comparison tests in create_index_spgist. This test script verifies that KNN searches of an SP-GiST index produce the same sort order as a seqscan-and-sort. The FULL JOINs used for that are exceedingly slow, however. Investigation shows that the problem is that the initial join is on the rank() values, and we have a lot of duplicates due to the data set containing 1000 duplicate points. We're therefore going to produce 1000000 join rows that have to be thrown away again by the join filter. We can improve matters by using row_number() instead of rank(), so that the initial join keys are unique. The catch is that that makes the results sensitive to the sorting of rows with equal distances from the reference point. That doesn't matter for the actually-equal points, but as luck would have it, the data set also contains two distinct points that have identical distances to the origin. So those two rows could legitimately appear in either order, causing unwanted output from the check queries. However, it doesn't seem like it's the job of this test to check whether the <-> operator correctly computes distances; its charter is just to verify that SP-GiST emits the values in distance order. So we can dodge the indeterminacy problem by having the check only compare row numbers and distances not the actual point values. This change reduces the run time of create_index_spgist by a good three-quarters, on my machine, with ensuing beneficial effects on the runtime of create_index (thanks to interactions with CREATE INDEX CONCURRENTLY tests in the latter). I see a net improvement of more than 2X in the runtime of their parallel test group. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/5874c7055702e1cf5e58543f11dfcff6de2cc260 - Re-order some regression test scripts for more parallelism. Move the strings, numerology, insert, insert_conflict, select and errors tests to be parts of nearby parallel groups, instead of executing by themselves. (Moving "select" required adjusting the constraints test, which uses a table named "tmp" as select also does. There don't seem to be any other conflicts.) Move psql and stats_ext to the next parallel group, where the rules test also has a long runtime. To make it safe to run stats_ext in parallel with rules, I adjusted the latter to only dump views/rules from the pg_catalog and public schemas, which was what it was doing anyway. stats_ext makes some views in a transient schema, which now will not affect rules. Reorder serial_schedule to match parallel_schedule. Discussion: https://postgr.es/m/735.1554935715@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/798070ec058fe75757587c9e9916cc35c623f427 - Prevent memory leaks associated with relcache rd_partcheck structures. The original coding of generate_partition_qual() just copied the list of predicate expressions into the global CacheMemoryContext, making it effectively impossible to clean up when the owning relcache entry is destroyed --- the relevant code in RelationDestroyRelation() only managed to free the topmost List header :-(. This resulted in a session-lifespan memory leak whenever a table partition's relcache entry is rebuilt. Fortunately, that's not normally a large data structure, and rebuilds shouldn't occur all that often in production situations; but this is still a bug worth fixing back to v10 where the code was introduced. To fix, put the cached expression tree into its own small memory context, as we do with other complicated substructures of relcache entries. Also, deal more honestly with the case that a partition has an empty partcheck list; while that probably isn't a case that's very interesting for production use, it's legal. In passing, clarify comments about how partitioning-related relcache data structures are managed, and add some Asserts that we're not leaking old copies when we overwrite these data fields. Amit Langote and Tom Lane Discussion: https://postgr.es/m/7961.1552498252@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/5f1433ac5e7f943b29ef01266b6b8fc915e6b917 Michaël Paquier pushed: - Add more tests for partition tuple routing with dropped attributes. As bug #15733 has proved, we are lacking coverage for partition tuple routing with dropped attributes when involving three levels of partitioning or more. There was only an active bug in this area for v11, and HEAD is proving to handle those scenarios fine, still it lacked some coverage for the previous problem. Author: Amit Langote, Michael Paquier Discussion: https://postgr.es/m/15733-7692379e310b80ec@postgresql.org https://git.postgresql.org/pg/commitdiff/964bae4d8456e5406753027fa5a70181ddb4c835 - Tweak wording of documentation for pg_checksums. Author: Justin Pryzby Discussion: https://postgr.es/m/20190329143210.GI5815@telsasoft.com https://git.postgresql.org/pg/commitdiff/e3865c37544d77bd4205dd5361592797b97d1e93 - Fix more strcmp() calls using boolean-like comparisons for result checks. Such calls can confuse the reader as strcmp() uses an integer as result. The places patched here have been spotted by Thomas Munro, David Rowley and myself. Author: Michael Paquier Reviewed-by: David Rowley Discussion: https://postgr.es/m/20190411021946.GG2728@paquier.xyz https://git.postgresql.org/pg/commitdiff/d527fda6216780281b90f48820ae978c61c7905c - Switch TAP tests of pg_rewind to use a role with minimal permissions. Up to now the tests of pg_rewind have been using a superuser for all the tests (which is the default of many tests actually, and something that ought to be reviewed) when involving an online source server, still it is possible to use a non-superuser role to do that as long as this role is granted permissions to execute all the source-side functions used for the rewind. This is possible since v11, and was already documented as of bfc8068. This will allow to catch up easily any change in pg_rewind if the tool begins to use more backend-side functions, so as the properties introduced by v11 are kept. Per suggestion from Peter Eisentraut. Author: Michael Paquier Reviewed-by: Magnus Hagander Discussion: https://postgr.es/m/20190411041336.GM2728@paquier.xyz https://git.postgresql.org/pg/commitdiff/d4e2a843e6d6f325c070ee80a0c117ec11675e74 - Fix typos in reloptions.c. Author: Kirk Jamison Discussion: https://postgr.es/m/D09B13F772D2274BB348A310EE3027C6493463@g01jpexmbkw24 https://git.postgresql.org/pg/commitdiff/d87ab88686fb60ad5a34373de05bb20e632cf003 - Revert "Switch TAP tests of pg_rewind to use a role with minimal permissions". This reverts commit d4e2a84, which added a new user with limited permissions to run the TAP tests of pg_rewind. Buildfarm machine members on Windows jacana and bowerbird have been complaining about that, the new role not being able to run the rewind because SSPI is not configured to allow it. Fixing the test requires passing down directly the new user to pg_regress with --create-role so as SSPI can work properly. Reported-by: Andrew Dunstan Discussion: https://postgr.es/m/3cd43d33-f415-cc41-ade3-7230ab15b2c9@2ndQuadrant.com https://git.postgresql.org/pg/commitdiff/db8db624e826efbe16aab1ae921bae071f98f099 - Switch TAP tests of pg_rewind to use non-superuser role, take two. Up to now the tests of pg_rewind have been using a superuser for all its tests (which is the default of many tests actually, and something that ought to be reviewed) when involving an online source server, still it is possible to use a non-superuser role to do that as long as this role is granted permissions to execute all the source-side functions used for the rewind. This is possible since v11, and was already documented as of bfc8068. PostgresNode::init is extended so as callers of this routine can add extra options to configure the authentication of a new node, which gets used by this commit, and allows the tests to work properly on Windows where SSPI is used. This will allow to catch up easily any change in pg_rewind if the tool begins to use more backend-side functions, so as the properties introduced by v11 are kept. Per suggestion from Peter Eisentraut. Author: Michael Paquier Reviewed-by: Magnus Hagander Discussion: https://postgr.es/m/20190411041336.GM2728@paquier.xyz https://git.postgresql.org/pg/commitdiff/d9f543e9e9be15f92abdeaf870e57ef289020191 Andres Freund pushed: - Fix a number of issues around modifying a previously updated row. This commit fixes three, unfortunately related, issues: 1) Since 5db6df0c01, the introduction of DML via tableam, it was possible to trigger "ERROR: unexpected table_lock_tuple status: 1" when updating a row that was previously updated in the same transaction - but only when the previously updated row was before updated in a concurrent transaction (and READ COMMITTED was used). The reason for that was that that case simply wasn't expected. Fixing that lead to: 2) Even before the above commit, there were error checks (introduced in 6868ed7491b7) preventing a row being updated by different commands within the same statement (say in a function called by an UPDATE) - but that check wasn't performed when the row was first updated in a concurrent transaction - instead the second update was silently skipped in that case. After this change we throw the same error as we'd without the concurrent transaction. 3) The error messages (introduced in 6868ed7491b7) preventing such updates emitted the same error message for both DELETE and UPDATE ("tuple to be updated was already modified by an operation triggered by the current command"). While that could be changed separately, it made it hard to write tests that verify the correct correct behavior of the code. This commit changes heap's implementation of table_lock_tuple() to return TM_SelfModified instead of TM_Invisible (previously loosely modeled after EvalPlanQualFetch), and teaches nodeModifyTable.c to handle that in response to table_lock_tuple() and not just in response to table_(delete|update). Additionally it fixes the wrong error message (see 3 above). The comment for table_lock_tuple() is also adjusted to state that TM_Deleted won't return information in TM_FailureData - it'll not always be available. This also adds tests to ensure that DELETE/UPDATE correctly error out when affecting a row that concurrently was modified by another transaction. Author: Andres Freund Reported-By: Tom Lane, when investigating a bug bug fix to another bug by Amit Langote Discussion: https://postgr.es/m/19321.1554567786@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/41f5e04aec6cf63ba8392adf70e9289e9c3706d6 - Reset memory context once per tuple in validateForeignKeyConstraint. When using tableam ExecFetchSlotHeapTuple() might return a separately allocated tuple. We could use the shouldFree argument to explicitly free it, but it seems more robust to to protect Also add a CHECK_FOR_INTERRUPTS() after each tuple. It's likely that each AM has (heap does) a CFI somewhere in the relevant path, but it seems more robust to have one in validateForeignKeyConstraint() itself. Note that this only affects the cases that couldn't be optimized to be verified with a query. Author: Andres Freund Reviewed-By: Tom Lane (in an earlier version) Discussion: https://postgr.es/m/19030.1554574075@sss.pgh.pa.us https://postgr.es/m/CAKJS1f_SHKcPYMsi39An5aUjhAcEMZb6Cx1Sj1QWEWSiKJkBVQ@mail.gmail.com https://postgr.es/m/20180711185628.mrvl46bjgk2uxoki@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/4c9e1bd0a37e7b79dfc797dd91627336e871c1b0 - tableam: comment and formatting fixes. Author: Heikki Linnakangas Discussion: https://postgr.es/m/9a7fb9cc-2419-5db7-8840-ddc10c93f122@iki.fi https://git.postgresql.org/pg/commitdiff/6421011ea24db3a59dc7a0058a03e91981d53635 Fujii Masao pushed: - Add vacuum_truncate reloption. vacuum_truncate controls whether vacuum tries to truncate off any empty pages at the end of the table. Previously vacuum always tried to do the truncation. However, the truncation could cause some problems; for example, ACCESS EXCLUSIVE lock needs to be taken on the table during the truncation and can cause the query cancellation on the standby even if hot_standby_feedback is true. Setting this reloption to false can be helpful to avoid such problems. Author: Tsunakawa Takayuki Reviewed-By: Julien Rouhaud, Masahiko Sawada, Michael Paquier, Kirk Jamison and Fujii Masao Discussion: https://postgr.es/m/CAHGQGwE5UqFqSq1=kV3QtTUtXphTdyHA-8rAj4A=Y+e4kyp3BQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/119dcfad988d5b5d9f52b256087869997670aa36 Peter Eisentraut pushed: - doc: Add note about generated columns in foreign tables. Explain that it is not enforced that querying a generated column returns data that is consistent with the data that was stored. This is similar to the note about constraints nearby. Reported-by: Amit Langote <amitlangote09@gmail.com> https://git.postgresql.org/pg/commitdiff/348f57ce5be96190491e2153abb47060884f8ebf - doc: Update serial explanation. The CREATE SEQUENCE command should include a data type specification, since PostgreSQL 10. Reported-by: mjf@pearson.co.uk https://git.postgresql.org/pg/commitdiff/c66000385fe1dc94a6d5525dcd192f17b551fb9b - doc: Fix whitespace. Author: Julien Rouhaud <rjuju123@gmail.com> https://git.postgresql.org/pg/commitdiff/122fa9f942478f8fdcfba961e01d172574369293 - Replace tabs with spaces in one .sql file. Let's at least keep this consistent within the same file. https://git.postgresql.org/pg/commitdiff/9efe068e48f09d915660576f5672bfa9f1c0eb53 - pg_restore: Make not verbose by default. This was accidentally changed in cc8d41511721d25d557fc02a46c053c0a602fed0. Reported-by: Christoph Berg <myon@debian.org> https://git.postgresql.org/pg/commitdiff/765525c8c2c6e55abe8c0cd43bf5c728926d76d4 - Fix REINDEX CONCURRENTLY of partitions. In case of a partition index, when swapping the old and new index, we also need to attach the new index as a partition and detach the old one. Also, to handle partition indexes, we not only need to change dependencies referencing the index, but also dependencies of the index referencing something else. The previous code did this only specifically for a constraint, but we also need to do this for partitioned indexes. So instead write a generic function that does it for all dependencies. Author: Michael Paquier <michael@paquier.xyz> Author: Peter Eisentraut <peter.eisentraut@2ndquadrant.com> Discussion: https://www.postgresql.org/message-id/flat/DF4PR8401MB11964EDB77C860078C343BEBEE5A0%40DF4PR8401MB1196.NAMPRD84.PROD.OUTLOOK.COM#154df1fedb735190a773481765f7b874 https://git.postgresql.org/pg/commitdiff/ef6f30fe77af69a8c775cca82bf993b10c9889ee Noah Misch pushed: - Avoid "could not reattach" by providing space for concurrent allocation. We've long had reports of intermittent "could not reattach to shared memory" errors on Windows. Buildfarm member dory fails that way when PGSharedMemoryReAttach() execution overlaps with creation of a thread for the process's "default thread pool". Fix that by providing a second region to receive asynchronous allocations that would otherwise intrude into UsedShmemSegAddr. In pgwin32_ReserveSharedMemoryRegion(), stop trying to free reservations landing at incorrect addresses; the caller's next step has been to terminate the affected process. Back-patch to 9.4 (all supported versions). Reviewed by Tom Lane. He also did much of the prerequisite research; see commit bcbf2346d69f6006f126044864dd9383d50d87b4. Discussion: https://postgr.es/m/20190402135442.GA1173872@rfd.leadboat.com https://git.postgresql.org/pg/commitdiff/617dc6d299c957e2784320382b3277ede01d9c63 - Define WIN32_STACK_RLIMIT throughout win32 and cygwin builds. The MSVC build system already did this, and commit 617dc6d299c957e2784320382b3277ede01d9c63 used it in a second file. Back-patch to 9.4, like that commit. Discussion: https://postgr.es/m/CAA8=A7_1SWc3+3Z=-utQrQFOtrj_DeohRVt7diA2tZozxsyUOQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/ba3fb5d4fb9227719759740b6d7771e719c3318b - Consistently test for in-use shared memory. postmaster startup scrutinizes any shared memory segment recorded in postmaster.pid, exiting if that segment matches the current data directory and has an attached process. When the postmaster.pid file was missing, a starting postmaster used weaker checks. Change to use the same checks in both scenarios. This increases the chance of a startup failure, in lieu of data corruption, if the DBA does "kill -9 `head -n1 postmaster.pid` && rm postmaster.pid && pg_ctl -w start". A postmaster will no longer stop if shmat() of an old segment fails with EACCES. A postmaster will no longer recycle segments pertaining to other data directories. That's good for production, but it's bad for integration tests that crash a postmaster and immediately delete its data directory. Such a test now leaks a segment indefinitely. No "make check-world" test does that. win32_shmem.c already avoided all these problems. In 9.6 and later, enhance PostgresNode to facilitate testing. Back-patch to 9.4 (all supported versions). Reviewed (in earlier versions) by Daniel Gustafsson and Kyotaro HORIGUCHI. Discussion: https://postgr.es/m/20190408064141.GA2016666@rfd.leadboat.com https://git.postgresql.org/pg/commitdiff/c098509927f9a49ebceb301a2cb6a477ecd4ac3c - When Perl "kill(9, ...)" fails, try "pg_ctl kill". Per buildfarm member jacana, the former fails under msys Perl 5.8.8. Back-patch to 9.6, like the code in question. Discussion: https://postgr.es/m/GrdLgAdUK9FdyZg8VIcTDKVOkys122ZINEb3CjjoySfGj2KyPiMKTh1zqtRp0TAD7FJ27G-OBB3eplxIB5GhcQH5o8zzGZfp0MuJaXJxVxk=@yesql.se https://git.postgresql.org/pg/commitdiff/947a35014fdc2ec74cbf06c7dbac6eea6fae90c6 - MSYS: Translate REGRESS_SHLIB to a Windows file name. Per buildfarm member jacana. Back-patch to v11; earlier branches skip the affected test under msys. Discussion: https://postgr.es/m/GrdLgAdUK9FdyZg8VIcTDKVOkys122ZINEb3CjjoySfGj2KyPiMKTh1zqtRp0TAD7FJ27G-OBB3eplxIB5GhcQH5o8zzGZfp0MuJaXJxVxk=@yesql.se https://git.postgresql.org/pg/commitdiff/9daefff1226087602d25837b6b30154b3a916ea8 Heikki Linnakangas pushed: - Fix example in comment. Author: Adrien Nayrat https://git.postgresql.org/pg/commitdiff/16954e22e2a881a4a5ead82f364635bfa63cc9df Álvaro Herrera pushed: - Fix memory leak in pgbench. Commit 25ee70511ec2 introduced a memory leak in pgbench: some PGresult structs were not being freed during error bailout, because we're now doing more PQgetResult() calls than previously. Since there's more cleanup code outside the discard_response() routine than in it, refactor the cleanup code, removing the routine. This has little effect currently, since we abandon processing after hitting errors, but if we ever get further pgbench features (such as testing for serializable transactions), it'll matter. Per Coverity. Reviewed-by: Michaël Paquier https://git.postgresql.org/pg/commitdiff/fe0e0b4fc7f0cdc2333bda08b199422a1e77a551 - Fix typo. https://git.postgresql.org/pg/commitdiff/4dba0f6dc407f965924d6c1007ac1bb5cc209dde - Fix declaration after statement. This style is frowned upon. I inadvertently introduced one in commit fe0e0b4fc7f0. (My compiler does not complain about it, even though -Wdeclaration-after-statement is specified. Weird.) Author: Masahiko Sawada https://git.postgresql.org/pg/commitdiff/65d857d92c418d732e3531a3761a32f2e352cb35 Thomas Munro pushed: - Fix typos. https://git.postgresql.org/pg/commitdiff/255044889d419354b46a2bf8907b83507d695af5 - Improve comment in sync.h. Per off-list complaint from Andres Freund. https://git.postgresql.org/pg/commitdiff/d614aae02e8f878438716b7fd2642b8240b3f2b3 - Fix GetNewTransactionId()'s interaction with xidVacLimit. Commit ad308058 switched to returning a FullTransactionId, but failed to load the potentially updated value in the case where xidVacLimit is reached and we release and reacquire the lock. Repair, closing bug #15727. While reviewing that commit, also fix the size computation used by EstimateTransactionStateSize() and switch to the mul_size() macro traditionally used in such expressions. Author: Thomas Munro Reported-by: Roman Zharkov Discussion: https://postgr.es/m/15727-0be246e7d852d229%40postgresql.org https://git.postgresql.org/pg/commitdiff/f7feb020c3d8d5aff24204af28359b99ee65bf8f Amit Kapila pushed: - Avoid counting transaction stats for parallel worker cooperating.transaction. The transaction that is initiated by the parallel worker to cooperate with the actual transaction started by the main backend to complete the query execution should not be counted as a separate transaction. The other internal transactions started and committed by the parallel worker are still counted as separate transactions as we that is what we do in other places like autovacuum. This will partially fix the bloat in transaction stats due to additional transactions performed by parallel workers. For a complete fix, we need to decide how we want to show all the transactions that are started internally for various operations and that is a matter of separate patch. Reported-by: Haribabu Kommi Author: Haribabu Kommi Reviewed-by: Amit Kapila, Jamison Kirk and Rahila Syed Backpatch-through: 9.6 Discussion: https://postgr.es/m/CAJrrPGc9=jKXuScvNyQ+VNhO0FZk7LLAShAJRyZjnedd2D61EQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/bdf35744bdf70208fc4d0f4b76f7d4bce3cf326b Bruce Momjian pushed: - doc: adjust libpq wording to be neither/nor. Reported-by: postgresql@cohi.at Discussion: https://postgr.es/m/155419437926.737.10876947446993402227@wrigleys.postgresql.org Backpatch-through: 9.4 https://git.postgresql.org/pg/commitdiff/d4888a3f766a7c02a9330c658c76c60e1ab8cc3e Michael Meskes pushed: - Fix off-by-one check that can lead to a memory overflow in ecpg. Patch by Liu Huailing <liuhuailing@cn.fujitsu.com> https://git.postgresql.org/pg/commitdiff/ed16ba3248d486ec8b2fdfc75e442620d675b617 Magnus Hagander pushed: - Show shared object statistics in pg_stat_database. This adds a row to the pg_stat_database view with datoid 0 and datname NULL for those objects that are not in a database. This was added particularly for checksums, but we were already tracking more satistics for these objects, just not returning it. Also add a checksum_last_failure column that holds the timestamptz of the last checksum failure that occurred in a database (or in a non-dataabase file), if any. Author: Julien Rouhaud <rjuju123@gmail.com> https://git.postgresql.org/pg/commitdiff/77bd49adba4711b4497e7e39a5ec3a9812cbd52a == Pending Patches == Justin Pryzby sent in another revision of a patch to clean up the docs for log_statement_sample_rate. Jie Zhang sent in a patch to update zh_CN.po. Fabien COELHO sent in a patch for pgbench to add an option to show the actual builtin script code. Fabien COELHO sent in a patch for pgbench to implement a strict TPC-B-conformant benchmark. Fabien COELHO sent in a patch for pgbench to add \aset, which stores all result sets. Yuzuko Hosoya and Kyotaro HORIGUCHI traded patches to fix an issue with inconsistent partition pruning. Heikki Linnakangas sent in three more revisions of a patch to fix confusion about the different kinds of slots in IndexOnlyScans, add a toy table AM implementation to play with, add a test for bug with index reorder queue slot type confusion, and fix confusion about the type of slot used for Index Scan's projections. Haribabu Kommi sent in another revision of a patch to MSVC Build support with Visual Studio 2019. Iwata Aya sent in another revision of a patch to make one-line libpq trace output for logging. Konstantin Knizhnik sent in another revision of a patch to implement autoprepare. Amit Langote and Jesper Pedersen traded patches to fix an infelicity between hash partitons and UPDATE. Anastasia Lubennikova and Andrey V. Lepikhov traded patches to fix a gist_optimized_wal_intarray_test bug. Robert Treat sent in a patch to fix a mistake in the documentation around target column type restrictions in logical replication. Amit Khandekar sent in another revision of a patch to implement minimal logical decoding on standbys. Ashwin Agrawal sent in a patch to implement compressed in-core columnar storage. Álvaro Herrera and David Rowley traded patches to fix an infelicity between pg_dump and partition tablespaces. Peifeng Qiu sent in another revision of a patch to speed up builds on Windows by generating symbol definitions batch-wise. Etsuro Fujita sent in another revision of a patch to fix a bug in tuple-routing for partitions some of which are foreign tables. Fabien COELHO sent in another revision of a patch for pgbench to add minimal stats on initialization. Justin Pryzby sent in another revision of a patch to clean up, remove, and update references to OID column. Konstantin Knizhnik sent in a patch to implement a ptrack utility, which detects updated blocks at the file level. This would be infrastructure for, among other things, a future incremental backup. Thomas Munro sent in a PoC patch to implement ExecScrollSlot() for hash join prefetch. Haribabu Kommi sent in another revision of a patch to libpq to support connecting to a standby server as a higher priority. Takeshi Ideriha sent in another revision of a patch to prevent syscache from bloating with negative cache entries. David Rowley and Justin Pryzby traded patches to reinstate the warnings regarding large partition heirarchies, and document some features of declarative partitioning for which there are no plans to port to legacy inheritance. Jeevan Chalke and Peter Eisentraut traded patches to fix an index optimization for LIKE on bytea. Antonin Houska sent in a patch to DRY up reading of XLOG pages. David Rowley sent in a patch to warn about the consequences of calling pg_stat_reset(). Juan José Santamaría Flecha to ensure that the TM format for to_char() respects encoding. Fabien COELHO sent in a patch to implement a new SHOW_ALL_RESULTS option for psql, which shows all results of a combined query (\;) instead of only the last one. Laurenz Albe sent in a patch to ensure that identity columns own only a single sequence.
pgsql-announce by date: