== PostgreSQL Weekly News - September 6, 2020 == - Mailing list pgsql-announce
| From | David Fetter |
|---|---|
| Subject | == PostgreSQL Weekly News - September 6, 2020 == |
| Date | |
| Msg-id | 20200906220034.GA9837@fetter.org Whole thread Raw |
| List | pgsql-announce |
== PostgreSQL Weekly News - September 6, 2020 == Person of the week: https://postgresql.life/post/marco_slot/ == PostgreSQL Product News == pgBackRest 2.29, a backup and restore system for PostgreSQL, released. https://pgbackrest.org/release.html#2.29 pgFormatter 4.4, a formatter/beautifier for SQL code, released. http://sqlformat.darold.net/ pgbitmap 0.91, a space-optimised, non-sparse, bitmap type, released. https://pgxn.org/dist/pgbitmap/ == PostgreSQL Jobs for September == http://archives.postgresql.org/pgsql-jobs/2020-09/ == PostgreSQL Local == pgDay Israel 2020 will take place on September 10, 2020 in Tel Aviv. http://pgday.org.il/ == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm PST8PDT to david@fetter.org. == Applied Patches == Tom Lane pushed: - Mark factorial operator, and postfix operators in general, as deprecated. Per discussion, we're planning to remove parser support for postfix operators in order to simplify the grammar. So it behooves us to put out a deprecation notice at least one release before that. There is only one built-in postfix operator, ! for factorial. Label it deprecated in the docs and in pg_description, and adjust some examples that formerly relied on it. (The sister prefix operator !! is also deprecated. We don't really have to remove that one, but since we're suggesting that people use factorial() instead, it seems better to remove both operators.) Also state in the CREATE OPERATOR ref page that postfix operators in general are going away. Although this changes the initial contents of pg_description, I did not force a catversion bump; it doesn't seem essential. In v13, also back-patch 4c5cf5431, so that there's someplace for the <link>s to point to. Mark Dilger and John Naylor, with some adjustments by me Discussion: https://postgr.es/m/BE2DF53D-251A-4E26-972F-930E523580E9@enterprisedb.com https://git.postgresql.org/pg/commitdiff/6ca547cf75ef6e922476c51a3fb5e253eef5f1b6 - Teach libpq to handle arbitrary-length lines in .pgpass files. Historically there's been a hard-wired assumption here that no line of a .pgpass file could be as long as NAMEDATALEN*5 bytes. That's a bit shaky to start off with, because (a) there's no reason to suppose that host names fit in NAMEDATALEN, and (b) this figure fails to allow for backslash escape characters. However, it fails completely if someone wants to use a very long password, and we're now hearing reports of people wanting to use "security tokens" that can run up to several hundred bytes. Another angle is that the file is specified to allow comment lines, but there's no reason to assume that long comment lines aren't possible. Rather than guessing at what might be a more suitable limit, let's replace the fixed-size buffer with an expansible PQExpBuffer. That adds one malloc/free cycle to the typical use-case, but that's surely pretty cheap relative to the I/O this code has to do. Also, add TAP test cases to exercise this code, because there was no test coverage before. This reverts most of commit 2eb3bc588, as there's no longer a need for a warning message about overlength .pgpass lines. (I kept the explicit check for comment lines, though.) In HEAD and v13, this also fixes an oversight in 74a308cf5: there's not much point in explicit_bzero'ing the line buffer if we only do so in two of the three exit paths. Back-patch to all supported branches, except that the test case only goes back to v10 where src/test/authentication/ was added. Discussion: https://postgr.es/m/4187382.1598909041@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/b55b4dad99e99d5306744a4e8ef8021fa3a922e4 - Set cutoff xmin more aggressively when vacuuming a temporary table. Since other sessions aren't allowed to look into a temporary table of our own session, we do not need to worry about the global xmin horizon when setting the vacuum XID cutoff. Indeed, if we're not inside a transaction block, we may set oldestXmin to be the next XID, because there cannot be any in-doubt tuples in a temp table, nor any tuples that are dead but still visible to some snapshot of our transaction. (VACUUM, of course, is never inside a transaction block; but we need to test that because CLUSTER shares the same code.) This approach allows us to always clean out a temp table completely during VACUUM, independently of concurrent activity. Aside from being useful in its own right, that simplifies building reproducible test cases. Discussion: https://postgr.es/m/3490536.1598629609@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/a7212be8b9e0885ee769e8c55f99ef742cda487b - Improve test coverage of ginvacuum.c. Add a test case that exercises vacuum's deletion of empty GIN posting pages. Since this is a temp table, it should now work reliably to delete a bunch of rows and immediately VACUUM. Before the preceding commit, this would not have had the desired effect, at least not in parallel regression tests. Discussion: https://postgr.es/m/3490536.1598629609@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/4c51a2d1e4b750bc11b8de9a85b079a14f798741 - Add string_to_table() function. This splits a string at occurrences of a delimiter. It is exactly like string_to_array() except for producing a set of values instead of an array of values. Thus, the relationship of these two functions is the same as between regexp_split_to_table() and regexp_split_to_array(). Although the same results could be had from unnest(string_to_array()), this is somewhat faster than that, and anyway it seems reasonable to have it for symmetry with the regexp functions. Pavel Stehule, reviewed by Peter Smith Discussion: https://postgr.es/m/CAFj8pRD8HOpjq2TqeTBhSo_QkzjLOhXzGCpKJ4nCs7Y9SQkuPw@mail.gmail.com https://git.postgresql.org/pg/commitdiff/66f163068030b5c5fe792a0daee27822dac43791 - Doc: mention packager-supplied tools for server start/stop, initdb, etc. The majority of our audience is probably using a pre-packaged Postgres build rather than raw sources. For them, much of runtime.sgml is not too relevant, and they should be reading the packager's docs instead. Add some notes pointing that way in appropriate places. Text by me; thanks to Daniel Gustafsson for review and discussion, and to Laurenz Albe for an earlier version. Discussion: https://postgr.es/m/159430831443.16535.11360317280100947016@wrigleys.postgresql.org https://git.postgresql.org/pg/commitdiff/d2511d71328cb30b0d6b3f43af4809df83c4f0cc - Allow records to span multiple lines in pg_hba.conf and pg_ident.conf. A backslash at the end of a line now causes the next line to be appended to the current one (effectively, the backslash and newline are discarded). This allows long HBA entries to be created without legibility problems. While we're here, get rid of the former hard-wired length limit on pg_hba.conf lines, by using an expansible StringInfo buffer instead of a fixed-size local variable. Since the same code is used to read the ident map file, these changes apply there as well. Fabien Coelho, reviewed by Justin Pryzby and David Zhang Discussion: https://postgr.es/m/alpine.DEB.2.21.2003251906140.15243@pseudo https://git.postgresql.org/pg/commitdiff/8f8154a503c71a18ad72c64f4aefb9d847c45b86 - Avoid lockup of a parallel worker when reporting a long error message. Because sigsetjmp() will restore the initial state with signals blocked, the code path in bgworker.c for reporting an error and exiting would execute that way. Usually this is fairly harmless; but if a parallel worker had an error message exceeding the shared-memory communication buffer size (16K) it would lock up, because it would wait for a resume-sending signal from its parallel leader which it would never detect. To fix, just unblock signals at the appropriate point. This can be shown to fail back to 9.6. The lack of parallel query infrastructure makes it difficult to provide a simple test case for 9.5; but I'm pretty sure the issue exists in some form there as well, so apply the code change there too. Vignesh C, reviewed by Bharath Rupireddy, Robert Haas, and myself Discussion: https://postgr.es/m/CALDaNm1d1hHPZUg3xU4XjtWBOLCrA+-2cJcLpw-cePZ=GgDVfA@mail.gmail.com https://git.postgresql.org/pg/commitdiff/be4b0c0077e6a1f7be0965f8d93696e0e0eadb52 - Remove arbitrary restrictions on password length. This patch started out with the goal of harmonizing various arbitrary limits on password length, but after awhile a better idea emerged: let's just get rid of those fixed limits. recv_password_packet() has an arbitrary limit on the packet size, which we don't really need, so just drop it. (Note that this doesn't really affect anything for MD5 or SCRAM password verification, since those will hash the user's password to something shorter anyway. It does matter for auth methods that require a cleartext password.) Likewise remove the arbitrary error condition in pg_saslprep(). The remaining limits are mostly in client-side code that prompts for passwords. To improve those, refactor simple_prompt() so that it allocates its own result buffer that can be made as big as necessary. Actually, it proves best to make a separate routine pg_get_line() that has essentially the semantics of fgets(), except that it allocates a suitable result buffer and hence will never return a truncated line. (pg_get_line has a lot of potential applications to replace randomly-sized fgets buffers elsewhere, but I'll leave that for another patch.) I built pg_get_line() atop stringinfo.c, which requires moving that code to src/common/; but that seems fine since it was a poor fit for src/port/ anyway. This patch is mostly mine, but it owes a good deal to Nathan Bossart who pressed for a solution to the password length problem and created a predecessor patch. Also thanks to Peter Eisentraut and Stephen Frost for ideas and discussion. Discussion: https://postgr.es/m/09512C4F-8CB9-4021-B455-EF4C4F0D55A0@amazon.com https://git.postgresql.org/pg/commitdiff/67a472d71c98c3d2fa322a1b4013080b20720b98 - Fix rare deadlock failure in create_am regression test. The "DROP ACCESS METHOD gist2" test will require locking the index to be dropped and then its table; while most ordinary operations lock a table first then its index. While no concurrent test scripts should be touching fast_emp4000, autovacuum might chance to be processing that table when the DROP runs, resulting in a deadlock failure. This is pretty rare but we see it in the buildfarm from time to time. To fix, acquire a lock on fast_emp4000 before issuing the DROP. Since the point of the exercise is mostly to prevent buildfarm failures, back-patch to 9.6 where this test was introduced. Discussion: https://postgr.es/m/839004.1599185607@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/d54f99e41541de848a6ca53b3ec060f461e9ab71 - Remove some more useless assignments. Found with clang's scan-build tool. It also whines about a lot of other dead stores that we should *not* change IMO, either as a matter of style or future-proofing. But these places seem like clear oversights. Discussion: https://postgr.es/m/CAEudQAo1+AcGppxDSg8k+zF4+Kv+eJyqzEDdbpDg58-=MQcerQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/38a2d703298c9a891dc9c24c0c087f417f555c70 - Fix over-eager ping'ing in logical replication receiver. Commit 3f60f690f only partially fixed the broken-status-tracking issue in LogicalRepApplyLoop: we need ping_sent to have the same lifetime as last_recv_timestamp. The effects are much less serious than what that commit fixed, though. AFAICS this would just lead to extra ping requests being sent, once per second until the sender responds. Still, it's a bug, so backpatch to v10 as before. Discussion: https://postgr.es/m/959627.1599248476@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/c8746f999ea2feba22832b485fba37b5301b54a3 - Remove still more useless assignments. Fix some more things scan-build pointed to as dead stores. In some of these cases, rearranging the code a little leads to more readable code IMO. It's all cosmetic, though. Discussion: https://postgr.es/m/CAEudQAo1+AcGppxDSg8k+zF4+Kv+eJyqzEDdbpDg58-=MQcerQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/9a851039aac6fc7b78a80e60699e7bd8924b6f12 - Make new authentication test case more robust. I happened to notice that the new test case I added in b55b4dad9 falls over if one runs "make check" repeatedly; though not in branches after v10. That's because it was assuming that tmp_check/pgpass wouldn't exist already. However, it's only been since v11 that the Makefiles forcibly remove all of tmp_check/ before starting a TAP run. This fix to unlink the file is therefore strictly necessary only in v10 ... but it seems wisest to do it across the board, rather than let the test rely on external logic to get the conditions right. https://git.postgresql.org/pg/commitdiff/4d41823c5267184fb021ac456caf95ef734cc8b0 - Yet more elimination of dead stores and useless initializations. I'm not sure what tool Ranier was using, but the ones I contributed were found by using a newer version of scan-build than I tried before. Ranier Vilela and Tom Lane Discussion: https://postgr.es/m/CAEudQAo1+AcGppxDSg8k+zF4+Kv+eJyqzEDdbpDg58-=MQcerQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/a5cc4dab6d1d694f113912a2aca7012a95262f0b - Improve some ancient, crufty code in bootstrap + initdb. At some point back in the last century, somebody felt that reading all of pg_type twice was cheaper, or at least easier, than using repalloc() to resize the Typ[] array dynamically. That seems like an entirely wacko proposition, so rewrite the code to do it the other way. (To add insult to injury, there were two not-quite-identical copies of said code.) initdb.c's readfile() function had the same disease of preferring to do double the I/O to avoid resizing its output array. Here, we can make things easier by using the just-invented pg_get_line() function to handle reading individual lines without a predetermined notion of how long they are. On my machine, it's difficult to detect any net change in the overall runtime of initdb from these changes; but they should help on slower buildfarm machines (especially since a buildfarm cycle involves a lot of initdb's these days). My attention was drawn to these places by scan-build complaints, but on inspection they needed a lot more work than just suppressing dead stores :-( https://git.postgresql.org/pg/commitdiff/e0f05cd5ba76a75e2ce3b85ba050e48e857dca00 - Remove useless lstat() call in pg_rewind. This is duplicative of an lstat that was just done by the calling function (traverse_datadir), besides which we weren't really doing anything with the results. There's not much point in checking to see if someone removed the file since the previous lstat, since the FILE_ACTION_REMOVE code would have to deal with missing-file cases anyway. Moreover, the "exists = false" assignment was a dead store; nothing was done with that value later. A syscall saved is a syscall earned, so back-patch to 9.5 where this code was introduced. Discussion: https://postgr.es/m/1221796.1599329320@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/666e9a90f09a1ab89959774273597764e6e0f9ad - Fix misleading error message about inconsistent moving-aggregate types. We reported the wrong types when complaining that an aggregate's moving-aggregate implementation is inconsistent with its regular implementation. This was wrong since the feature was introduced, so back-patch to all supported branches. Jeff Janes Discussion: https://postgr.es/m/CAMkU=1x808LH=LPhZp9mNSP0Xd1xDqEd+XeGcvEe48dfE6xV=A@mail.gmail.com https://git.postgresql.org/pg/commitdiff/19ad7e1d7b8b54ea3aa405e60057ab4baefcedbf Magnus Hagander pushed: - Fix docs bug stating file_fdw requires absolute paths. It has always (since the first commit) worked with relative paths, so use the same wording as other parts of the documentation. Author: Bruce Momjian Discussion: https://postgr.es/m/CABUevExx-hm=cit+A9LeKBH39srvk8Y2tEZeEAj5mP8YfzNKUg@mail.gmail.com https://git.postgresql.org/pg/commitdiff/3a788db60108fed2e51f62a79a5f425401300338 Bruce Momjian pushed: - doc: clarify the useful features of procedures. This was not clearly documented when procedures were added in PG 11. Reported-by: Robin Abbi Discussion: https://postgr.es/m/CAGmg_NX327KKVuJmbWZD=pGutYFxzZjX1rU+3ji8UuX=8ONn9Q@mail.gmail.com Backpatch-through: 11 https://git.postgresql.org/pg/commitdiff/243a3b92a67519d4da1712b4fbad194bded3afb5 - docs: improve 'capitals' inheritance example. Adds constraints and improves wording. Reported-by: 2552891@gmail.com Discussion: https://postgr.es/m/159586122762.680.1361378513036616007@wrigleys.postgresql.org Backpatch-through: 9.5 https://git.postgresql.org/pg/commitdiff/47c427d006267d752fee4655543ec99dabe9e61d - doc: improve description of subscripting of arrays. It wasn't clear the non-integers are cast to integers for subscripting, rather than throwing an error. Reported-by: sean@materialize.io Discussion: https://postgr.es/m/159538675800.624.7728794628229799531@wrigleys.postgresql.org Backpatch-through: 9.5 https://git.postgresql.org/pg/commitdiff/06eba0fd10b1c19d578b90f6ab792834fe9a7418 - C comment: remove mention of use of t_hoff WAL structure member. Reported-by: Antonin Houska Discussion: https://postgr.es/m/21643.1595353537@antos Backpatch-through: 9.5 https://git.postgresql.org/pg/commitdiff/1e0512ff23e600e9bc19e7f1a1c5ce0597c7bd47 - docs: replace "stable storage" with "durable" in descriptions. For PG, "durable storage" has a clear meaning, while "stable storage" does not, so use the former. Discussion: https://postgr.es/m/20200817165222.GA31806@momjian.us Backpatch-through: 9.5 https://git.postgresql.org/pg/commitdiff/70e791f47e976a810254d52d38fbc33acdc8705f - docs: clarify intermediate certificate creation instructions. Specifically, explain the v3_ca openssl specification. Discussion: https://postgr.es/m/20200824175653.GA32411@momjian.us Backpatch-through: 9.5 https://git.postgresql.org/pg/commitdiff/b1ae70b3b4fd54220a0901eaf3dd4c5ca0827108 - doc: cross-link file-fdw and CSV config log sections. There is an file-fdw example that reads the server config file, so cross link them. Reported-by: Oleg Samoilov Discussion: https://postgr.es/m/159800192078.2886.10431506404995508950@wrigleys.postgresql.org Backpatch-through: 9.5 https://git.postgresql.org/pg/commitdiff/de2d1920ddf54ec8dbcc87821ade5346f0cf8926 - docs: in mapping SQL to C data types, timestamp isn't a pointer. It is an int64. Reported-by: ajulien@shaktiware.fr Discussion: https://postgr.es/m/159845038271.24995.15682121015698255155@wrigleys.postgresql.org Backpatch-through: 9.5 https://git.postgresql.org/pg/commitdiff/9524fa1aa557b19c9f049b3d79dc93e53cb9d57b - pg_upgrade doc: mention saving postgresql.conf.auto files. Also mention files included by postgresql.conf. Reported-by: Álvaro Herrera Discussion: https://postgr.es/m/08AD4526-75AB-457B-B2DD-099663F28040@yesql.se Backpatch-through: 9.5 https://git.postgresql.org/pg/commitdiff/50ed605b3e311f26497cea979887a8deed3eabf6 - doc: add commas after 'i.e.' and 'e.g.'. This follows the American format, https://jakubmarian.com/comma-after-i-e-and-e-g/. There is no intention of requiring this format for future text, but making existing text consistent every few years makes sense. Discussion: https://postgr.es/m/20200825183619.GA22369@momjian.us Backpatch-through: 9.5 https://git.postgresql.org/pg/commitdiff/953c64e0f67d44b1db97a3d33b329ccd4691f4c6 - doc: document how the backup manifest is transferred. Reported-by: Bernd Helmle Discussion: https://postgr.es/m/31acf8b0f1f701d53245e0cae38abdf5c3a0d559.camel@oopsware.de Backpatch-through: 13 https://git.postgresql.org/pg/commitdiff/ab3c6d41552411ea2fe4904ec8294951c52c113d - doc: clarify that max_wal_size is "during" checkpoints. Previous wording was "between". Reported-by: Pavel Luzanov Discussion: https://postgr.es/m/26906a54-d7cb-2f8e-eed7-e31660024694@postgrespro.ru Backpatch-through: 9.5 https://git.postgresql.org/pg/commitdiff/db864c3c36035e4620afd114c783af7d777d78b0 - remove redundant initializations. Reported-by: Ranier Vilela Discussion: https://postgr.es/m/CAEudQAo1+AcGppxDSg8k+zF4+Kv+eJyqzEDdbpDg58-=MQcerQ@mail.gmail.com Author: Ranier Vilela Backpatch-through: master https://git.postgresql.org/pg/commitdiff/e36e936e0ee664e07588732d1a8d105c5ec8e57d - C comment: correct use of 64-"byte" cache line size. Reported-by: Kelly Min Discussion: https://postgr.es/m/CAPSbxatOiQO90LYpSC3+svAU9-sHgDfEP4oFhcEUt_X=DqFA9g@mail.gmail.com Backpatch-through: 9.5 https://git.postgresql.org/pg/commitdiff/3b5af0e95ad5a3d9b478826336a11ad1d201c378 Álvaro Herrera pushed: - doc: Update partitioning limitation on BEFORE triggers. Reported-by: Erwin Brandstetter <brsaweda@gmail.com> Discussion: https://postgr.es/m/CAGHENJ6Le7S3qJJx2TvWvTwRNS3N=BtoNeb7AF2rZvfNBMeQcg@mail.gmail.com https://git.postgresql.org/pg/commitdiff/0ebe82a9413fcd5621266ec57d9f4bc50f0457d8 - Raise error on concurrent drop of partitioned index. We were already raising an error for DROP INDEX CONCURRENTLY on a partitioned table, albeit a different and confusing one: ERROR: DROP INDEX CONCURRENTLY must be first action in transaction Change that to throw a more comprehensible error: ERROR: cannot drop partitioned index \"%s\" concurrently Michael Paquier authored the test case for indexes on temporary partitioned tables. Backpatch to 11, where indexes on partitioned tables were added. Reported-by: Jan Mussler <jan.mussler@zalando.de> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://postgr.es/m/16594-d2956ca909585067@postgresql.org https://git.postgresql.org/pg/commitdiff/afc7e0ad556a4f720c466cb4815fc77d310fc50a - Fix typo in comment. Introduced by 8b08f7d4820f; backpatch to 11. Discussion: https://postgr.es/m/20200812214918.GA30353@alvherre.pgsql https://git.postgresql.org/pg/commitdiff/05c16b827f3ae66ec718e9af49e4d96906fa2ffb - Report expected contrecord length on mismatch. When reading a WAL record fails to find continuation record(s) of the proper length, report what it expects, for clarity. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/20200903212152.GA15319@alvherre.pgsql https://git.postgresql.org/pg/commitdiff/f43e295f68c3e04ef891627f62016a5b3d8ed4a8 Amit Kapila pushed: - Fix the SharedFileSetUnregister API. Commit 808e13b282 introduced a few APIs to extend the existing Buffile interface. In SharedFileSetDeleteOnProcExit, it tries to delete the list element while traversing the list with 'foreach' construct which makes the behavior of list traversal unpredictable. Author: Amit Kapila Reviewed-by: Dilip Kumar Tested-by: Dilip Kumar and Neha Sharma Discussion: https://postgr.es/m/CAA4eK1JhLatVcQ2OvwA_3s0ih6Hx9+kZbq107cXVsSWWukH7vA@mail.gmail.com https://git.postgresql.org/pg/commitdiff/4ab77697f67aa5b90b032b9175b46901859da6d7 - Add support for streaming to built-in logical replication. To add support for streaming of in-progress transactions into the built-in logical replication, we need to do three things: * Extend the logical replication protocol, so identify in-progress transactions, and allow adding additional bits of information (e.g. XID of subtransactions). * Modify the output plugin (pgoutput) to implement the new stream API callbacks, by leveraging the extended replication protocol. * Modify the replication apply worker, to properly handle streamed in-progress transaction by spilling the data to disk and then replaying them on commit. We however must explicitly disable streaming replication during replication slot creation, even if the plugin supports it. We don't need to replicate the changes accumulated during this phase, and moreover we don't have a replication connection open so we don't have where to send the data anyway. Author: Tomas Vondra, Dilip Kumar and Amit Kapila Reviewed-by: Amit Kapila, Kuntal Ghosh and Ajin Cherian Tested-by: Neha Sharma, Mahendra Singh Thalor and Ajin Cherian Discussion: https://postgr.es/m/688b0b7f-2f6c-d827-c27b-216a8e3ea700@2ndquadrant.com https://git.postgresql.org/pg/commitdiff/464824323e57dc4b397e8b05854d779908b55304 - Fix inline marking introduced in commit 464824323e. Forgot to add inline marking in changes_filename() declaration. In the passing, add inline marking for a similar function subxact_filename(). Reported-By: Nathan Bossart Discussion: https://postgr.es/m/E98FBE8F-B878-480D-A728-A60C6EED3047@amazon.com https://git.postgresql.org/pg/commitdiff/ac15b499f7f92c26661835c327bfb0228a9b5e73 Michaël Paquier pushed: - Improve handling of dropped relations for REINDEX DATABASE/SCHEMA/SYSTEM. When multiple relations are reindexed, a scan of pg_class is done first to build the list of relations to work on. However the REINDEX logic has never checked if a relation listed still exists when beginning the work on it, causing for example sudden cache lookup failures. This commit adds safeguards against dropped relations for REINDEX, similarly to VACUUM or CLUSTER where we try to open the relation, ignoring it if it is missing. A new option is added to the REINDEX routines to control if a missed relation is OK to ignore or not. An isolation test, based on REINDEX SCHEMA, is added for the concurrent and non-concurrent cases. Author: Michael Paquier Reviewed-by: Anastasia Lubennikova Discussion: https://postgr.es/m/20200813043805.GE11663@paquier.xyz https://git.postgresql.org/pg/commitdiff/1d65416661bbb0b165865a521ce038ffb61b12ad - Fix thinko with definition of REINDEXOPT_MISSING_OK. This had no direct consequences, but let's be consistent and it would be confusing when adding new flags. Oversight in 1d65416. Reported-by: Justin Pryzby Discussion: https://postgr.es/m/20200902024148.GB20149@telsasoft.com https://git.postgresql.org/pg/commitdiff/01767533e37fa255722cc0088ed67a0309875963 - Add access method names to \d[i|m|t]+ in psql. Listing a full set of relations with those psql meta-commands, without a matching pattern, has never showed the access method associated with each relation. This commit adds the access method of tables, indexes and matviews, masking it for relation kinds where it does not apply. Note that when HIDE_TABLEAM is enabled, the information does not show up. This is available when connecting to a backend version of at least 12, where table AMs have been introduced. Author: Georgios Kokolatos Reviewed-by: Vignesh C, Michael Paquier, Justin Pryzby Discussion: https://postgr.es/m/svaS1VTOEscES9CLKVTeKItjJP1EEJuBhTsA0ESOdlnbXeQSgycYwVlliL5zt8Jwcfo4ATYDXtEqsExxjkSkkhCSTCL8fnRgaCAJdr0unUg=@protonmail.com https://git.postgresql.org/pg/commitdiff/07f386ede026ae8c3f2adeba0c22139df19bf2ff - Remove variable "concurrent" from ReindexStmt. This node already handles multiple options using a bitmask, so having a separate boolean flag is not necessary. This simplifies the code a bit with less arguments to give to the reindex routines, by replacing the boolean with an equivalent bitmask value. Reviewed-by: Julien Rouhaud Discussion: https://postgr.es/m/20200902110326.GA14963@paquier.xyz https://git.postgresql.org/pg/commitdiff/844c05abc3f1c1703bf17cf44ab66351ed9711d2 - Use multi-inserts for pg_depend. This is a follow-up of the work done in e3931d01. This case is a bit different than pg_attribute and pg_shdepend: the maximum number of items to insert is known in advance, but there is no need to handle pinned dependencies. Hence, the base allocation for slots is done based on the number of items and the maximum allowed with a cap at 64kB. Slots are initialized once used to minimize the overhead of the operation. The insertions can be done for dependencies of the same type. More could be done by grouping the insertion of multiple dependency types in a single batch. This is left as future work. Some of the multi-insert logic is also simplified for pg_shdepend, as per the feedback discussed for this specific patch. This also moves to indexing.h the variable capping the maximum amount of data that can be used at once for a multi-insert, instead of having separate definitions for pg_attribute, pg_depend and pg_shdepend. Author: Daniel Gustafsson, Michael Paquier Reviewed-by: Andres Freund, Álvaro Herrera Discussion: https://postgr.es/m/20200807061619.GA23955@paquier.xyz https://git.postgresql.org/pg/commitdiff/63110c6264a5363863209c8fbdd0498c03535507 - Switch to multi-inserts when registering dependencies for many code paths. This commit improves the dependency registrations by taking advantage of the preliminary work done in 63110c62, to group together the insertion of dependencies of the same type to pg_depend. With the current layer of routines available, and as only dependencies of the same type can be grouped, there are code paths still doing more than one multi-insert when it is necessary to register dependencies of multiple types (constraint and index creation are two cases doing that). While on it, this refactors some of the code to use ObjectAddressSet() when manipulating object addresses. Author: Daniel Gustafsson, Michael Paquier Reviewed-by: Andres Freund, Álvaro Herrera Discussion: https://postgr.es/m/20200807061619.GA23955@paquier.xyz https://git.postgresql.org/pg/commitdiff/8febfd1855450f50f17419def41c2ea9bcf994d5 Fujii Masao pushed: - Avoid unnecessary acquisition of SyncRepLock in transaction commit time. In SyncRepWaitForLSN() routine called in transaction commit time, SyncRepLock is necessary to atomically both check the shared sync_standbys_defined flag and operate the sync replication wait-queue. On the other hand, when the flag is false, the lock is not necessary because the wait-queue is not touched. But due to the changes by commit 48c9f49265, previously the lock was taken whatever the flag was. This could cause unnecessary performance overhead in every transaction commit time. Therefore this commit avoids that unnecessary aquisition of SyncRepLock. Author: Fujii Masao Reviewed-by: Asim Praveen, Masahiko Sawada, Discussion: https://postgr.es/m/20200406050332.nsscfqjzk2d57zyx@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/be9788e9989a0744ba60ab100153340fd123b786 Peter Eisentraut pushed: - Remove unused parameter. unused since 39bd3fd1db6f3aa3764d4a1bebcd71c4e9c00281 Discussion: https://www.postgresql.org/message-id/flat/511bb100-f829-ba21-2f10-9f952ec06ead%402ndquadrant.com https://git.postgresql.org/pg/commitdiff/fd5e3b291415e6cf55408af1282585c945464c8f - Fix XML id to match containing page. This was apparently a typo when this part of the documentation was first added. https://git.postgresql.org/pg/commitdiff/4220e5721a27286aab726e87d3db5d1ee475fe23 - doc: Add missing cross-links in system catalog documentation. This makes the first mention of a system catalog or view in each paragraph in the system system catalog and view documentation pages hyperlinks, for easier navigation. Also linkify the first mention of pg_hba.conf in pg_hba_file_rules, as that's more specific and easier to spot than the link to the client authentication chapter. Author: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> Discussion: https://www.postgresql.org/message-id/flat/87mu5xqc11.fsf@wibble.ilmari.org https://git.postgresql.org/pg/commitdiff/cb6eb4a09e8adf3cc25e8eff9fdd43d8d9cd9c86 - doc: Make SQL command names in the catalog documentation links. In passing, fix the initdb references to be <application> rather than <command>, which is what we normally use. Author: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> Discussion: https://www.postgresql.org/message-id/flat/87mu5xqc11.fsf@wibble.ilmari.org https://git.postgresql.org/pg/commitdiff/5ccf32211832477a147b1428b190817baea08f2f - Remove unused parameter. unused since 93ee38eade1b2b4964354b95b01b09e17d6f098d Discussion: https://www.postgresql.org/message-id/flat/511bb100-f829-ba21-2f10-9f952ec06ead%402ndquadrant.com https://git.postgresql.org/pg/commitdiff/96cfcadd26e26e138ae60f4f3f94beddafba60bb - doc: Use tags consistently in the tutorial chapter. Make more consistent use of <screen> and <programlisting>. Author: Jürgen Purtz <juergen@purtz.de> Discussion: https://www.postgresql.org/message-id/flat/158996922318.7035.10603922579567326239@wrigleys.postgresql.org https://git.postgresql.org/pg/commitdiff/6eee73e4e5b6d499f1a5529f209ce4af86ef5152 - doc: Fix whitespace issue in PDF. Move <indexterm> outside of <para> to avoid whitespace issue in PDF output. Author: Jürgen Purtz <juergen@purtz.de> Discussion: https://www.postgresql.org/message-id/flat/158996922318.7035.10603922579567326239@wrigleys.postgresql.org https://git.postgresql.org/pg/commitdiff/79fd620b20b7721c2b591ef495be79b6347286c1 - doc: Change table alias names to lower case in tutorial chapter. This is needlessly different from our usual style otherwise. Author: Jürgen Purtz <juergen@purtz.de> Discussion: https://www.postgresql.org/message-id/flat/158996922318.7035.10603922579567326239@wrigleys.postgresql.org https://git.postgresql.org/pg/commitdiff/49d7165117893405ae9b5b8d8e7877acff33c0e7 - Fix typo in comment. https://git.postgresql.org/pg/commitdiff/556cbdfce4ffe01410b89dbf01b27315aa201bbf - Extend SQL function tests lightly. The basic tests that defined SQL functions didn't actually run the functions to see if they worked. Add that, and also fix a minor mistake in a function that was revealed by this. (This is not a question of test coverage, since there are other places where SQL functions are run, but it is a bit of a silly test design.) Discussion: https://www.postgresql.org/message-id/flat/1c11f1eb-f00c-43b7-799d-2d44132c02d7@2ndquadrant.com https://git.postgresql.org/pg/commitdiff/11b80d900fe4297e8e4bc231f6a41b53d604ed9e - Remove unused parameter. unused since 84d723b6cefcf25b8c800f8aa6cf3c9538a546b4 Discussion: https://www.postgresql.org/message-id/flat/511bb100-f829-ba21-2f10-9f952ec06ead%402ndquadrant.com https://git.postgresql.org/pg/commitdiff/76af9744db168cfe96f45769b8ec68eb222d8fcf - doc: Don't hide the "Up" link when it is the same as "Home". The original stylesheets seemed to think this was a good idea, but our users find it confusing and unhelpful, so undo that logic. Reported-by: Fabien COELHO <coelho@cri.ensmp.fr> Discussion: https://www.postgresql.org/message-id/flat/alpine.DEB.2.22.394.2006210914370.859381%40pseudo https://git.postgresql.org/pg/commitdiff/cd153b54eb6542d0e0b4addf1f10a1b34a328e17 Andrew Dunstan pushed: - Collect attribute data on extension owned tables being dumped. If this data is not collected, pg_dump segfaults if asked for column inserts. Fix by Fabrízio de Royes Mello Backpatch to release 12 where the bug was introduced. https://git.postgresql.org/pg/commitdiff/3eb3d3e7822d5eecfcaba871a90263c6025c5216 Jeff Davis pushed: - Fix bogus MaxAllocSize check in logtape.c. Reported-by: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-Wz=NZPZc3-fkdmvu=w2itx0PiB-G6QpxHXZOjuvFAzPdZw@mail.gmail.com Backpatch-through: 13 https://git.postgresql.org/pg/commitdiff/0852006a946aa9795b4913bccebb88d623942ca6 == Pending Patches == Kyotaro HORIGUCHI sent in two revisions of a patch to fix NaN handling of some geometric operators and functions. Tom Lane sent in two revisions of a patch to get rid of runtime handling of AlternativeSubPlan. Thomas Munro and Li Japin traded patches to reduce uses of setitimer and make it possible to terminate idle sessions via a new GUC, idle_session_timeout. Kyotaro HORIGUCHI sent in another revision of a patch to allow a directory name for GUC ssl_crl_file and connection option sslcrl. X509_STORE_load_locations accepts a directory, which leads to on-demand loading method with which method only relevant CRLs are loaded. Asim Praveen sent in another revision of a patch to Add syntax to the isolation tester that makes it possible to declare a step that is expected to block, and add a speculative insert isolation test spec using a fault injector. Victor Spirin sent in a patch to fix an issue that manifested as output to STDOUT sometimes disappearing on Windows. Justin Pryzby sent in another revision of a patch to fix up some docs for 13. Stephen Frost sent in another revision of a patch to implement a new role: pg_read_all_data. Pavel Borisov sent in two more revisions of a patch to implement covering indexes for the SP-GiST index AM. Tsutomu Yamada sent in another revision of a patch to add a command to psql, \dX[+], which lists extended statistics. Atsushi Torikoshi sent in another revision of a patch to enable pg_get_backend_memory_contexts() to collect arbitrary backend process's memory contexts. Previously, pg_get_backend_memory_contexts() could only get the memory contexts of the process which kicked it. This patch enables to get memory contexts of the arbitrary process which PID is specified by the argument. Álvaro Herrera sent in a patch to use an LWLock rather than spinlock for Logwrt{Result,Rqst}. Matthieu Garrigues sent in another revision of a patch to implement batch/pipelining support for libpq. Amit Kapila, Dilip Kumar, and Neha Sharma traded patches to add support for streaming to built-in logical replication. Justin Pryzby sent in a patch to show the stats target of extended statistics in psql. Georgios Kokolatos and Tom Lane traded patches to use StringInfo for replace_string. Etsuro Fujita sent in another revision of a patch to use Append with naive multiplexing of FDWs. Magnus Hagander sent in a patch to change the documentation for file FDW to a relative path, removing a reference to a home path that's hard to decipher in the process. Melanie Plageman sent in another revision of a patch to implement adaptive hashjoin. Amul Sul sent in another revision of a patch to implement ALTER SYSTEM SET READ {ONLY,WRITE}. Li Japin sent in a patch to clarify the documentation of where settings go. Michael Banck sent in a patch to clarify "rewritten" in the pg_checksums docs. Kirk Jamison sent in two more revisions of a patch to speed up dropping of relation buffers during recovery. Michael Banck sent in a patch to fix checksum verification in base backups for zero page headers. Dmitry Dolgov sent in a patch to reorder GROUP BY clauses to do sorting more efficiently. Jeff Davis sent in a patch to fix the handling of reloptions for table access methods. Mark Dilger sent in two more revisions of a patch to remove postfix operators generally and the postfix factorial operator in particular. Dave Page sent in two revisions of a patch to fix broken Kerberos support in MSVC builds on Win64. Daniel Gustafsson sent in another revision of a patch to support NSS as a libpq TLS backend. Daniel Gustafsson sent in another revision of a patch to enable checksums online. David Rowley sent in another revision of a patch to implement some resultcache in nestloop hacks. Tom Lane sent in a patch to label some parameter names in builtin functions. Vigneshwaran C sent in a patch to fix an issue where PostgreSQL's describe-config option prints reset_val for int & real configuration parameters which is not useful as it is not updated. Fix this by printing boot_val. Michaël Paquier sent in two more revisions of a patch to fix some warnings in CIC for partitioned tables. Michaël Paquier sent in two revisions of a patch to refactor ReindexStmt and its "concurrent" boolean. Justin Pryzby and Aleksey Kondratov traded patches to allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly. Masahiro Ikeda sent in another revision of a patch to implement a pg_stat_walwriter view. Thomas Munro, Ranier Vilela, and Juan José Santamaría Flecha traded patches to do a micro-optimization for walkdir(). Michaël Paquier sent in a patch to make index_set_state_flags() transactional. Michaël Paquier sent in another revision of a patch to remove currtid() and currtid2(). Pavel Borisov sent in another revision of a patch to implement a custom signal handler for extensions. Tom Lane sent in a WIP patch to clean up background SIGQUIT handling. Amit Langote and Álvaro Herrera traded patches to check the default partition's constraint even after tuple routing. Craig Ringer sent in a patch to add support for detecting coding errors where a stack-allocated ErrorContextCallback is not popped from the error_context_stack before the variable leaves scope. Konstantin Knizhnik sent in another revision of a patch to implement ON CONFLICT DO SELECT. Andrey V. Lepikhov sent in another revision of a patch to make it possible to pull statistics for a foreign table from remote server. Konstantin Knizhnik sent in two more revisions of a patch to implement ON LOGIN triggers. Jeff Davis sent in a patch to fix hashagg costing. Pavel Stěhule sent in another revision of a patch to implement filter files for pg_dump. Kasahara Tatsuhito sent in a patch to enable a flag in the table_recheck_autovac function to use the existing stats next time if VACUUM (or ANALYZE) has already been done by another worker on the check after the stats have been updated. If the tables continue to require VACUUM after the refresh, then a refresh will be required instead of using the existing statistics. Laurenz Albe sent in another revision of a patch to add session statistics to pg_stat_database. John Hsu sent in a patch to pg_dump that has it iterate through all publication tables in pg_publication_rel at once in order to avoid querying on a per-table basis so that we don't need to check if every table has a publication Aleksey Kondratov sent in a patch to add a postgres_fdw.use_twophase GUC to use 2PC for transactions involving several servers. Peter Eisentraut sent in a patch to add libpq's openssl dependencies to the pkg-config file. Peter Eisentraut sent in a patch to support negative years in make_timestamp(). Ranier Vilela sent in another revision of a patch to remove some redundant initializations. Peter Eisentraut sent in a patch to use <unnamed> for name of unnamed portal's memory context. Andrey Borodin sent in another revision of a patch to add sort support for point gist_point_sortsupport, and implement GiST index builds using same. Tomáš Vondra sent in another revision of a patch to implement BRIN multi-range indexes. Michaël Paquier sent in another revision of a patch to fix the range checks of pg_test_fsync. Jeff Janes sent in a patch to fix a bad error message in moving aggregates.
pgsql-announce by date: