== PostgreSQL Weekly News - April 19, 2020 == - Mailing list pgsql-announce
| From | David Fetter |
|---|---|
| Subject | == PostgreSQL Weekly News - April 19, 2020 == |
| Date | |
| Msg-id | 20200419222534.GA25823@fetter.org Whole thread Raw |
| List | pgsql-announce |
== PostgreSQL Weekly News - April 19, 2020 == == PostgreSQL Product News == pg_normalize_query 1.0, an extension to expose to SQL level a way to normalize query similar that used by to pg_stat_statements, released. https://github.com/fabriziomello/pg_normalize_query pspg 3.0.0 a pager designed for PostgreSQL, released. https://github.com/okbob/pspg/releases/tag/3.0.0 pgtt 2.0, an extension to implement global temporary tables, released. https://github.com/darold/pgtt/releases/tag/v2.0 repmgr 5.1 a replication manager for PostgreSQL, released. https://repmgr.org/docs/current/release-5.1.0.html pgmetrics 1.9, a command-line tool for PostgreSQL metrics, released. https://pgmetrics.io/ Pyrseas 0.9.1, a set of utilities for comparing and synchronizing Postgres database schemas, released. https://github.com/pyrseas/Pyrseas pg_readonly 0.0.1, an extension that makes it possible to set all cluster databases read-only, released. https://github.com/pierreforstmann/pg_readonly == PostgreSQL Jobs for April == http://archives.postgresql.org/pgsql-jobs/2020-04/ == PostgreSQL Local == PGCon 2020 will take place online on May 26-29, 2020. https://www.pgcon.org/2020/ PostgresLondon 2020 will be July 7-8, 2020 with an optional training day on July 6. http://postgreslondon.org PG Day Russia will be in Saint Petersburg on July 10, 2020. https://pgday.ru/en/2020/ FOSS4G 2020, will take place in Calgary, Alberta, Canada August 24-29 2020. the Call for Papers is currently open at https://2020.foss4g.org/speakers/ https://2020.foss4g.org/ PGDay Ukraine will take place September 5th, 2020 in Lviv at the Bank Hotel. https://pgday.org.ua/ pgDay Israel 2020 will take place on September 10, 2020 in Tel Aviv. http://pgday.org.il/ PGDay Austria will take place September 18, 2020 at Schloss Schoenbrunn (Apothekertrakt) in Vienna. https://pgday.at/en/ == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm PST8PDT to david@fetter.org. == Applied Patches == Tom Lane pushed: - Doc: introduce and document "&zwsp;" for allowing optional line breaks. We already had a couple of places using zero-width spaces for formatting hackery, and we're going to need more if we ever want the PDF manuals to look decent. But please let's not write hard-coded Unicode escapes. We can avoid that by using a custom entity, which also provides a place to put a teeny bit of documentation about what it is and how to use it. I'd previously posted a patch using "&break;" for this, but on reflection that would be horrible to grep for. Instead let's use "&zwsp;", based on the name of the Unicode symbol ("zero width space"). Discussion: https://postgr.es/m/9326.1581457869@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/88d934f0387a66ba372643913f99e845d0ea144a - Doc: introduce new layout for tables of functions and operators. We've long fought with the draconian space limitations of our traditional table layout for describing SQL functions and operators. This commit introduces a new approach, though so far I've only applied it to a few of those tables. The new way makes use of DocBook's support for different layouts in different rows of a table, and allows the descriptions and examples for a function or operator to run to several lines without as much ugliness and wasted space as before. The core layout concept is now Name Signature Description Example Example Result so that a function or operator really has three table rows not one, but we group them to look like one row by having the name column have only one entry for all three rows. (Actually, there could be four or more rows if you wanted to have more than one example, which is another thing that was painful before but works easily now.) This is handled by a "morerows" annotation on the name entry, which isn't perfect (notably, the toolchain is not smart enough to avoid breaking these row groups across PDF pages) but there seems no better solution in DocBook. The name column is normally fairly narrow, allowing plenty of space for the other column(s), and not wasting too much space when one of the other components runs to multiple lines. The varying row layout is managed by defining named "spans" and then tagging entries with a "spanname" of "name", "sig", "desc", "example", or "exresult". This provides a bit of semantic annotation to go with the formatting improvement, which seems like a good thing. (It seems that we have to re-define these spans afresh for each table, which is annoying, but it's not any worse than the duplication involved in the table headers. At least that gives us an opportunity to vary the relative column widths per-table, which is handy since function tables sometimes need much wider name columns than operator tables.) Signature entries should be written in the style <function>fname</function>(<type>typename</type> ...) <returnvalue>typename</returnvalue> The <returnvalue> tag produces a right arrow before the result type name. (I'll document that convention in a user-visible place later.) While this provides significantly more horizontal space than before for examples, it's still true that PDF output is a lot narrower than typical webpage viewing windows, so some examples need to be broken in places where there is no whitespace. I've added &zwsp; markers in suitable places to allow the tables to render warning-free in PDF. I've so far converted only the date/time operator, date/time function, and enum function tables in sections 9.9 and 9.10; these were chosen to provide a reasonable sample of the formatting problems that need to be solved. Assuming that this looks good on the website and doesn't provoke howls of anguish, I'll work on the other similar tables in the near future. There's a moderate amount of new editorial content in this patch along with the raw formatting changes; for instance I had to write text descriptions for operators that lacked them. I failed to resist the temptation to improve some other descriptions and examples, too. Patch by me, with thanks to Alexander Lakhin for assistance with figuring out some formatting issues. Discussion: https://postgr.es/m/9326.1581457869@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/e894c61836e4b967f7ec65358fdaed2ba86ed238 - Account for collation when coercing the output of a SQL function. Commit 913bbd88d overlooked that the result of coerce_to_target_type might need collation fixups. Per report from Andreas Joseph Krogh. Discussion: https://postgr.es/m/VisenaEmail.72.37d08ec2b8cb8fb5.17179940cd3@tc7-visena https://git.postgresql.org/pg/commitdiff/2d59643dbccaf73bc1f90875ea02dcad641379cd - Fix cache reference leak in contrib/sepgsql. fixup_whole_row_references() did the wrong thing with a dropped column, resulting in a commit-time warning about a cache reference leak. I (tgl) added a test case exercising this, but back-patched the test only as far as v10; the patch didn't apply cleanly to 9.6 and it didn't seem worth the trouble to adapt it. The bug is pretty old though, so apply the code change all the way back. Michael Luo, with cosmetic improvements by me Discussion: https://postgr.es/m/BYAPR08MB5606D1453D7F50E2AF4D2FD29AD80@BYAPR08MB5606.namprd08.prod.outlook.com https://git.postgresql.org/pg/commitdiff/fc576b7c4f3a5f045d443fc771d9e0f54b33e72b - Fix possible future cache reference leak in ALTER EXTENSION ADD/DROP. recordExtObjInitPriv and removeExtObjInitPriv were sloppy about calling ReleaseSysCache. The cases cannot occur given current usage in ALTER EXTENSION ADD/DROP, since we wouldn't get here for these relkinds; but it seems wise to clean up better. In passing, extend test logic in test_pg_dump to exercise the dropped-column code paths here. Since the case is unreachable at present, there seems no great need to back-patch; hence fix HEAD only. Kyotaro Horiguchi, with test case and comment adjustments by me Discussion: https://postgr.es/m/20200417.151831.1153577605111650154.horikyota.ntt@gmail.com https://git.postgresql.org/pg/commitdiff/3125a5baec1cf6d3aaeb8964bc3b3c49835e0452 - Doc: revise formatting of function/operator tables. The table layout ideas proposed in commit e894c6183 were not as widely popular as I'd hoped. After discussion, we've settled on a layout that's effectively a single-column table with cell contents much like a <varlistentry> description of the function or operator; though we're not actually using <varlistentry>, because it'd add way too much vertical space. Instead the effect is accomplished using line-break processing instructions to separate the description and example(s), plus CSS or FO customizations to produce indentation of all but the first line in each cell. While technically this is a bit grotty, it does have the advantage that we won't need to write nearly as much boilerplate markup. This patch updates tables 9.30, 9.31, and 9.33 (which were touched by the previous patch) to the revised style, and additionally converts table 9.10. A lot of work still remains to do, but hopefully it won't be too controversial. Thanks to Andrew Dunstan, Pierre Giraud, Robert Haas, Alvaro Herrera, David Johnston, Jonathan Katz, Isaac Morland for valuable ideas. Discussion: https://postgr.es/m/8691.1586798003@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/737d69ffc3cfb2e093975411c1becd65ad029478 - Fix race conditions in synchronous standby management. We have repeatedly seen the buildfarm reach the Assert(false) in SyncRepGetSyncStandbysPriority. This apparently is due to failing to consider the possibility that the sync_standby_priority values in shared memory might be inconsistent; but they will be whenever only some of the walsenders have updated their values after a change in the synchronous_standby_names setting. That function is vastly too complex for what it does, anyway, so rewriting it seems better than trying to apply a band-aid fix. Furthermore, the API of SyncRepGetSyncStandbys is broken by design: it returns a list of WalSnd array indexes, but there is nothing guaranteeing that the contents of the WalSnd array remain stable. Thus, if some walsender exits and then a new walsender process takes over that WalSnd array slot, a caller might make use of WAL position data that it should not, potentially leading to incorrect decisions about whether to release transactions that are waiting for synchronous commit. To fix, replace SyncRepGetSyncStandbys with a new function SyncRepGetCandidateStandbys that copies all the required data from shared memory while holding the relevant mutexes. If the associated walsender process then exits, this data is still safe to make release decisions with, since we know that that much WAL *was* sent to a valid standby server. This incidentally means that we no longer need to treat sync_standby_priority as protected by the SyncRepLock rather than the per-walsender mutex. SyncRepGetSyncStandbys is no longer used by the core code, so remove it entirely in HEAD. However, it seems possible that external code is relying on that function, so do not remove it from the back branches. Instead, just remove the known-incorrect Assert. When the bug occurs, the function will return a too-short list, which callers should treat as meaning there are not enough sync standbys, which seems like a reasonably safe fallback until the inconsistent state is resolved. Moreover it's bug-compatible with what has been happening in non-assert builds. We cannot do anything about the walsender-replacement race condition without an API/ABI break. The bogus assertion exists back to 9.6, but 9.6 is sufficiently different from the later branches that the patch doesn't apply at all. I chose to just remove the bogus assertion in 9.6, feeling that the probability of a bad outcome from the walsender-replacement race condition is too low to justify rewriting the whole patch for 9.6. Discussion: https://postgr.es/m/21519.1585272409@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/f332241a60aa9c0945d74642cb3dbcbc11621154 - Doc: sync functableentry markup choices with website style. Jonathan Katz felt that slightly different indentation settings made for a better-looking result, so sync stylesheet-fo.xsl (for PDF) and stylesheet.css (for non-website-style HTML) with those choices. Discussion: https://postgr.es/m/31464.1587156281@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/00f4aba46d537452da41e6d398f8f4c915de586d - Doc: update sections 9.1-9.3 for new function table layout. I took the opportunity to do some copy-editing in this area as well, and to add some new material such as a note about BETWEEN's syntactical peculiarities. Of note is that quite a few of the examples of transcendental functions needed to be updated, because the displayed output no longer matched what you get on a modern server. I believe some of these cases are side-effects of the new Ryu algorithm in float8out. Others appear to be because the examples predate the addition of type numeric, and were expecting that float8 calculations would be done although the given syntax would actually lead to calling the numeric function nowadays. https://git.postgresql.org/pg/commitdiff/81e83216d5b6826167d5116bd084b7b3a364b0b3 Peter Eisentraut pushed: - Improve error messages after LoadLibrary(). Move the file name to a format parameter to ease translatability. Add error code where missing. Make the wording consistent. https://git.postgresql.org/pg/commitdiff/0c620a5803afbb667da7ac4bcc91b245a214a80c - Fix update-unicode target. The normalization-check target needs to be run last, after moving the newly generated files into place. Also, we need an additional dependency so that unicode_norm.o is rebuilt first. Otherwise, norm_test will still test the old files but against the new expected results, which will probably fail. https://git.postgresql.org/pg/commitdiff/73afabcdc20e227beb8094efb44753b4de1c5c0a Amit Kapila pushed: - Cosmetic fixups for WAL usage work. Reported-by: Justin Pryzby and Euler Taveira Author: Justin Pryzby and Julien Rouhaud Reviewed-by: Amit Kapila Discussion: https://postgr.es/m/CAB-hujrP8ZfUkvL5OYETipQwA=e3n7oqHFU=4ZLxWS_Cza3kQQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/ef08ca113fe302b9ddf298964756eadcd41e77a1 - Comments and doc fixes for commit 40d964ec99. Reported-by: Justin Pryzby Author: Justin Pryzby, with few changes by me Reviewed-by: Amit Kapila and Sawada Masahiko Discussion: https://postgr.es/m/20200322021801.GB2563@telsasoft.com https://git.postgresql.org/pg/commitdiff/a6fea120a7e3858e642bb5e96027f166a1a6f134 - Fix the usage of parallel and full options of vacuum command. Earlier we were inconsistent in allowing the usage of parallel and full options. Change it such that we disallow them only when they are combined in a way that we don't support. In passing, improve the comments in some of the existing tests of parallel vacuum. Reported-by: Tushar Ahuja Author: Justin Pryzby, Amit Kapila Reviewed-by: Sawada Masahiko, Michael Paquier, Mahendra Singh Thalor and Amit Kapila Discussion: https://postgr.es/m/58c8d171-e665-6fa3-a9d3-d9423b694dae%40enterprisedb.com https://git.postgresql.org/pg/commitdiff/24d2d38b1eb86c0b410ad0f07f66566a83c6f05c Robert Haas pushed: - Rename pg_validatebackup to pg_verifybackup some more. The previous commit missed an instance. Noriyoshi Shinoda Discussion: http://postgr.es/m/TU4PR8401MB115291AE850BA7CF1AEB2F0BEEDD0@TU4PR8401MB1152.NAMPRD84.PROD.OUTLOOK.COM https://git.postgresql.org/pg/commitdiff/7a6b017b34783eb7ee4e93f93a52bde2bb017c8c - Document the backup manifest file format. Patch by me, at the request of Andres Freund. Reviewed by Justin Pryzby, Erik Rijkers, Álvaro Herrera, and Andrew Dunstan. Discussion: http://postgr.es/m/20200327203225.hcm6ag4grwsiruea@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/149f2ae88ab0510be3239caf45699d7373f2c6e8 Andrew Dunstan pushed: - Print policy name in perlcritic messages. This makes it easier to do a web search for details of the policy that's been violated, as well as displaying the name that might be needed for a policy override. Various perlcritic settings changes are being discussed, but this one should be uncontroversial. https://git.postgresql.org/pg/commitdiff/8930e43ecd3f683c457865131d7a932401a2188f - Use perl warnings pragma consistently. We've had a mixture of the warnings pragma, the -w switch on the shebang line, and no warnings at all. This patch removes the -w swicth and add the warnings pragma to all perl sources missing it. It raises the severity of the TestingAndDebugging::RequireUseWarnings perlcritic policy to level 5, so that we catch any future violations. Discussion: https://postgr.es/m/20200412074245.GB623763@rfd.leadboat.com https://git.postgresql.org/pg/commitdiff/7be5d8df1f74b78620167d3abf32ee607e728919 - Use perl's $/ more idiomatically. This replaces a few occurrences of ugly code with a more clean and idiomatic usage. The problem was highlighted by perlcritic, but we're not enforcing the policy that led to the discovery. Discussion: https://postgr.es/m/20200412074245.GB623763@rfd.leadboat.com https://git.postgresql.org/pg/commitdiff/8f00d84afc0dad577b65df5a313e5306cee3d11f - Set Perl search path more idiomatically. Back in commits 1df92eeafe, f884a96819, and 592123efbb I used some hackish code to set the script search path, unaware despite decades of perl that there was a completely standard way to do this. This patch changes those cases to use the standard perl FindBin package. https://git.postgresql.org/pg/commitdiff/e60c6f6ea17b50833242653fe2781f6d47429540 - Stop requiring an explicit return from perl subroutines. The consensus of the project appears to be that this provides little benefit and is simply an annoyance. Discussion: https://postgr.es/m/27481.1586618092@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/0516f94d18c57cc5dce72ff8fb84a05d24a67063 - Only provide new libpq sslpasskey hook for openssl-enabled builds. In commit 4dc6355210 I neglected to put #ifdef USE_OPENSSL around the declarations of the new items. This is remedied here. Per complaint from Daniel Gustafsson. https://git.postgresql.org/pg/commitdiff/9e24109f1a4e4d8d1d372b004d6a0dd06e673fe7 - Use a slightly more liberal regex to detect Visual Studio version. Apparently in some language versions of Visual Studio nmake outputs some material after the version number and before the end of the line. This has been seen in Chinese versions. Therefore, we no longer demand that the version string comes at the end of a line. Per complaint from Cuiping Lin. Backpatch to all live branches. https://git.postgresql.org/pg/commitdiff/a9659fb65497af8d2db66101f2ff7233d02997f1 - Only provide openssl_tls_init_hook if building with openssl. This should have been protected by #ifdef USE_OPENSSL in commit 896fcdb230. Per the real complaint this time from Daniel Gustafsson. https://git.postgresql.org/pg/commitdiff/f342d7ad03e61a1cea0339cf0c5aea0d01c3b43e - Revert "Only provide new libpq sslpasskey hook for openssl-enabled builds". This reverts commit 9e24109f1a4e4d8d1d372b004d6a0dd06e673fe7. This caused build errors when building without openssl, and it's simplest just to revert it. https://git.postgresql.org/pg/commitdiff/6741cfa5964768279a13e01e46931664fe0903a4 Peter Geoghegan pushed: - Harmonize nbtree page split point code. An nbtree split point can be thought of as a point between two adjoining tuples from an imaginary version of the page being split that includes the incoming/new item (in addition to the items that really are on the page). These adjoining tuples are called the lastleft and firstright tuples. The variables that represent split points contained a field called firstright, which is an offset number of the first data item from the original page that goes on the new right page. The corresponding tuple from origpage was usually the same thing as the actual firstright tuple, but not always: the firstright tuple is sometimes the new/incoming item instead. This situation seems unnecessarily confusing. Make things clearer by renaming the origpage offset returned by _bt_findsplitloc() to "firstrightoff". We now have a firstright tuple and a firstrightoff offset number which are comparable to the newitem/lastleft tuples and the newitemoff/lastleftoff offset numbers respectively. Also make sure that we are consistent about how we describe nbtree page split point state. Push the responsibility for dealing with pg_upgrade'd !heapkeyspace indexes down to lower level code, relieving _bt_split() from dealing with it directly. This means that we always have a palloc'd left page high key on the leaf level, no matter what. This enables simplifying some of the code (and code comments) within _bt_split(). Finally, restructure the page split code to make it clearer why suffix truncation (which only takes place during leaf page splits) is completely different to the first data item truncation that takes place during internal page splits. Tuples are marked as having fewer attributes stored in both cases, and the firstright tuple is truncated in both cases, so it's easy to imagine somebody missing the distinction. https://git.postgresql.org/pg/commitdiff/bc3087b626d1073c9b7c9687b334785909ca2237 - Make _bt_insertonpg() more like _bt_split(). It seems like a good idea for nbtree's retail insert code to be absolutely consistent with nbtree's page split code for anything that naturally requires equivalent handling. Anything that concerns inserting newitem (which is handled as part of the page split atomic action when a page split is required) should work in exactly the same way. With that in mind, make _bt_insertonpg() handle 'cbuf' in a way that matches _bt_split(). https://git.postgresql.org/pg/commitdiff/826ee1a019127d611bb0fd22ca878142bfb077ac - Add defensive "split_only_page" nbtree assertion. Clearly it's not okay for nbtree to split a page that is the only page on its level, and then find that it has to split the parent one level up in turn. There is simply no code to handle the split_only_page case in the _bt_insertonpg() "newitem won't fit" branch (only the "newitem fits" branch handles split_only_page). Add a defensive assertion that will fail if a split_only_page call to _bt_insertonpg() somehow ends up splitting the target/parent page. I (pgeoghegan) believe that we don't need split_only_page handling for the "newitem won't fit" branch because anybody calling _bt_insertonpg() like this would have to hold a lock on the same one and only child page. https://git.postgresql.org/pg/commitdiff/f762b2feba276a627585cb7e834fb7a1bf4c549d - Rearrange _bt_insertonpg() "update metapage" code. Nest the "update metapage as part of insert into root-like page" branch inside the broader "insert into internal page" branch. This improves readability. https://git.postgresql.org/pg/commitdiff/80634e3b181562015762b5846583ff67f8037a80 - Remove obsolete "hole in center of page" comment. A comment from the Berkeley days incorrectly claimed that the page management code cares about the contents of the hole in the center of the page (at least in the case of the left half of an nbtree page split). Commit 8fa30f906be added an addendum that stated that the original comment was "probably obsolete". It's definitely obsolete, though, so remove the original comment plus the addendum. https://git.postgresql.org/pg/commitdiff/4a05a6409567719829ec84d7689b41c55009d75f - Slightly simplify nbtree split point choice loop. Spotted during post-commit review of the nbtree deduplication commit (commit 0d861bbb). https://git.postgresql.org/pg/commitdiff/f0ca378d4c139eda99ef14998115c1674dac3fc5 Álvaro Herrera pushed: - Silence Perl warning. Now that warnings are enabled across the board, this code that tries to print an undef variable emits one. Silently printing the empty string achieves the previous behavior. Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Andrew Dunstan <andrew.dunstan@2ndquadrant.com> Discussion: https://postgr.es/m/E1jO1VT-0008Qk-TM@gemulon.postgresql.org https://git.postgresql.org/pg/commitdiff/e56d717d8a23c48e4ab27b6bd6815c50dc3a378a Noah Misch pushed: - Add a wait_for_catchup() before immediate stop of a test master. Per buildfarm member hoverfly, a slow walsender could make the test fail. Back-patch to v10, where the test was introduced. Discussion: https://postgr.es/m/20200414013849.GA886648@rfd.leadboat.com https://git.postgresql.org/pg/commitdiff/d60cfb6bf2b4812e97271e7b5ba3ad4713406b9d Michaël Paquier pushed: - Fix collection of typos and grammar mistakes in the tree, volume 2. This fixes some comments and documentation new as of Postgres 13, and is a follow-up of the work done in dd0f37e. Author: Justin Pryzby Discussion: https://postgr.es/m/20200408165653.GF2228@telsasoft.com https://git.postgresql.org/pg/commitdiff/8128b0c152a67917535f50738ac26da4f984ddd9 - Fix minor memory leak in pg_dump. A query used to read default ACL information from the catalogs did not free a set of PQExpBuffer. Oversight in commit e2090d9, so backpatch down to 9.6. Author: Jie Zhang Reviewed-by: Sawada Masahiko Discussion: https://postgr.es/m/05bcbc5857f948efa0b451b85a48ae10@G08CNEXMBPEKD06.g08.fujitsu.local Backpatch-through: 9.6 https://git.postgresql.org/pg/commitdiff/8f4ee44bcdbcd437f860f2d048125a118335b088 - Disable silently generation of manifests with servers <= 12 in pg_basebackup. Since 0d8c9c1, pg_basebackup would generate an error if connected to a backend version older than 12 where backup manifests are not supported. Avoiding this error is possible by using the --no-manifest option. This error handling could be confusing for some users, where patching a backup script that interacts with multiple backend versions would cause the addition of --no-manifest to potentially not generate a backup manifest even for Postgres 13 and newer versions. As we want to encourage the use of backup manifests as much as possible, this commit silently disables manifests where not supported, instead of generating an error. While on it, rework a bit the code to make it more consistent with the surroundings when generating the BASE_BACKUP command. Per discussion with Andres Freund, Stephen Frost, Robert Haas, Álvaro Herrera, Kyotaro Horiguchi, Tom Lane, David Steele, and me. Author: Michael Paquier Discussion: https://postgr.es/m/20200410080910.GZ1606@paquier.xyz https://git.postgresql.org/pg/commitdiff/542d7817f774ea9d94798eb95cdf250d4f1527d9 - Fix minor memory leak in pg_basebackup and pg_receivewal. The result of the query used to retrieve the WAL segment size from the backend was not getting freed in two code paths. Both pg_basebackup and pg_receivewal exit immediately if a failure happened on this query, so this was not an actual problem, but it could be an issue if this code gets used for other tools in different ways, be they future tools in this code tree or external, existing, ones. Oversight in commit fc49e24, so backpatch down to 11. Author: Jie Zhang Discussion: https://postgr.es/m/970ad9508461469b9450b64027842331@G08CNEXMBPEKD06.g08.fujitsu.local Backpatch-through: 11 https://git.postgresql.org/pg/commitdiff/198efe774b0d2065d828cf7822376a2871c671fc Fujii Masao pushed: - Code review for backup manifest. This commit prevents pg_basebackup from receiving backup_manifest file when --no-manifest is specified. Previously, when pg_basebackup was writing a tarfile to stdout, it tried to receive backup_manifest file even when --no-manifest was specified, and reported an error. Also remove unused -m option from pg_basebackup. Also fix typo in BASE_BACKUP command documentation. Author: Fujii Masao Reviewed-by: Michael Paquier, Robert Haas Discussion: https://postgr.es/m/01e3ed3a-8729-5aaa-ca84-e60e3ca59db8@oss.nttdata.com https://git.postgresql.org/pg/commitdiff/a2ac73e7be7adf2a9248d14322d3a5e055ea4fd0 - Add index term for backup manifest in documentation. Author: Fujii Masao Reviewed-by: Robert Haas Discussion: https://postgr.es/m/951743d0-fd7e-8e2b-d489-1368a58b7304@oss.nttdata.com https://git.postgresql.org/pg/commitdiff/4db819ba403901d38c4e4328883412c59061bb58 David Rowley pushed: - Remove unneeded constraint dependency tracking. It was previously thought that remove_useless_groupby_columns() needed to keep track of which constraints the generated plan depended upon, however, this is unnecessary. The confusion likely arose regarding this because of check_functional_grouping(), which does need to track the dependency to ensure VIEWs with columns which are functionally dependant on the GROUP BY remain so. For remove_useless_groupby_columns(), cached plans will just become invalidated when the primary key's underlying index is removed through the normal relcache invalidation code. Here we just remove the unneeded code which records the dependency and updates the comments. The previous comments claimed that we could not use UNIQUE constraints for the same optimization due to lack of a pg_constraint record for NOT NULL constraints (which are required because NULLs can be duplicated in a unique index). Since we don't actually need a pg_constraint record to handle the invalidation, it looks like we could add code to do this in the future. But not today. We're not really fixing any bug in the code here, this fix is just to set the record straight on UNIQUE constraints. This code was added back in 9.6, but due to lack of any bug, we'll not be backpatching this. Reviewed-by: Tom Lane Discussion: https://postgr.es/m/CAApHDvrdYa=VhOoMe4ZZjZ-G4ALnD-xuAeUNCRTL+PYMVN8OnQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/5b736e9cf95793ca6a4c00d291a06dfe3559c8ec - Fix possible crash with GENERATED ALWAYS columns. In some corner cases, this could also lead to corrupted values being included in the tuple. Users who are concerned that they are affected by this should first upgrade and then perform a base backup of their database and restore onto an off-line server. They should then query each table with generated columns to ensure there are no rows where the generated expression does not match a newly calculated version of the GENERATED ALWAYS expression. If no crashes occur and no rows are returned then you're not affected. Fixes bug #16369. Reported-by: Cameron Ezell Discussion: https://postgr.es/m/16369-5845a6f1bef59884@postgresql.org Backpatch-through: 12 (where GENERATED ALWAYS columns were added.) https://git.postgresql.org/pg/commitdiff/3cb02e307e350caf3c9099c6f661a95fd00e7e4c == Pending Patches == Yugo Nagata sent in another revision of a patch to implement incremental maintenance of materialized views. Fabien COELHO sent in another revision of a patch to pgbench to enable testing whether a variable exists. Zeng Wenjing sent in another revision of a patch to implement global temporary tables. Peter Eisentraut sent in a patch to improve the error messages about mismatching relkind. Julien Rouhaud and Amit Kapila traded patches to fix up the WAL usage calculation patch. David Rowley and Andy Fan traded patches to introduce UniqueKeys to determine RelOptInfo unique properties, and use same to skip DISTINCT / GROUP BY if the input is already unique. Asif Rehman sent in two more revisions of a patch to implement parallel basebackup. Dilip Kumar sent in two more revisions of a patch to fix an infelicity between logical_work_mem and logical streaming of large in-progress transactions. Davinder Singh, Juan José Santamaría Flecha, Amit Kapila, and Ranier Vilela traded patches to fix a compilation error with Microsoft Visual Studio 2015/2017/2019. Mark Dilger sent in a patch to add HeapFile related perl modules. Andrey Borodin and Magnus Hagander traded patches to allow pg_read_all_stats to read pg_stat_progress_*. Thomas Munro sent in a patch to implement snapshot_too_old using a timer. Kyotaro HORIGUCHI sent in a patch to pg restore to handle cfclose error properly. Cary Huang sent in another revision of a patch to support sequences in logical replication. Dilip Kumar sent in a patch to fix an issue with REPLICA IDENTITY FULL. Richard Guo sent in a patch to fix an incremental sort crash found by sqlsmith. Robert Haas and Thomas Munro traded patches to fix old_snapshot_threshold's time->xid mapping. Ranier Vilela sent in a patch to nbtree to avert some some unneeded calls and set vars. Ranier Vilela sent in a patch to fix a possible overflow on tuplesort.c. Amit Langote sent in a patch to avoid partial nested append, which could break run-time partition pruning. Peter Eisentraut sent in a patch to make pgstattuple_approx accept TOAST tables. Masahiko Sawada sent in a patch to fix an error in pg_dump involving generated columns. Amit Langote sent in a patch to rearrange some code in pgoutput.c, and fix some partition logical replication TAP tests for better coverage. Nikita Glukhov sent in a patch to return FALSE instead of NULL in jsonpath operators. Kyotaro HORIGUCHI sent in a patch to fix an aclchk cacheref leak. Masahiro Ikeda sent in a patch to fix the documentation about the way to exit standby mode. Peter Geoghegan sent in a patch to redefine split interval to be space-wise. Thomas Munro sent in a patch to rename xid to xid4. Pavel Stěhule sent in two revisions of a patch to implement a string_to_table() function. Thomas Munro sent in two revisions of a patch to expose oldSnapshotControl, add a time->xid mapping to contrib/old_snapshot, fix some bugs in MaintainOldSnapshotTimeMapping, add a pg_clobber_current_snapshot_timestamp(), and truncate old snapshot XIDs before truncating CLOG. Noah Misch sent in a patch to revert "When WalSndCaughtUp, sleep only in WalSndWaitForWal(). Robert Haas sent in a patch to move the server's backup manifest code to a separate file. Justin Pryzby sent in two revisions of a patch to fix explain output for incremental sort, and fix some comment typos in same. James Coleman sent in a patch to disable mark/restore for incremental sort, and fix up EXEC_FLAG_REWIND for incremental sort. Pavel Stěhule sent in a patch to make it possible to use a pager for psql's \watch command. Justin Pryzby sent in three more revisions of a patch to fix detaching tables with inherited row triggers. Ranier Vilela sent in a patch to remove a few unneeded uses of strlen(). Jeff Davis sent in a patch to fix a performance regression related to FORTIFY_SOURCE.
pgsql-announce by date: