== PostgreSQL Weekly News - May 19, 2019 == - Mailing list pgsql-announce
| From | David Fetter |
|---|---|
| Subject | == PostgreSQL Weekly News - May 19, 2019 == |
| Date | |
| Msg-id | 20190519180053.GA24627@fetter.org Whole thread Raw |
| List | pgsql-announce |
== PostgreSQL Weekly News - May 19, 2019 == == PostgreSQL Product News == Pgpool-II 4.0.5, 3.7.10, 3.6.17, 3.5.21 and 3.4.24 released. http://pgpool.net/mediawiki/index.php/Downloads == PostgreSQL Jobs for May == http://archives.postgresql.org/pgsql-jobs/2019-05/ == PostgreSQL Local == PGCon 2019 will take place in Ottawa on May 28-31, 2019. Registration is open. https://www.pgcon.org/2019 pgibz will be held in Ibiza, Spain on June 19-23, 2019. The CfP is open. https://pgibz.io/ Swiss PGDay 2019 will take place in Rapperswil (near Zurich) on June 28, 2019. Registration is open. http://www.pgday.ch/2019/ PostgresLondon 2019 will be July 2-3, 2019 with an optional training day on July 1. http://postgreslondon.org PGConf.Brazil 2019 will take place August 1-3, 2019 in São Paulo. http://pgconf.com.br The first Austrian pgDay, will take place September 6, 2019 at the Hilton Garden Inn in Wiener Neustadt. https://pgday.at/en/ PostgresConf South Africa 2019 will take place in Johannesburg on October 8-9, 2019 The Call for Papers is open through June 30, 2019. https://postgresconf.org/conferences/SouthAfrica2019 == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm PST8PDT to david@fetter.org. == Applied Patches == Noah Misch pushed: - Fail pgwin32_message_to_UTF16() for SQL_ASCII messages. The function had been interpreting SQL_ASCII messages as UTF8, throwing an error when they were invalid UTF8. The new behavior is consistent with pg_do_encoding_conversion(). This affects LOG_DESTINATION_STDERR and LOG_DESTINATION_EVENTLOG, which will send untranslated bytes to write() and ReportEventA(). On buildfarm member bowerbird, enabling log_connections caused an error whenever the role name was not valid UTF8. Back-patch to 9.4 (all supported versions). Discussion: https://postgr.es/m/20190512015615.GD1124997@rfd.leadboat.com https://git.postgresql.org/pg/commitdiff/d02768ddd1791bab742cf131de726d0a36b3f337 Tom Lane pushed: - Fix misoptimization of "{1,1}" quantifiers in regular expressions. A bounded quantifier with m = n = 1 might be thought a no-op. But according to our documentation (which traces back to Henry Spencer's original man page) it still imposes greediness, or non-greediness in the case of the non-greedy variant "{1,1}?", on whatever it's attached to. This turns out not to work though, because parseqatom() optimizes away the m = n = 1 case without regard for whether it's supposed to change the greediness of the argument RE. We can fix this by just not applying the optimization when the greediness needs to change; the subsequent general cases handle it fine. The three cases in which we can still apply the optimization are (a) no quantifier, or quantifier does not impose a preference; (b) atom has no greediness property, implying it cannot match a variable amount of text anyway; or (c) quantifier's greediness is same as atom's. Note that in most cases where one of these applies, we'd have exited earlier in the "not a messy case" fast path. I think it's now only possible to get to the optimization when the atom involves capturing parentheses or a non-top-level backref. Back-patch to all supported branches. I'd ordinarily be hesitant to put a subtle behavioral change into back branches, but in this case it's very hard to see a reason why somebody would write "{1,1}?" unless they're trying to get the documented change-of-greediness behavior. Discussion: https://postgr.es/m/5bb27a41-350d-37bf-901e-9d26f5592dd0@charter.net https://git.postgresql.org/pg/commitdiff/8a29ed05302ea18b4be177c2d1181b43706af45f - Fix misuse of an integer as a bool. pgtls_read_pending is declared to return bool, but what the underlying SSL_pending function returns is a count of available bytes. This is actually somewhat harmless if we're using C99 bools, but in the back branches it's a live bug: if the available-bytes count happened to be a multiple of 256, it would get converted to a zero char value. On machines where char is signed, counts of 128 and up could misbehave as well. The net effect is that when using SSL, libpq might block waiting for data even though some has already been received. Broken by careless refactoring in commit 4e86f1b16, so back-patch to 9.5 where that came in. Per bug #15802 from David Binderman. Discussion: https://postgr.es/m/15802-f0911a97f0346526@postgresql.org https://git.postgresql.org/pg/commitdiff/ddf927fb13471bec0cc76794eaf552df8a1178fb - Improve commentary about hack in is_publishable_class(). The FirstNormalObjectId test here is a kluge that needs to go away, but the only substitute we can think of is to add a column to pg_class, which will take more work than can be handled right now. Add some commentary in the meanwhile. Discussion: https://postgr.es/m/15150.1557257111@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/e34ee993fbc38c8538f9342c5961e5f61fd45180 - Fix logical replication's ideas about which type OIDs are built-in. Only hand-assigned type OIDs should be presumed to match across different PG servers; those assigned during genbki.pl or during initdb are likely to change due to addition or removal of unrelated objects. This means that the cutoff should be FirstGenbkiObjectId (in HEAD) or FirstBootstrapObjectId (before that), not FirstNormalObjectId. Compare postgres_fdw's is_builtin() test. It's likely that this error has no observable consequence in a normally-functioning system, since ATM the only affected type OIDs are system catalog rowtypes and information_schema types, which would not typically be interesting for logical replication. But you could probably break it if you tried hard, so back-patch. Discussion: https://postgr.es/m/15150.1557257111@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/32ebb35128c3d16b64039f6d6774afd96a3d91b9 - In bootstrap mode, use default signal handling for SIGINT etc. Previously, the code pointed the standard process-termination signals to postgres.c's die(). That would typically result in an attempt to execute a transaction abort, which is not possible in bootstrap mode, leading to PANIC. This choice seems to be a leftover from an old code structure in which the same signal-assignment code was used for many sorts of auxiliary processes, including interactive standalone backends. It's not very sensible for bootstrap mode, which has no interest in either interactivity or continuing after an error. We can get better behavior with less effort by just letting normal process termination happen, after which the parent initdb process will clean up. This is basically cosmetic in any case, since initdb will react the same way whether bootstrap dies on a signal or abort(). Given the lack of previous complaints, I don't feel a need to back-patch, even though the behavior is old. Discussion: https://postgr.es/m/3850b11a.5121.16aaf827e4a.Coremail.thunder1@126.com https://git.postgresql.org/pg/commitdiff/fb489e4b3195fc33cccc0fd308e5a0ab502cf199 - Fix SQL-style substring() to have spec-compliant greediness behavior. SQL's regular-expression substring() function is defined to have a pattern argument that's separated into three subpatterns by escape- double-quote markers; the function result is the part of the input matching the second subpattern. The standard makes it clear that if there is ambiguity about how to match the input to the subpatterns, the first and third subpatterns should be taken to match the smallest possible amount of text (i.e., they're "non greedy", in the terms of our regex code). We were not doing it that way: the first subpattern would eat the largest possible amount of text, causing the function result to be shorter than what the spec requires. Fix that by attaching explicit greediness quantifiers to the subpatterns. (This depends on the regex fix in commit 8a29ed053; before that, this didn't reliably change the regex engine's behavior.) Also, by adding parentheses around each subpattern, we ensure that "|" (OR) in the subpatterns behave sanely. Previously, "|" in the first or third subpatterns didn't work. This patch also makes the function throw error if you write more than two escape-double-quote markers, and do something sane if you write just one, and document that behavior. Previously, an odd number of markers led to a confusing complaint about unbalanced parentheses, while extra pairs of markers were just ignored. (Note that the spec requires exactly two markers, but we've historically allowed there to be none, and this patch preserves the old behavior for that case.) In passing, adjust some substring() test cases that didn't really prove what they said they were testing for: they used patterns that didn't match the data string, so that the output would be NULL whether or not the function was really strict. Although this is certainly a bug fix, changing the behavior in back branches seems undesirable: applications could perhaps be depending on the old behavior, since it's not obviously wrong unless you read the spec very closely. Hence, no back-patch. Discussion: https://postgr.es/m/5bb27a41-350d-37bf-901e-9d26f5592dd0@charter.net https://git.postgresql.org/pg/commitdiff/7c850320d8cfa5503ecec61c2559661b924f7595 - Remove pg_rewind's private logging.h/logging.c files. The existence of these files became rather confusing with the introduction of a widely-known logging.h header in commit cc8d41511. (Indeed, there's already some duplicative #includes here, perhaps betraying such confusion.) The only thing left in them, after that commit, is a progress-reporting function that's neither general-purpose nor tied in any way to other logging infrastructure. Hence, let's just move that function to pg_rewind.c, and get rid of the separate files. Discussion: https://postgr.es/m/3971.1557787914@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/53ddefbaf8a0493d2c1fa0b18bfaba72da556985 - Move logging.h and logging.c from src/fe_utils/ to src/common/. The original placement of this module in src/fe_utils/ is ill-considered, because several src/common/ modules have dependencies on it, meaning that libpgcommon and libpgfeutils now have mutual dependencies. That makes it pointless to have distinct libraries at all. The intended design is that libpgcommon is lower-level than libpgfeutils, so only dependencies from the latter to the former are acceptable. We already have the precedent that fe_memutils and a couple of other modules in src/common/ are frontend-only, so it's not stretching anything out of whack to treat logging.c as a frontend-only module in src/common/. To the extent that such modules help provide a common frontend/backend environment for the rest of common/ to use, it's a reasonable design. (logging.c does not yet provide an ereport() emulation, but one can dream.) Hence, move these files over, and revert basically all of the build-system changes made by commit cc8d41511. There are no places that need to grow new dependencies on libpgcommon, further reinforcing the idea that this is the right solution. Discussion: https://postgr.es/m/a912ffff-f6e4-778a-c86a-cf5c47a12933@2ndquadrant.com https://git.postgresql.org/pg/commitdiff/fc9a62af3f87f4bec1e8c904ea99ae50f3c881ef - Fix "make clean" to clean out junk files left behind after ssl tests. We .gitignore'd this junk, but we didn't actually remove it. https://git.postgresql.org/pg/commitdiff/6d2fba3189608186d1a196a595f21760412ec5e8 - Remove no-longer-used typedef. struct ClonedConstraint is no longer needed, so delete it. Discussion: https://postgr.es/m/18102.1557947143@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/8a0f0ad54047c8dc1430e521e2dc90f2cd138a9d - Fix partition pruning to treat stable comparison operators properly. Cross-type comparison operators in a btree or hash opclass might be only stable not immutable (this is true of timestamp vs. timestamptz for example). partprune.c ignored this possibility and would perform plan-time pruning with them anyway, possibly leading to wrong answers if the environment changed between planning and execution. To fix, teach gen_partprune_steps() to do things differently when creating plan-time pruning steps vs. run-time pruning steps. analyze_partkey_exprs() also needs an extra check, which is rather annoying but now is not the time to restructure things enough to avoid that. While at it, simplify the logic for the plan-time case a little by insisting that the comparison value be a Const and nothing else. This relies on the assumption that eval_const_expressions will have reduced any immutable expression to a Const; which is not quite 100% true, but certainly any case that comes up often enough to be interesting should have simplification logic there. Also improve a bunch of inadequate/obsolete/wrong comments. Per discussion of a report from Alan Jackson (though this fixes only one aspect of that problem). Back-patch to v11 where this code came in. David Rowley, with some further hacking by me Discussion: https://postgr.es/m/FAD28A83-AC73-489E-A058-2681FA31D648@tvsquared.com https://git.postgresql.org/pg/commitdiff/4b1fcb43d070ba8b34ea01d5a657630d76e4b33f - Fix bogus logic for combining range-partitioned columns during pruning. gen_prune_steps_from_opexps's notion of how to do this was overly complicated and underly correct. Per discussion of a report from Alan Jackson (though this fixes only one aspect of that problem). Back-patch to v11 where this code came in. Amit Langote Discussion: https://postgr.es/m/FAD28A83-AC73-489E-A058-2681FA31D648@tvsquared.com https://git.postgresql.org/pg/commitdiff/3922f10646fc79264228ea3f6a7343cc3f8f528a - Restructure creation of run-time pruning steps. Previously, gen_partprune_steps() always built executor pruning steps using all suitable clauses, including those containing PARAM_EXEC Params. This meant that the pruning steps were only completely safe for executor run-time (scan start) pruning. To prune at executor startup, we had to ignore the steps involving exec Params. But this doesn't really work in general, since there may be logic changes needed as well --- for example, pruning according to the last operator's btree strategy is the wrong thing if we're not applying that operator. The rules embodied in gen_partprune_steps() and its minions are sufficiently complicated that tracking their incremental effects in other logic seems quite impractical. Short of a complete redesign, the only safe fix seems to be to run gen_partprune_steps() twice, once to create executor startup pruning steps and then again for run-time pruning steps. We can save a few cycles however by noting during the first scan whether we rejected any clauses because they involved exec Params --- if not, we don't need to do the second scan. In support of this, refactor the internal APIs in partprune.c to make more use of passing information in the GeneratePruningStepsContext struct, rather than as separate arguments. This is, I hope, the last piece of our response to a bug report from Alan Jackson. Back-patch to v11 where this code came in. Discussion: https://postgr.es/m/FAD28A83-AC73-489E-A058-2681FA31D648@tvsquared.com https://git.postgresql.org/pg/commitdiff/6630ccad7a25cad32e2d1a6833fb971602cb67fe - "A void function may not return a value". Per buildfarm. https://git.postgresql.org/pg/commitdiff/d307954a7d88ed45655d9206bbe1aefc280c09c7 - Make BufFileCreateTemp() ensure that temp tablespaces are set up. If PrepareTempTablespaces() has never been called in the current transaction, OpenTemporaryFile() will fall back to using the default tablespace, which is a bug if the user wanted temp files placed elsewhere. gistInitBuildBuffers() appears to have this disease already, and it seems like an easy trap for future coders to fall into. We discussed other ways to close this gap, but none of them are prettier or more reliable than just having BufFileCreateTemp do it. In particular, having fd.c do this creates layering issues that we could do without. Per suggestion from Melanie Plageman. Arguably this is a bug fix, but nobody seems very excited about back-patching, so change in HEAD only. Discussion: https://postgr.es/m/CAAKRu_YwzjuGAmmaw4-8XO=OVFGR1QhY_Pq-t3wjb9ribBJb_Q@mail.gmail.com https://git.postgresql.org/pg/commitdiff/93f03dad824f14f40519597e5e4a8fe7b6df858e - ANSI-ify a few straggler K&R-style function definitions. We still had a couple of these left in ancient src/port/ files. Convert them to modern style in preparation for switching to a version of pg_bsd_indent that doesn't cope well with K&R style. Discussion: https://postgr.es/m/16886.1558104483@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/da71f98efba9aed493a178aa29f25dbca7407eaf - Revert "postmaster: Start syslogger earlier". This commit reverts 57431a911d3a650451d198846ad3194900666152. While that's still a good idea in the abstract, we found out that there are multiple crasher bugs in it on Windows builds, making the logging_collector option unusable on Windows. There's no time left to fix these issues before 12beta1, so revert the patch to allow Windows beta testing to proceed. We'll try again at some future date. Per bug #15804 from Yulian Khodorkovskiy and additional investigation by Michael Paquier. Discussion: https://postgr.es/m/15804-3721117bf40fb654@postgresql.org https://git.postgresql.org/pg/commitdiff/833451552925d0175e1e15128e411ddef9a36996 Michaël Paquier pushed: - Fix incorrect return value in JSON equality function for scalars. equalsJsonbScalarValue() uses a boolean as return type, however for one code path -1 gets returned, which is confusing. The origin of the confusion is visibly that this code got copy-pasted from compareJsonbScalarValue() since it has been introduced in d1d50bf. No backpatch, as this is only cosmetic. Author: Rikard Falkeborn Discussion: https://postgr.es/m/CADRDgG7mJnek6HNW13f+LF6V=6gag9PM+P7H5dnyWZAv49aBGg@mail.gmail.com https://git.postgresql.org/pg/commitdiff/1171dbde2daef8f0dcd1dc1e54531a0d8dd34d88 - Fix duplicated words in comments. Author: Stephen Amell Discussion: https://postgr.es/m/539fa271-21b3-777e-a468-d96cffe9c768@gmail.com https://git.postgresql.org/pg/commitdiff/7e19929ea29668f3d01280262e407b93042c959a - Fix typos in documentatoin of GSSAPI encryption. Author: Daniel Gustafsson Discussion: https://postgr.es/m/5520EDD8-7AC7-4307-8171-400DD1D84FDC@yesql.se https://git.postgresql.org/pg/commitdiff/41998f90ce66425ca6071ac5e170a19bb1498414 - Fix regression test outputs. 75445c1 has caused various failures in tests across the tree after updating some error messages, so fix the newly-expected output. Author: Michael Paquier Reviewed-by: Tom Lane Discussion: https://postgr.es/m/8332.1558048838@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/6ba500cae6f57acf8f392d9e1671640f7c9d43b3 Bruce Momjian pushed: - docs: fix typo in mention of MSVC. https://git.postgresql.org/pg/commitdiff/fefb6a75386b72a40f697523953eafc7cb1c7a5b - doc: adjust PG 12 release note sections. Tighten section designations. https://git.postgresql.org/pg/commitdiff/a6927996be59ac329d65d41cf058d0b1b0978eeb - doc: PG 12 release notes: normalize attribution names. Reported-by: David Rowley Discussion: https://postgr.es/m/CAKJS1f-ktEhmQ2zJQ1L1niuJ9KB8WPA-bE-AhGiFsSO6QASB_w@mail.gmail.com https://git.postgresql.org/pg/commitdiff/f86b0c3c465319b80d71db7d7a9de636950e104a - doc: properly attibute PG 12 pgbench release note item. Reported-by: Fabien COELHO Discussion: https://postgr.es/m/alpine.DEB.2.21.1905130839140.13487@lancre https://git.postgresql.org/pg/commitdiff/5d971565a7993ba8dcd06d34d1387af48fdbd034 - doc: improve wording of PG 12 releaase note partition item. Reported-by: Amit Langote Discussion: https://postgr.es/m/d5267ae5-bd4a-3e96-c21b-56bfa9fec7e8@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/f4125278e3c92d8cdd83c77d0b54f468ee81c750 - doc: Update OID item in PG 12 release notes. Reported-by: Justin Pryzby Discussion: https://postgr.es/m/20190513174759.GE23251@telsasoft.com https://git.postgresql.org/pg/commitdiff/0b62f0f2552d895301fdae56aa839c1eaae24a3a - docs: fix duplicate wording in PG 12 release notes. Reported-by: nickb@imap.cc Discussion: https://postgr.es/m/6b3414e1-fcef-4ad9-b123-b3ab3702d3db@www.fastmail.com https://git.postgresql.org/pg/commitdiff/34d40becfa7a5c7343d0079f181597d7e06122e1 - docs: update partition item in PG 12 release notes. Reported-by: Amit Langote Discussion: https://postgr.es/m/b7954643-41ef-a174-479d-1f8d4834f40a@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/356c83795aaa39cfd9bca588bfe4f0196710962e - docs: Indent listitem tags in PG 12 release notes. https://git.postgresql.org/pg/commitdiff/b71dad22ce8a645a47c01e544f640f35b91bfbd3 - docs: properly indent PG 12 release notes. https://git.postgresql.org/pg/commitdiff/a429164ef31fc6899cef4d5e4f3f2ce8fc33aa27 - docs: split out sort-skip partition item in PG 12 release notes. Discussion: https://postgr.es/m/0cf10a27-c6a0-de4a-cd20-ab7493ea7422@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/05685897f07e7ae5684f0247157faad1f782f889 - docs: tighten up PG 12 release note item on 1k partitions. https://git.postgresql.org/pg/commitdiff/b12db9ff5f09dc60dc05d0e208a5b28728970e55 Etsuro Fujita pushed: - postgres_fdw: Fix typo in comment. https://git.postgresql.org/pg/commitdiff/cc866941ad4583f07e5637c145f6f6ee8a573e11 - Fix typo. https://git.postgresql.org/pg/commitdiff/7d9eca59cf958eb8c8ff47c40003392681a9ce68 Robert Haas pushed: - Improve comment for att_isnull. The comment implies that a 1 in the null bitmap indicates a null value, but actually a 0 in the null bitmap indicates a null value. Try to be more clear. Patch by me; proposed wording reviewed by Alvaro Herrera and Tom Lane. Discussion: http://postgr.es/m/CA+TgmobHOP8r6cG+UnsDFMrS30-m=jRrCBhgw-nFkn0k9QnFsg@mail.gmail.com https://git.postgresql.org/pg/commitdiff/221b377f09a30dd7c74c26c8c1db439afcc59141 Peter Geoghegan pushed: - Don't leave behind junk nbtree pages during split. Commit 8fa30f906be reduced the elevel of a number of "can't happen" _bt_split() errors from PANIC to ERROR. At the same time, the new right page buffer for the split could continue to be acquired well before the critical section. This was possible because it was relatively straightforward to make sure that _bt_split() could not throw an error, with a few specific exceptions. The exceptional cases were safe because they involved specific, well understood errors, making it possible to consistently zero the right page before actually raising an error using elog(). There was no danger of leaving around a junk page, provided _bt_split() stuck to this coding rule. Commit 8224de4f, which introduced INCLUDE indexes, added code to make _bt_split() truncate away non-key attributes. This happened at a point that broke the rule around zeroing the right page in _bt_split(). If truncation failed (perhaps due to palloc() failure), that would result in an errant right page buffer with junk contents. This could confuse VACUUM when it attempted to delete the page, and should be avoided on general principle. To fix, reorganize _bt_split() so that truncation occurs before the new right page buffer is even acquired. A junk page/buffer will not be left behind if _bt_nonkey_truncate()/_bt_truncate() raise an error. Discussion: https://postgr.es/m/CAH2-WzkcWT_-NH7EeL=Az4efg0KCV+wArygW8zKB=+HoP=VWMw@mail.gmail.com Backpatch: 11-, where INCLUDE indexes were introduced. https://git.postgresql.org/pg/commitdiff/9b42e713761a43e9ade3965285f077e2ba25bbb7 - Doc: Refer to line pointers as item identifiers. An upcoming HEAD-only patch will standardize the terminology around ItemIdData variables/line pointers, ending the practice of referring to them as "item pointers". Make the "Database Page Layout" docs consistent with the new policy. The term "item identifier" is already used in the same section, so stick with that. Discussion: https://postgr.es/m/CAH2-Wz=c=MZQjUzde3o9+2PLAPuHTpVZPPdYxN=E4ndQ2--8ew@mail.gmail.com Backpatch: All supported branches. https://git.postgresql.org/pg/commitdiff/08ca9d7feca890e97f77ef1fde02d7542a54ac5e - Standardize ItemIdData terminology. The term "item pointer" should not be used to refer to ItemIdData variables, since that is needlessly ambiguous. Only ItemPointerData/ItemPointer variables should be called item pointers. To fix, establish the convention that ItemIdData variables should always be referred to either as "item identifiers" or "line pointers". The term "item identifier" already predominates in docs and translatable messages, and so should be the preferred alternative there. Discussion: https://postgr.es/m/CAH2-Wz=c=MZQjUzde3o9+2PLAPuHTpVZPPdYxN=E4ndQ2--8ew@mail.gmail.com https://git.postgresql.org/pg/commitdiff/ae7291acbc5ae4c3751615177fc256f9f3399403 - Reverse order of newitem nbtree candidate splits. Commit fab25024, which taught nbtree to choose candidate split points more carefully, had _bt_findsplitloc() record all possible split points in an initial pass over a page that is about to be split. The order that candidate split points were processed and stored in was assumed to match the offset number order of split points on an imaginary version of the page that contains the same items as the original, but also fits newitem (the item that provoked the split precisely because it didn't fit). However, the order of split points in the final array was not quite what was expected: the split point that makes newitem the firstright item came after the split point that makes newitem the lastleft item -- not before. As a result, _bt_findsplitloc() could get confused about the leftmost and rightmost tuples among all possible split points recorded for the page. This seems to have no appreciable impact on the quality of the final split point chosen by _bt_findsplitloc(), but it's still wrong. To fix, switch the order in which newitem candidate splits are recorded in. This also makes it possible to describe candidate split points in terms of which pair of adjoining tuples enclose the split point within _bt_findsplitloc(), making it clearer why it's generally safe for _bt_split() to expect lastleft and firstright tuples. https://git.postgresql.org/pg/commitdiff/7505da2f459776a397177f7d591991f5591c2812 - Remove obsolete nbtree insertion comment. Remove a Berkeley-era comment above _bt_insertonpg() that admonishes the reader to grok Lehman and Yao's paper before making any changes. This made a certain amount of sense back when _bt_insertonpg() was responsible for most of the things that are now spread across _bt_insertonpg(), _bt_findinsertloc(), _bt_insert_parent(), and _bt_split(), but it doesn't work like that anymore. I believe that this comment alludes to the need to "couple" or "crab" buffer locks as we ascend the tree as page splits cascade upwards. The nbtree README already explains this in detail, which seems sufficient. Besides, the changes to page splits made by commit 40dae7ec537 altered the exact details of how buffer locks are retained during splits; Lehman and Yao's original algorithm seems to release the lock on the left child page/buffer slightly earlier than _bt_insertonpg()/_bt_insert_parent() can. https://git.postgresql.org/pg/commitdiff/489e431ba56b10f5736fb54a045ca40880f11bbc - Remove extra nbtree half-dead internal page check. It's not safe for nbtree VACUUM to attempt to delete a target page whose right sibling is already half-dead, since that would fail the cross-check when VACUUM attempts to re-find a downlink to the right sibling in the parent page. Logic to prevent this from happening was added by commit 8da31837803, which addressed a bug in the overhaul of page deletion that went into PostgreSQL 9.4 (commit efada2b8e92). VACUUM was made to check the right sibling page, and back off when it happened to be half-dead already. However, it is only truly necessary to do the right sibling check on the leaf level, since that transitively determines if the deletion target's parent's right sibling page is itself undergoing deletion. Remove the internal page level check, and add a comment explaining why the leaf level check alone suffices. The extra check is also unnecessary due to the fact that internal pages that are marked half-dead are generally considered corrupt. Commit efada2b8e92 established the principle that there should never be half-dead internal pages (internal pages pending deletion are possible, but that status is never directly represented in the internal page). VACUUM will complain about corruption when it encounters half-dead internal pages, so VACUUM is bound to raise an error one way or another when an nbtree index has a half-dead internal page (contrib/amcheck will also report that the page is corrupt). It's possible that a pg_upgrade'd 9.3 database will still have half-dead internal pages, so it may seem like there is an argument for leaving the check in place to reliably get a cleaner error message that advises the user to REINDEX. However, leaf pages are also deleted in the first phase of deletion prior to PostgreSQL 9.4, so I believe we won't even attempt to re-find the parent page anyway (we won't have the fully deleted leaf page as the right sibling of our target page, so we won't even try to find a downlink for it). Discussion: https://postgr.es/m/CAH2-Wzm_ntmqJjWLRyKzimFmFvk+BnVAvUpaA4s1h9Ja58woaQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/3f58cc6dd8fc821992da7ed8099d283b014fc2dc Heikki Linnakangas pushed: - Fix comment on when HOT update is possible. The conditions listed in this comment have changed several times, and at some point the thing that the "if so" referred to was negated. The text was OK up to 9.6. It was differently wrong in v10, v11 and master, so fix in all those versions. https://git.postgresql.org/pg/commitdiff/e95d550bbbe21f51d329fbae84cbac3545a9e650 - Detect internal GiST page splits correctly during index build. As we descend the GiST tree during insertion, we modify any downlinks on the way down to include the new tuple we're about to insert (if they don't cover it already). Modifying an existing downlink might cause an internal page to split, if the new downlink tuple is larger than the old one. If that happens, we need to back up to the parent and re-choose a page to insert to. We used to detect that situation, thanks to the NSN-LSN interlock normally used to detect concurrent page splits, but that got broken by commit 9155580fd5. With that commit, we now use a dummy constant LSN value for every page during index build, so the LSN-NSN interlock no longer works. I thought that was OK because there can't be any other backends modifying the index during index build, but missed that the insertion itself can modify the page we're inserting to. The consequence was that we would sometimes insert the new tuple to an incorrect page, one whose downlink doesn't cover the new tuple. To fix, add a flag to the stack that keeps track of the state while descending tree, to indicate that a page was split, and that we need to retry the descend from the parent. Thomas Munro first reported that the contrib/intarray regression test was failing occasionally on the buildfarm after commit 9155580fd5. The failure was intermittent, because the gistchoose() function is not deterministic, and would only occasionally create the right circumstances for this bug to cause the failure. Patch by Anastasia Lubennikova, with some changes by me to make it work correctly also when the internal page split also causes the "grandparent" to be split. Discussion: https://www.postgresql.org/message-id/CA%2BhUKGJRzLo7tZExWfSbwM3XuK7aAK7FhdBV0FLkbUG%2BW0v0zg%40mail.gmail.com https://git.postgresql.org/pg/commitdiff/22251686f07f70527aecda22ab5402986884f6f5 Peter Eisentraut pushed: - Update information_schema for SQL:2016. This is mainly a light renumbering to match the sections in the standard. https://git.postgresql.org/pg/commitdiff/eb3a1376c98a1d220354b468f4b09e4a1dca909a - Update SQL features/conformance information to SQL:2016. https://git.postgresql.org/pg/commitdiff/037165ca95d854e04c0c28cfa85f1515bd852892 - Update SQL keywords list to SQL:2016. Per previous convention (see ace397e9d24eddc56e7dffa921f506117b602d78), drop SQL:2008 and only keep the latest two standards and SQL-92. Note: SQL:2016-2 lists a large number of non-reserved keywords that are really just information_schema column names related to new features. Those kinds of thing have not previously been listed as keywords, and this was apparently done here by mistake, since these keywords have been removed again in post-2016 working drafts. So in order to avoid bloating the keywords table unnecessarily, I have omitted these erroneous keywords here. https://git.postgresql.org/pg/commitdiff/c29ba98189190288c2251c6073c13c6234479e5c Andres Freund pushed: - Add isolation test for INSERT ON CONFLICT speculative insertion failure. This path previously was not reliably covered. There was some heuristic coverage via insert-conflict-toast.spec, but that test is not deterministic, and only tested for a somewhat specific bug. Backpatch, as this is a complicated and otherwise untested code path. Unfortunately 9.5 cannot handle two waiting sessions, and thus cannot execute this test. Triggered by a conversion with Melanie Plageman. Author: Andres Freund Discussion: https://postgr.es/m/CAAKRu_a7hbyrk=wveHYhr4LbcRnRCG=yPUVoQYB9YO1CdUBE9Q@mail.gmail.com Backpatch: 9.5- https://git.postgresql.org/pg/commitdiff/08e2edc0767ab6e619970f165cb34d4673105f23 - Handle table_complete_speculative's succeeded argument as documented. For some reason both callsite and the implementation for heapam had the meaning inverted (i.e. succeeded == true was passed in case of conflict). That's confusing. I (Andres) briefly pondered whether it'd be better to rename table_complete_speculative's argument to 'bool specConflict' or such, but decided not to. The 'complete' in the function name for me makes `succeeded` sound a bit better. Reported-By: Ashwin Agrawal, Melanie Plageman, Heikki Linnakangas Discussion: https://postgr.es/m/CALfoeitk7-TACwYv3hCw45FNPjkA86RfXg4iQ5kAOPhR+F1Y4w@mail.gmail.com https://postgr.es/m/97673451-339f-b21e-a781-998d06b1067c@iki.fi https://git.postgresql.org/pg/commitdiff/aa4b8c61d2cd57b53be03defb04d59b232a0e150 - tableam: Don't assume that every AM uses md.c style storage. Previously various parts of the code routed size requests through RelationGetNumberOfBlocks[InFork]. That works if md.c is used by the AM, but not otherwise. Add a tableam callback to return the size of the table. As not every AM will use postgres' BLCKSZ, have it return bytes, and have RelationGetNumberOfBlocksInFork() round the byte size up into blocks. To allow code outside of the AM to determine the actual relation size map InvalidForkNumber the total size of a relation, as not every AM might just need the postgres defined forks. A few users of RelationGetNumberOfBlocks() ought to be converted away from that. One case, the use of it to determine whether a tid is valid, will be fixed in a follow up commit. Others will have to wait for v13. Author: Andres Freund Discussion: https://postgr.es/m/20190423225201.3bbv6tbqzkb5w7cw@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/7f44ede5941499c4cee13b812dd93335f4005095 - tableam: Avoid relying on relation size to determine validity of tids. Instead add a tableam callback to do so. To avoid adding per validation overhead, pass a scan to tuple_tid_valid. In heap's case we'd otherwise incurred a RelationGetNumberOfBlocks() call for each tid - which'd have added noticable overhead to nodeTidscan.c. Author: Andres Freund Reviewed-By: Ashwin Agrawal Discussion: https://postgr.es/m/20190515185447.gno2jtqxyktylyvs@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/147e3722f7e531f15ba389a4d518efe8cd0bd736 Álvaro Herrera pushed: - More message style fixes. Discussion: https://postgr.es/m/20190515183005.GA26486@alvherre.pgsql https://git.postgresql.org/pg/commitdiff/75445c1515fffa581585f1b72e5b5314c1c395ce Alexander Korotkov pushed: - Improve documentation for array subscription in jsonpath. Usage of expressions and multiple ranges in jsonpath array subscription was undocumented. This commit adds lacking documentation. https://git.postgresql.org/pg/commitdiff/87bcc02f4f739d76703feee0ff8346b98fa9b907 - Document jsonpath .** accessor with nesting level filter. It appears that some variants of .** jsonpath accessor are undocumented. In particular undocumented variants are: .**{level} .**{lower_level to upper_level} .**{lower_level to last} This commit adds missing documentation for them. https://git.postgresql.org/pg/commitdiff/e0e3dad770d3eefc4ee6cd562d9c25b61c263a6e - Fix declarations of couple jsonpath functions. Make jsonb_path_query_array() and jsonb_path_query_first() use PG_FUNCTION_ARGS macro instead of its expansion. https://git.postgresql.org/pg/commitdiff/da24961e9e1d887111d17cc2dcece2e3946ca5a6 == Pending Patches == Michaël Paquier sent in another revision of a patch to clean up and refactor reindexdb.c. Alexander Kuzmenkov sent in another revision of a patch to remove unneeded self-joins. Zhang Lian Zhuang sent in a patch to ensure that pg_stat_database's stats_reset is only updated by pg_stat_reset. Yugo Nagata sent in another revision of a patch to implement incremental maintenance of materialized views. Stas Kelvich sent in a patch to ensure read-only access to temp tables for 2PC transactions. Thomas Munro sent in two revisions of a patch to add tab completion to psql for CREATE TYPE. Takeshi Ideriha sent in another revision of a patch to implement another PoC of MemoryContext for DSA. Thomas Munro sent in a patch to implement ldapbindpasswdfile. Ashwin Agrawal sent in another revision of a patch to rename table AM wrappers to match the corresponding table AM callback names. Fujii Masao sent in two revisions of a patch to ensure that VACUUM processes 0 and 1 as booleans, as much of the other code does. Michaël Paquier sent in another revision of a patch to fix some inconsistent error message wording for REINDEX CONCURRENTLY. Dhruv Goel sent in a patch to avoid deadlock errors in CREATE INDEX CONCURRENTLY. Dilip Kumar sent in another revision of a patch to clean up orphaned files using undo logs. Peter Eisentraut sent in another revision of a patch to convert ExecComputeStoredGenerated to use tuple slots. Álvaro Herrera sent in a patch to fix more messsages. Hubert Zhang sent in a patch to replace to replace growEnable with growPenalty in hashtable. Masahiko Sawada sent in another revision of a patch to add --index-cleanup and --truncate options to vacuumdb. David Fetter sent in two more revisions of a patch to change EXPLAIN's ANALYZE option to the clearer and shorter EXEC, preserving backward compatibility. Tom Lane sent in two revisions of a patch to fix bsdindent's treatment of function declarations. Fabien COELHO sent in another revision of a patch to add a SHOW_ALL_RESULTS option to psql for multi-statement cases. Thomas Munro sent in a patch to avoid hash join batch explosions with extreme skew and weird stats. Rushabh Lathia sent in a patch to consider the default_tablespace if the parent partitioned tablespace is an InvalidOid. Kyotaro HORIGUCHI sent in two more revisions of a patch to implement a shared-memory-based stats collector. Etsuro Fujita sent in a patch to fix estimate_path_cost_size in the PostgreSQL FDW. Robert Haas sent in a patch to move heap-specific logic from needs_toast_table below tableam, split tuptoaster.c into three separate files, create an API for inserting and deleting rows in TOAST tables, and allow TOAST tables to be implemented using table AMs other than heap. Ashwin Agrawal and Robert Haas traded patches to create a TOAST table only if the table AM needs it. Alexander Lakhin sent in a patch to fix some unique identifiers/entities. Jeff Janes sent in a patch to make bitmap-only heap scans more transparent. Paul A Jungwirth sent in a patch to implement temporal foreign keys. David Rowley sent in a patch to fix the argument count for the "combine" functions used in aggregates.
pgsql-announce by date: