== PostgreSQL Weekly News - January 27, 2019 == - Mailing list pgsql-announce
From | David Fetter |
---|---|
Subject | == PostgreSQL Weekly News - January 27, 2019 == |
Date | |
Msg-id | 20190128004839.GA7527@fetter.org Whole thread Raw |
List | pgsql-announce |
== PostgreSQL Weekly News - January 27, 2019 == == PostgreSQL Product News == datasketches 1.0.0, an extension that provides approximate algorithms for big data analysis based on the Datasketches library, released. https://pgxn.org/dist/datasketches/1.0.0/ == PostgreSQL Jobs for January == http://archives.postgresql.org/pgsql-jobs/2019-01/ == PostgreSQL Local == FOSDEM PGDay 2019, a one day conference held before the main FOSDEM event will be held in Brussels, Belgium, on Feb 1st, 2019. https://2019.fosdempgday.org/ Prague PostgreSQL Developer Day 2019 (P2D2 2019) is a two-day conference that will be held on February 13-14, 2019 in Prague, Czech Republic. http://www.p2d2.cz/ PGConf India 2019 will be on February 13-15, 2019 in Bengaluru, Karnataka. http://pgconf.in/ PostgreSQL@SCaLE is a two day, two track event which takes place on March 7-8, 2019, at Pasadena Convention Center, as part of SCaLE 17X. https://www.socallinuxexpo.org/scale/17x/postgresscale pgDay Paris 2019 will be held in Paris, France on March 12, 2019 at 199bis rue Saint-Martin. http://2019.pgday.paris/ PGConf APAC 2019 will be held in Singapore March 19-21, 2019. http://2019.pgconfapac.org/ The German-speaking PostgreSQL Conference 2019 will take place on May 10, 2019 in Leipzig. The CfP is open until February 26, 2019 at http://2019.pgconf.de/cfp http://2019.pgconf.de/ PGDay.IT 2019 will take place May 16th and May 17th in Bologna, Italy. https://2019.pgday.it/en/ PGCon 2019 will take place in Ottawa on May 28-31, 2019. https://www.pgcon.org/2019 Swiss PGDay 2019 will take place in Rapperswil (near Zurich) on June 28, 2019. The CfP is open through April 18, 2019, and registration is open. http://www.pgday.ch/2019/ == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm PST8PDT to david@fetter.org. == Applied Patches == Etsuro Fujita pushed: - Postpone generating tlists and EC members for inheritance dummy children. Previously, in set_append_rel_size(), we generated tlists and EC members for dummy children for possible use by partition-wise join, even if partition-wise join was disabled or the top parent was not a partitioned table, but adding such EC members causes noticeable planning speed degradation for queries with certain kinds of join quals like "(foo.x + bar.y) = constant" where foo and bar are partitioned tables in cases where there are lots of dummy children, as the EC members lists grow huge, especially for the ECs derived from such join quals, which makes the search for the parent EC members in add_child_rel_equivalences() very time-consuming. Postpone the work until such children are actually involved in a partition-wise join. Reported-by: Sanyo Capobiango Analyzed-by: Justin Pryzby and Alvaro Herrera Author: Amit Langote, with a few additional changes by me Reviewed-by: Ashutosh Bapat Backpatch-through: v11 where partition-wise join was added Discussion: https://postgr.es/m/CAO698qZnrxoZu7MEtfiJmpmUtz3AVYFVnwzR%2BpqjF%3DrmKBTgpw%40mail.gmail.com https://git.postgresql.org/pg/commitdiff/8d8dcead1295a0d718b08e84d62913b587501425 - postgres_fdw: Account for tlist eval costs in estimate_path_cost_size(). Previously, estimate_path_cost_size() didn't account for tlist eval costs, except when costing a foreign-grouping path using local statistics, but such costs should be accounted for when costing that path using remote estimates, because some of the tlist expressions might be evaluated locally. Also, such costs should be accounted for in the case of a foreign-scan or foreign-join path, because the tlist might contain PlaceHolderVars, which postgres_fdw currently evaluates locally. This also fixes an oversight in my commit f8f6e44676. Like that commit, apply this to HEAD only to avoid destabilizing existing plan choices. Author: Etsuro Fujita Discussion: https://postgr.es/m/5BFD3EAD.2060301%40lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/fd1afdbafd4fbb0ce23a3f319adc177e4cf8fe99 Tom Lane pushed: - Revert "Fix under-quoted filename pattern in pgbench TAP test.". This reverts commit 458a1244f1fcf407874482a93b7631ecf5303d6e. It has portability problems on Windows, which will require a little bit of research to fix. Discussion: https://postgr.es/m/20202.1548035461@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/f4593bd2ff0376fbc56dd1b384c7d51e1a7062f0 - Avoid thread-safety problem in ecpglib. ecpglib attempts to force the LC_NUMERIC locale to "C" while reading server output, to avoid problems with strtod() and related functions. Historically it's just issued setlocale() calls to do that, but that has major problems if we're in a threaded application. setlocale() itself is not required by POSIX to be thread-safe (and indeed is not, on recent OpenBSD). Moreover, its effects are process-wide, so that we could cause unexpected results in other threads, or another thread could change our setting. On platforms having uselocale(), which is required by POSIX:2008, we can avoid these problems by using uselocale() instead. Windows goes its own way as usual, but we can make it safe by using _configthreadlocale(). Platforms having neither continue to use the old code, but that should be pretty much nobody among current systems. This should get back-patched, but let's see what the buildfarm thinks of it first. Michael Meskes and Tom Lane; thanks also to Takayuki Tsunakawa. Discussion: https://postgr.es/m/31420.1547783697@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/8eb4a9312c95be56cdb31f5411eddc2cb2ba89be - Sort the dependent objects before recursing in findDependentObjects(). Historically, the notices output by DROP CASCADE tended to come out in uncertain order, and in some cases you might get different claims about which object depends on which other one. This is because we just traversed the dependency tree in the order in which pg_depend entries are seen, and nbtree has never promised anything about the order of equal-keyed index entries. We've put up with that for years, hacking regression tests when necessary to prevent them from emitting unstable output. However, it's a problem for pending work that will change nbtree's behavior for equal keys, as that causes unexpected changes in the regression test results. Hence, adjust findDependentObjects to sort the results of each indexscan before processing them. The sort is on descending OID of the dependent objects, hence more or less reverse creation order. While this rule could still result in bogus regression test failures if an OID wraparound occurred mid-test, that seems unlikely to happen in any plausible development or packaging-test scenario. This is enough to ensure output stability for ordinary DROP CASCADE commands, but not for DROP OWNED BY, because that has a different code path with the same problem. We might later choose to sort in the DROP OWNED BY code as well, but this patch doesn't do so. I've also not done anything about reverting the existing hacks to suppress unstable DROP CASCADE output in specific regression tests. It might be worth undoing those, but it seems like a distinct question. The first indexscan loop in findDependentObjects is not touched, meaning there is a hazard of unstable error reports from that too. However, said hazard is not the fault of that code: it was designed on the assumption that there could be at most one "owning" object to complain about, and that assumption does not seem unreasonable. The recent patch that added the possibility of multiple DEPENDENCY_INTERNAL_AUTO links broke that assumption, but we should fix that situation not band-aid around it. That's a matter for another patch, though. Discussion: https://postgr.es/m/12244.1547854440@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/f1ad067fc3ae3d34dc6b8826913d454cc3fe354f - Fix sepgsql regression test. Message order in the expected output changes due to commit f1ad067fc. Per buildfarm. Discussion: https://postgr.es/m/20190121201134.dyx6anto6akflh5d@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/071e11898dd42c89dc68ddc4e4fe6a35049729d6 - Second try at fixing ecpglib thread-safety problem. While Windows (allegedly) has _configthreadlocale() pretty far back, it seems MinGW didn't acquire support for that till more recently. Fortunately, we can use an autoconf probe on that toolchain, instead of guessing whether it's there. (Hm, I wonder whether Cygwin will need this also.) Per buildfarm. Discussion: https://postgr.es/m/20190121193512.tdmcnic2yjxlufaw@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/ee27584c4a40620fb26d1729e9cc449d54d62b07 - Remove useless bms_copy step in RelationGetIndexAttrBitmap. Seems to be from a bad case of copy-and-paste-itis in commit 665d1fad9. It wouldn't be quite so annoying if it didn't contradict the comment half a dozen lines above. David Rowley Discussion: https://postgr.es/m/CAKJS1f95Dyf8Qkdz4W+PbCmT-HTb54tkqUCC8isa2RVgSJ_pXQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/8f9e934ab746907abbeeed06a1a9b995e5b4d339 - Blind attempt to fix _configthreadlocale() failures on MinGW. Apparently, some builds of MinGW contain a version of _configthreadlocale() that always returns -1, indicating failure. Rather than treating that as a curl-up-and-die condition, soldier on as though the function didn't exist. This leaves us without thread safety on such MinGW versions, but we didn't have it anyway. Discussion: https://postgr.es/m/d06a16bc-52d6-9f0d-2379-21242d7dbe81@2ndQuadrant.com https://git.postgresql.org/pg/commitdiff/2cf91ccb73ce888c44e3751548fb7c77e87335f2 - Fix portability problem in pgbench. The pgbench regression test supposed that srandom() with a specific value would result in deterministic output from random(), as required by POSIX. It emerges however that OpenBSD is too smart to be constrained by mere standards, so their random() emits nondeterministic output anyway. While a workaround does exist, what seems like a better fix is to stop relying on the platform's srandom()/random() altogether, so that what you get from --random-seed=N is not merely deterministic but platform independent. Hence, use a separate pg_jrand48() random sequence in place of random(). Also adjust the regression test case that's supposed to detect nondeterminism so that it's more likely to detect it; the original choice of random_zipfian parameter tended to produce the same output all the time even if the underlying behavior wasn't deterministic. In passing, improve pgbench's docs about random_zipfian(). Back-patch to v11 where this code was introduced. Fabien Coelho and Tom Lane Discussion: https://postgr.es/m/4615.1547792324@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/e6c3ba7fbfd59ceabcf9bdaf52d71b44831b09d2 - Remove infinite-loop hazards in ecpg test suite. A report from Andrew Dunstan showed that an ecpglib breakage that causes repeated query failures could lead to infinite loops in some ecpg test scripts, because they contain "while(1)" loops with no exit condition other than successful test completion. That might be all right for manual testing, but it seems entirely unacceptable for automated test environments such as our buildfarm. We don't want buildfarm owners to have to intervene manually when a test goes wrong. To fix, just change all those while(1) loops to exit after at most 100 iterations (which is more than any of them expect to iterate). This seems sufficient since we'd see discrepancies in the test output if any loop executed the wrong number of times. I tested this by dint of intentionally breaking ecpg_do_prologue to always fail, and verifying that the tests still got to completion. Back-patch to all supported branches, since the whole point of this exercise is to protect the buildfarm against future mistakes. Discussion: https://postgr.es/m/18693.1548302004@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/d5a1fde397269bd60b99e70a4eb2934b75fd31ec - Remove _configthreadlocale() calls in ecpg test suite. This essentially reverts commits a772624b1 and 04fbe0e45, which added "_configthreadlocale(_ENABLE_PER_THREAD_LOCALE)" calls to the thread-related ecpg test programs. That was nothing but a hack, because we shouldn't expect that ecpg-using applications have done that for us; and now that we've inserted such calls into ecpglib, the tests should still pass without it. (If they don't, it would be good to know that.) HEAD only; there seems no big need to change this in the back branches. Discussion: https://postgr.es/m/22937.1548307384@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/e3565fd61cee16414a682d2be0d44c112a80a30c - Fix possibly-uninitialized-variable warning from commit 9556aa01c. Heikki's compiler doesn't complain about end_ptr, apparently, but mine does. In passing, I failed to resist the temptation to remove the no-longer-used fldnum variable, and relocate chunk_len's declaration to a narrower scope. https://git.postgresql.org/pg/commitdiff/6119060d01ab2dfb9e2c1c101595f2de7890965d - Teach nulltestsel() that system columns are never NULL. While it's perhaps unlikely that users would write an explicit test like "ctid IS NULL", this function is also used in range estimation, and an incorrect answer can throw off the results for tight ranges. Anyway it's not much code so we might as well do it. Edmund Horner Discussion: https://postgr.es/m/CAMyN-kCa3BFUFrCTtQeprxTU1anCd3Pua7zXstGCKq4pXgjukw@mail.gmail.com https://git.postgresql.org/pg/commitdiff/ff750ce2d82979e9588c629955e161a9379b05f3 - Split QTW_EXAMINE_RTES flag into QTW_EXAMINE_RTES_BEFORE/_AFTER. This change allows callers of query_tree_walker() to choose whether to visit an RTE before or after visiting the contents of the RTE (i.e., prefix or postfix tree order). All existing users of QTW_EXAMINE_RTES want the QTW_EXAMINE_RTES_BEFORE behavior, but an upcoming patch will want QTW_EXAMINE_RTES_AFTER, and it seems like a potentially useful change on its own. Andreas Karlsson (extracted from CTE inlining patch) Discussion: https://postgr.es/m/8810.1542402910@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/18c0da88a5d9da566c3bfac444366b73bd0b57da - Allow UNLISTEN in hot-standby mode. Since LISTEN is (still) disallowed, UNLISTEN must be a no-op in a hot-standby session, and so there's no harm in allowing it. This change allows client code to not worry about whether it's connected to a primary or standby server when performing session-state-reset type activities. (Note that DISCARD ALL, which includes UNLISTEN, was already allowed, making it inconsistent to reject UNLISTEN.) Per discussion, back-patch to all supported versions. Shay Rojansky, reviewed by Mi Tar Discussion: https://postgr.es/m/CADT4RqCf2gA_TJtPAjnGzkC3ZiexfBZiLmA-mV66e4UyuVv8bA@mail.gmail.com https://git.postgresql.org/pg/commitdiff/ebfe20dc706bd3238a9bdf3b44cd8f82337e86a8 - Fix psql's "\g target" meta-command to work with COPY TO STDOUT. Previously, \g would successfully execute the COPY command, but the target specification if any was ignored, so that the data was always dumped to the regular query output target. This seems like a clear bug, so let's not just fix it but back-patch it. While at it, adjust the documentation for \copy to recommend "COPY ... TO STDOUT \g foo" as a plausible alternative. Back-patch to 9.5. The problem exists much further back, but the code associated with \g was refactored enough in 9.5 that we'd need a significantly different patch for 9.4, and it doesn't seem worth the trouble. Daniel Vérité, reviewed by Fabien Coelho Discussion: https://postgr.es/m/15dadc39-e050-4d46-956b-dcc4ed098753@manitou-mail.org https://git.postgresql.org/pg/commitdiff/6d3ede5f1c654f923b2767b0b0c3b09569adaa18 - Allow for yet another crash symptom in 013_crash_restart.pl. Given the right timing, psql could emit "connection to server was lost" rather than one of the other messages that this test script checked for. It looks like commit 4247db625 may have made this more likely, but I don't really believe it was impossible before then. Rather than stress about it, just add that spelling as one of the crash-successfully- detected cases. Discussion: https://postgr.es/m/19344.1548554028@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/d6f6f0fc2d84906985b21d959622e4cab6f8f9b1 Andres Freund pushed: - Adjust some more comments for WITH OIDS removal. I missed these in 578b229718e8f. Author: Andres Freund https://git.postgresql.org/pg/commitdiff/93507e67c9ca54026019ebec3026de35d30370f9 - Fix ALTER TRIGGER ... RENAME, broken in WITH OIDS removal. I (Andres) broke this in 578b229718e. Author: Rushabh Lathia Discussion: https://postgr.es/m/CAGPqQf04PywZX3sVQaF6H=oLiW9GJncRW+=e78vTy4MokEWcZw@mail.gmail.com https://git.postgresql.org/pg/commitdiff/8cc157b23413c21f229e376cb2982a2086655fc9 - Introduce access/{table.h, relation.h}, for generic functions from heapam.h. access/heapam contains functions that are very storage specific (say heap_insert() and a lot of lower level functions), and fairly generic infrastructure like relation_open(), heap_open() etc. In the upcoming pluggable storage work we're introducing a layer between table accesses in general and heapam, to allow for different storage methods. For a bit cleaner separation it thus seems advantageous to move generic functions like the aforementioned to their own headers. access/relation.h will contain relation_open() etc, and access/table.h will contain table_open() (formerly known as heap_open()). I've decided for table.h not to include relation.h, but we might change that at a later stage. relation.h already exists in another directory, but the other plausible name (rel.h) also conflicts. It'd be nice if there were a non-conflicting name, but nobody came up with a suggestion. It's possible that the appropriate way to address the naming conflict would be to rename nodes/relation.h, which isn't particularly well named. To avoid breaking a lot of extensions that just use heap_open() etc, table.h has macros mapping the old names to the new ones, and heapam.h includes relation, table.h. That also allows to keep the bulk renaming of existing callers in a separate commit. Author: Andres Freund Discussion: https://postgr.es/m/20190111000539.xbv7s6w7ilcvm7dp@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/4b21acf522d751ba5b6679df391d5121b6c4a35f - Replace uses of heap_open et al with the corresponding table_* function. Author: Andres Freund Discussion: https://postgr.es/m/20190111000539.xbv7s6w7ilcvm7dp@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/e0c4ec07284db817e1f8d9adfb3fffc952252db0 - Replace heapam.h includes with {table, relation}.h where applicable. A lot of files only included heapam.h for relation_open, heap_open etc - replace the heapam.h include in those files with the narrower header. Author: Andres Freund Discussion: https://postgr.es/m/20190111000539.xbv7s6w7ilcvm7dp@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/111944c5ee567f1c45bf0f1ecfdec682af467aa6 - Remove superfluous tqual.h includes. Most of these had been obsoleted by 568d4138c / the SnapshotNow removal. This is is preparation for moving most of tqual.[ch] into either snapmgr.h or heapam.h, which in turn is in preparation for pluggable table AMs. Author: Andres Freund Discussion: https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/e7cc78ad43eb1a6ea43dbb46c83264d2d00e402c - Fix "Remove superfluous tqual.h includes" by adding back one include. I removed one include too many in e7cc78ad43eb, not sure why that escaped my test script. Author: Andres Freund https://git.postgresql.org/pg/commitdiff/527114e51e45dbd91d46171fa4a111355f0dfc3b - Change snapshot type to be determined by enum rather than callback. This is in preparation for allowing the same snapshot be used for different table AMs. With the current callback based approach we would need one callback for each supported AM, which clearly would not be extensible. Thus add a new Snapshot->snapshot_type field, and move the dispatch into HeapTupleSatisfiesVisibility() (which is now a function). Later work will then dispatch calls to HeapTupleSatisfiesVisibility() and other AMs visibility functions depending on the type of the table. The central SnapshotType enum also seems like a good location to centralize documentation about the intended behaviour of various types of snapshots. As tqual.h isn't included by bufmgr.h any more (as HeapTupleSatisfies* isn't referenced by TestForOldSnapshot() anymore) a few files now need to include it directly. Author: Andres Freund, loosely based on earlier work by Haribabu Kommi Discussion: https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql https://git.postgresql.org/pg/commitdiff/63746189b23815415cacc715fdc4f6b991f1a5e7 - Move generic snapshot related code from tqual.h to snapmgr.h. The code in tqual.c is largely heap specific. Due to the upcoming pluggable storage work, it therefore makes sense to move it into access/heap/ (as the file's header notes, the tqual name isn't very good). But the various statically allocated snapshot and snapshot initialization functions are now (see previous commit) generic and do not depend on functions declared in tqual.h anymore. Therefore move. Also move XidInMVCCSnapshot as that's useful for future AMs, and already used outside of tqual.c. Author: Andres Freund Discussion: https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/b7eda3e0e33402a7727253e6f307ca96b92eb23d - Rephrase references to "time qualification". Now that the relevant code has, for other reasons, moved out of tqual.[ch], it seems time to refer to visiblity rather than time qualification. Author: Andres Freund Discussion: https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/ebcc7bf949bae614cccc6b86e3ef9b926a6e2f99 - Move remaining code from tqual.[ch] to heapam.h / heapam_visibility.c. Given these routines are heap specific, and that there will be more generic visibility support in via table AM, it makes sense to move the prototypes to heapam.h (routines like HeapTupleSatisfiesVacuum will not be exposed in a generic fashion, because they are too storage specific). Similarly, the code in tqual.c is specific to heap, so moving it into access/heap/ makes sense. Author: Andres Freund Discussion: https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/c91560defc57f89f7e88632ea14ae77b5cec78ee - Rename RelationData.rd_amroutine to rd_indam. The upcoming table AM support makes rd_amroutine to generic, as its only about index AMs. The new name makes that clear, and is shorter to boot. Author: Andres Freund Discussion: https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/346ed70b0ad7efc574711a97812692dab4542712 - llvm: Fix file-ending in IDENTIFICATION comments. Author: Amit Langote Discussion: https://postgr.es/m/9a54dcef-c799-ce89-2e47-0a7fc12d5fc2@lab.ntt.co.jp Backpatch: 11-, where llvm was introduced. https://git.postgresql.org/pg/commitdiff/005881033d4e34c0b2e0a157464dc1929db52e17 - Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/a9c35cf85ca1ff72f16f0f10d7ddee6e582b62b8 Álvaro Herrera pushed: - Add 'id' to Acknowledgments section. Per note from Erik Rijkers Discussion: https://postgr.es/m/3db724af16ee009ab7f812a6a1d9354e@xs4all.nl https://git.postgresql.org/pg/commitdiff/fcea1e10904856bbc77b701a0195d0524b9705d6 - Flush relcache entries when their FKs are meddled with. Back in commit 100340e2dcd0, we made relcache entries keep lists of the foreign keys applying to the relation -- but we forgot to update CacheInvalidateHeapTuple to flush those entries when new FKs got created or existing ones updated/deleted. No bugs appear to have been reported that would be explained by this ommission, but I noticed the problem while working on an unrelated bugfix which clearly showed it. Fix by adding relcache flush on relevant foreign key changes. Backpatch to 9.6, like the aforementioned commit. Discussion: https://postgr.es/m/201901211927.7mmhschxlejh@alvherre.pgsql Reviewed-by: Tom Lane https://git.postgresql.org/pg/commitdiff/175544093524bef9c28c59c8d49c77b1fcf4fb92 - Create action triggers when partitions are detached. Detaching a partition from a partitioned table that's constrained by foreign keys requires additional action triggers on the referenced side; otherwise, DELETE/UPDATE actions there fail to notice rows in the table that was partition, and so are incorrectly allowed through. With this commit, those triggers are now created. Conversely, when a table that has a foreign key is attached as a partition to a table that also has the same foreign key, those action triggers are no longer needed, so we remove them. Add a minimal test case verifying (part of) this. Authors: Amit Langote, Álvaro Herrera Discussion: https://postgr.es/m/f2b8ead5-4131-d5a8-8016-2ea0a31250af@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/0464fdf07f69452dd67dd798c026d82c956bda33 - Detach constraints when partitions are detached. I (Álvaro) forgot to do this in eb7ed3f30634, leading to undroppable constraints after partitions are detached. Repair. Reported-by: Amit Langote Author: Amit Langote Discussion: https://postgr.es/m/c1c9b688-b886-84f7-4048-1e4ebe9b1d06@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/ae366aa57762ad0e6a1a0885a7644e79541afe39 - Simplify coding to detach constraints when detaching partition. The original coding was too baroque and led to an use-after-release mistake, noticed by buildfarm member prion. Discussion: https://postgr.es/m/21693.1548305934@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/19184fcc09739abf75ccdada965ed6135c6d07c3 - Fix droppability of constraints upon partition detach. We were failing to set conislocal correctly for constraints in partitions after partition detach, leading to those constraints becoming undroppable. Fix by setting the flag correctly. Existing databases might contain constraints with the conislocal wrongly set to false, for partitions that were detached; this situation should be fixable by applying an UPDATE on pg_constraint to set conislocal true. This problem should otherwise be innocuous and should disappear across a dump/restore or pg_upgrade. Secondarily, when constraint drop was attempted in a partitioned table, ATExecDropConstraint would try to recurse to partitions after doing performDeletion() of the constraint in the partitioned table itself; but since the constraint in the partitions are dropped by the initial call of performDeletion() (because of following dependencies), the recursion step would fail since it would not find the constraint, causing the whole operation to fail. Fix by preventing recursion. Reported-by: Amit Langote Diagnosed-by: Amit Langote Author: Amit Langote, Álvaro Herrera Discussion: https://postgr.es/m/f2b8ead5-4131-d5a8-8016-2ea0a31250af@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/efd9366dcedec8db9fc85e77d071b0c5a6badea2 Michaël Paquier pushed: - Adjust documentation for vacuumdb --disable-page-skipping. This makes the description more consistent with the other options, and the mapping with VACUUM is intuitive. Author: Nathan Bossart Discussion: https://postgr.es/m/FFE5373C-E26A-495B-B5C8-911EC4A41C5E@amazon.com https://git.postgresql.org/pg/commitdiff/efab708997dcd7a78cce289bcd9d3643a3455465 - Make vacuumdb test regex more modular for its query output. This is in preparation for always using a catalog query to discover tables, where the ANALYZE and VACUUM queries get completed with relation names. Author: Nathan Bossart Discussion: https://postgr.es/m/20190122060730.GD8719@paquier.xyz https://git.postgresql.org/pg/commitdiff/adaaacae658e37a3e3c836266681774395b437fe - Fix typo in pgbench.c. Author: Moon, Insung Discussion: https://postgr.es/m/008001d4b2db$1f772170$5e656450$@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/1699e6dd1fa7fbc7c3d114b121828eb27eb523b1 - Remove argument isprimary from index_build(). The flag was introduced in 3fdeb18, but f66e8bf actually forgot to finish the cleanup as index_update_stats() has simplified its interface. Author: Michael Paquier Discussion: https://postgr.es/m/20190122080852.GB3873@paquier.xyz https://git.postgresql.org/pg/commitdiff/289198c0d93a3d591f379c4134e90275bd426426 - Simplify restriction handling of two-phase commit for temporary objects. There were two flags used to track the access to temporary tables and to the temporary namespace of a session which are used to restrict PREPARE TRANSACTION, however the first control flag is a concept included in the second. This removes the flag for temporary table tracking, keeping around only the one at namespace level. Author: Michael Paquier Reviewed-by: Álvaro Herrera Discussion: https://postgr.es/m/20190118053126.GH1883@paquier.xyz https://git.postgresql.org/pg/commitdiff/c9b75c5838feeae73dbae00bce9d8f650b80ba38 - Add TAP tests for vacuumdb with column lists. vacuumdb generates by itself SQL queries to run ANALYZE or VACUUM on the backend, but we never actually checked for query patterns with column lists defined. Author: Michael Paquier Reviewed-by: Nathan Bossart Discussion: https://postgr.es/m/FFE5373C-E26A-495B-B5C8-911EC4A41C5E@amazon.com https://git.postgresql.org/pg/commitdiff/0803b0ae1ed6e5825eee4d1ba227e96cd8a328d8 Tomáš Vondra pushed: - Fix handling of volatile expressions in COPY FROM ... WHERE. The checking for calls to volatile functions in the COPY FROM ... WHERE expression was treating all WHERE clauses as if containing such calls. While that does not produce incorrect results, this disables batching which may result in significant performance regression. Discussion: https://www.postgresql.org/message-id/flat/CALAY4q_DdpWDuB5-Zyi-oTtO2uSk8pmy+dupiRe3AvAc++1imA@mail.gmail.com https://git.postgresql.org/pg/commitdiff/4a8283d0ec5a6781b9e0d4ba16b44da8a0856d02 Tatsuo Ishii pushed: - Doc: fix typo in URL of OASIS group web site. In other places that has been changed from http://www.oasis-open.org/ https://www.oasis-open.org/ but there's a place where the change was missed. Discussion: https://postgr.es/m/20190121.222844.399814306477973879.t-ishii%40sraoss.co.jp https://git.postgresql.org/pg/commitdiff/78855e798325f64e36e2775031137ea83ad0c3ef Heikki Linnakangas pushed: - Fix misc typos in comments. Spotted mostly by Fabien Coelho. Discussion: https://www.postgresql.org/message-id/alpine.DEB.2.21.1901230947050.16643@lancre https://git.postgresql.org/pg/commitdiff/95931133a959404cf7320b49058b2f1c2e4606b9 - Fix comments that claimed that mblen() only looks at first byte. GB18030's mblen() function looks at the first and the second byte of the multibyte character, to determine its length. copy.c had made the assumption that mblen() only looks at the first byte, but it turns out to work out fine, because of the way the GB18030 encoding works. COPY will see a 4-byte encoded character as two 2-byte encoded characters, which is enough for COPY's purposes. It cannot mix those up with delimiter or escaping characters, because only single-byte ASCII characters are supported as delimiters or escape characters. Discussion: https://www.postgresql.org/message-id/7704d099-9643-2a55-fb0e-becd64400dcb%40iki.fi https://git.postgresql.org/pg/commitdiff/a5be6e9a1dfe820807f9ccb21dec5144982618e6 - Use single-byte Boyer-Moore-Horspool search even with multibyte encodings. The old implementation first converted the input strings to arrays of wchars, and performed the conversion on those. However, the conversion is expensive, and for a large input string, consumes a lot of memory. Allocating the large arrays also meant that these functions could not be used on strings larger 1 GB / pg_encoding_max_length() (256 MB for UTF-8). Avoid the conversion, and instead use the single-byte algorithm even with multibyte encodings. That can get fooled, if there is a matching byte sequence in the middle of a multi-byte character, so to eliminate false positives like that, we verify any matches by walking the string character by character with pg_mblen(). Also, if the caller needs the position of the match, as a character-offset, we also need to walk the string to count the characters. Performance testing shows that walking the whole string with pg_mblen() is somewhat slower than converting the whole string to wchars. It's still often a win, though, because we don't need to do it if there is no match, and even when there is, we only need to walk up to the point where the match is, not the whole string. Even in the worst case, there would be room for optimization: Much of the CPU time in the current loop with pg_mblen() is function call overhead, and could be improved by inlining pg_mblen() and/or the encoding-specific mblen() functions. But I didn't attempt to do that as part of this patch. Most of the callers of text_position_setup/next functions were actually not interested in the position of the match, counted in characters. To cater for them, refactor the text_position_next() interface into two parts: searching for the next match (text_position_next()), and returning the current match's position as a pointer (text_position_get_match_ptr()) or as a character offset (text_position_get_match_pos()). Getting the pointer to the match is a more convenient API for many callers, and with UTF-8, it allows skipping the character-walking step altogether, because UTF-8 can't have false matches even when treated like raw byte strings. Reviewed-by: John Naylor Discussion: https://www.postgresql.org/message-id/3173d989-bc1c-fc8a-3b69-f24246f73876%40iki.fi https://git.postgresql.org/pg/commitdiff/9556aa01c69a26ca726d8dda8e395acc7c1e30fc Peter Eisentraut pushed: - Fix whitespace. https://git.postgresql.org/pg/commitdiff/bf2fb2e03ebd897575d7428c73ae6ba7b1f91fc7 - PL/pgSQL: Add statement ID to statement structures. This can be used by a profiler as the index for an array of per-statement metrics. Author: Pavel Stehule <pavel.stehule@gmail.com> Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com> Discussion: https://www.postgresql.org/message-id/flat/CAFj8pRDRCjN6rpM9ZccU7Ta_afsNX7mg9=n34F+r445Nt9v2tA@mail.gmail.com/ https://git.postgresql.org/pg/commitdiff/bbd5c207b960dd072445a3547f16abea0c27f726 - Allow generalized expression syntax for partition bounds. Previously, only literals were allowed. This change allows general expressions, including functions calls, which are evaluated at the time the DDL command is executed. Besides offering some more functionality, it simplifies the parser structures and removes some inconsistencies in how the literals were handled. Author: Kyotaro Horiguchi, Tom Lane, Amit Langote Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com> Discussion: https://www.postgresql.org/message-id/flat/9f88b5e0-6da2-5227-20d0-0d7012beaa1c@lab.ntt.co.jp/ https://git.postgresql.org/pg/commitdiff/7c079d7417a8f2d4bf5144732e2f85117db9214f - Make regression test output locale-independent. In some locales, letters sort before numbers, so change the object naming to not depend on that. Introduced by commit 7c079d7417a8f2d4bf5144732e2f85117db9214f. https://git.postgresql.org/pg/commitdiff/1e4730c639acb967b8e40e6f33e81ebe1721ea5f Bruce Momjian pushed: - SQL comment: remove extra word in heading comment. Reported-by: Daniel Gustafsson Discussion: https://postgr.es/m/431D5BC1-9696-43FA-B54C-39D5503EB753@yesql.se Backpatch-through: master https://git.postgresql.org/pg/commitdiff/df4c9044406f1e907268930dd12ba6c3642d21dd == Pending Patches == Masahiko Sawada sent in a patch to add a DISABLE_INDEX_CLEANUP option to VACUUM. Fnu Nishant sent in another revision of a patch to ensure that locks for heap pages are acquired in a consistent order. Aleksey Kondratov sent in another revision of a patch to pg_rewind to add options to use restore_command from postgresql.conf or command line. Peter Geoghegan sent in a patch to demonstrate how to adjust the harmless objects-to-drop output from DROP ROLE to make relevant tests pass, and another to make the regression tests actively ignore visible manifestations of the current DEPENDENCY_INTERNAL_AUTO bug. Andreas Karlsson sent in two more revisions of a patch to implement temporary materialized views. Kyotaro HORIGUCHI sent in two more revisions of a patch to reimplement the stats collector so it uses shared memory instead of files. Haribabu Kommi sent in two more revisions of a patch to reduce the use of HeapTuple t_tableOid and remove scan_update_snapshot. Amit Langote sent in two more revisions of a patch to speed up planning with partitions. Kyotaro HORIGUCHI sent in two more revisions of a patch to remove old catcache entries, collect syscache statistics and expose them in a new view, and make catcache remote-settable. David Rowley sent in a patch to fix a mismatch between the comments and code in RelationGetIndexAttrBitmap(). Hubert Zhang sent in another revision of a patch to add smgr*_hook() hook points to extend the logic of storage management. Dilip Kumar sent in another revision of a patch to implement logical decoding for operations on zheap tables. Fabien COELHO sent in another revision of a patch to fix an issue where Perl special characters in a path name were being treated differently from other path names. Etsuro Fujita sent in another revision of a patch to improve the algorithm for partition-wise JOINs. James Coleman sent in two more revisions of a patch to enable proving IS NOT NULL inference for ScalarArrayOpExpr's. Pavel Stěhule sent in another revision of a patch to implement schema variables. David Rowley sent in another revision of a patch to allow lock acquisitions for partitions to be delayed. Álvaro Herrera sent in another revision of a patch to make it possible to point foreign key constraints at partitioned tables. Tomáš Vondra sent in two revisions of a patch to separate the per-tuple and per-batch contexts in COPY ... FROM ... WHERE. Michaël Paquier sent in a patch to remove the isprimary argument from index_build(). Surafel Temesgen and David Rowley traded patches to make it possible for pg_dump's --inserts and --column-inserts options to emit multi-VALUES INSERTs. Fabien COELHO sent in another revision of a patch to fix usage of PRNGs in pgbench. Fabien COELHO sent in another revision of a patch to add a Zipf distribution to pgbench. David Rowley sent in another revision of a patch to delay locking of partitions during INSERT and UPDATE. Alexander Korotkov and Nikita Glukhov traded patches to implement JSONPATH. Andreas Karlsson and Tom Lane traded patches to make CTEs inlined by default, with controls to get different behaviors including the legacy "run exactly 0 or 1 times all the way through" behavior. Chris Travers sent in another revision of a patch to clarify the documentation for signal detection. Corey Huinker sent in a patch to change referential integrity check triggers from per-row to per-statement, speeding up the process by up to 25%. Dmitry Dolgov sent in two revisions of a patch to factor out arguments for ArchiveEntry into a new ArchiveOpts structure, to make it more flexible for changes we anticipate making fairly frequently in the near term. Amit Kapila sent in two more revisions of a patch to prevent bootstrap mode from creating files if they don't already exist, and avoid creating FSM files for small heap relations. Peter Geoghegan sent in another revision of a patch to make all nbtree entries unique by having heap TIDs participate in comparisons. Masahiko Sawada sent in another revision of a patch to implement block-level parallel vacuum. Kyotaro HORIGUCHI sent in a patch to make it possible to control whether symbols in external modules are exported. Dilip Kumar sent in another revision of a patch to implement an UNDO worker and ROLLBACK on zheap that can go through same. Konstantin Knizhnik sent in another revision of a patch to implement a built-in connection pooler. Tom Lane sent in two revisions of a patch to enable extensions to supply operator- and function-specific info. Surafel Temesgen sent in another revision of a patch to implement FETCH FIRST ... PERCENT. Corey Huinker sent in a patch to implement \describe in psql, expanding the somewhat cryptic \d syntax into an optional form that's more descriptive. Etsuro Fujita sent in a patch to fix a cost size estimate. John Naylor and Amit Kapila traded patches to conditionally skip transfer of FSMs during pg_upgrade. Takashi Menjo sent in another revision of a patch to add a configure option for PMDK, read and write WALs using PMDK, and enable walreceiver use PMDK. Peter Eisentraut sent in another revision of a patch to implement collations with nondeterministic comparison, of which case-insensitive collation is a special instance. Alexey Klyukin sent in another revision of a patch to add a max_wal_senders GUC. Chapman Flack sent in two more revisions of a patch to fix some infelicities in handling of XML. Adrien Nayrat sent in another revision of a patch to make it possible to log a sample of transactions. Dmitry Dolgov sent in two more revisions of a patch to implement index skip scan a.k.a. "loose index scan." Lætitia Avrot sent in two revisions of a patch to implement log10 and hyperbolic functions for SQL:2016 compliance. Alexey Bashtanov sent in another revision of a patch to log bind parameter values on error.
pgsql-announce by date: