== PostgreSQL Weekly News - October 04 2015 == - Mailing list pgsql-announce
| From | David Fetter |
|---|---|
| Subject | == PostgreSQL Weekly News - October 04 2015 == |
| Date | |
| Msg-id | 20151005003148.GB30093@fetter.org Whole thread Raw |
| List | pgsql-announce |
== PostgreSQL Weekly News - October 04 2015 == PostgreSQL@SCaLE is a two day, two track event which takes place on Jan. 21-22, 2016, at Pasadena Convention Center, as part of SCaLE 14X. The CfP is open until Oct 30, 2015. https://www.socallinuxexpo.org/scale/14x/cfp == PostgreSQL Product News == barman 1.5.0, a backup and recovery manager for PostgreSQL, released. http://www.pgbarman.org/barman-1-5-0-released/ == PostgreSQL Jobs for October == http://archives.postgresql.org/pgsql-jobs/2015-10/ == PostgreSQL Local == PGDay.IT 2015 will take place in Prato on October 23, 2015. http://pgday.it PostgreSQL Conference Europe 2015 will be held on October 27-30 in the Vienna Marriott Hotel, in Vienna, Austria. The schedule is posted. http://2015.pgconf.eu/ PGConf Silicon Valley 2015 is November 17-18 at the South San Francisco Convention Center. http://www.pgconfsv.com PgCUBA will take place October 19-23, 2015 in Havana. Spanish language information below: http://www.postgresql.org/about/event/1813/ PGBR2015 (The Brazilian PostgreSQL Conference) will take place in Porto Alegre, Rio Grande do Sul, on November 18, 19 and 20. Registration is open. http://pgbr.postgresql.org.br/2015/en/ PGConf.DE will be held on November 26-27, 2015 in Hamburg, Germany, at the Lindner Hotel am Michel. http://2015.pgconf.de/ == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm Pacific time. Please send English language ones to david@fetter.org, German language to pwn@pgug.de, Italian language to pwn@itpug.org. Spanish language to pwn@arpug.com.ar. == Applied Patches == Peter Eisentraut pushed: - pg_dump: Fix some messages. Make quoting style match existing style. Improve plural support. http://git.postgresql.org/pg/commitdiff/883af819c1ef7ec975f5c180a3434ff690c23606 - Fix message punctuation according to style guide http://git.postgresql.org/pg/commitdiff/87c2b517accf287cb746b58af570e7a29b5ad26a - pg_rewind: Improve some messages. The output of a typical pg_rewind run contained a mix of capitalized and not-capitalized and punctuated and not-punctuated phrases for no apparent reason. Make that consistent. Also fix some problems in other messages. http://git.postgresql.org/pg/commitdiff/cdcae2b6a7d263b2771b6386af626b47ce17da19 - doc: Make some index terms and terminology more consistent http://git.postgresql.org/pg/commitdiff/eff091cc19118c1be7f692c969ca3ebb6994552a - doc: Update URLs of external projects http://git.postgresql.org/pg/commitdiff/1023194b7aba88680a5c943d5122d086c3431d19 - Group cluster_name and update_process_title settings together http://git.postgresql.org/pg/commitdiff/6390c8c654d07c08686adbbc595a13d76b573653 Robert Haas pushed: - Don't try to create a temp install without abs_top_builddir. Otherwise, we effectively act as if abs_top_builddir were the root directory, which is quite dangerous if the user happens to have permissions to do things there. This can crop up in PGXS builds, for example. Report by Sandro Santilli, patch by me, review by Noah Misch. http://git.postgresql.org/pg/commitdiff/45e5b4ef5c392ddfcd72693f65659bafdda7ee1b - Use LOCKBIT_ON() instead of a bit shift in a few places. We do this mostly everywhere, so it seems just as well to do it here, too. Thomas Munro http://git.postgresql.org/pg/commitdiff/f40792a93ccdc4d01c448121da67693fc2f6297a - Parallel executor support. This code provides infrastructure for a parallel leader to start up parallel workers to execute subtrees of the plan tree being executed in the master. User-supplied parameters from ParamListInfo are passed down, but PARAM_EXEC parameters are not. Various other constructs, such as initplans, subplans, and CTEs, are also not currently shared. Nevertheless, there's enough here to support a basic implementation of parallel query, and we can lift some of the current restrictions as needed. Amit Kapila and Robert Haas http://git.postgresql.org/pg/commitdiff/d1b7c1ffe72e86932b5395f29e006c3f503bc53d - Comment update for join pushdown. Etsuro Fujita http://git.postgresql.org/pg/commitdiff/758fcfdc016ac687c4f4094e0dca6ae641400820 - Don't dump core when destroying an unused ParallelContext. If a transaction or subtransaction creates a ParallelContext but ends without calling InitializeParallelDSM, the previous code would seg fault. Fix that. http://git.postgresql.org/pg/commitdiff/227d57f3587d7d2a7d0792011f5ac952ba763681 - Add a Gather executor node. A Gather executor node runs any number of copies of a plan in an equal number of workers and merges all of the results into a single tuple stream. It can also run the plan itself, if the workers are unavailable or haven't started up yet. It is intended to work with the Partial Seq Scan node which will be added in future commits. It could also be used to implement parallel query of a different sort by itself, without help from Partial Seq Scan, if the single_copy mode is used. In that mode, a worker executes the plan, and the parallel leader does not, merely collecting the worker's results. So, a Gather node could be inserted into a plan to split the execution of that plan across two processes. Nested Gather nodes aren't currently supported, but we might want to add support for that in the future. There's nothing in the planner to actually generate Gather nodes yet, so it's not quite time to break out the champagne. But we're getting close. Amit Kapila. Some designs suggestions were provided by me, and I also reviewed the patch. Single-copy mode, documentation, and other minor changes also by me. http://git.postgresql.org/pg/commitdiff/3bd909b220930f21d6e15833a17947be749e7fde - Fix readfuncs/outfuncs problems in last night's Gather patch. KaiGai Kohei, with one correction by me. http://git.postgresql.org/pg/commitdiff/286a3a68dcafa15f28a169daac38659816911940 - Clarify FDW documentation about ON CONFLICT. Etsuro Fujita, reviewed by Peter Geoghegan http://git.postgresql.org/pg/commitdiff/01bc589a464964475a0718497ebd21af5bc7338e Andres Freund pushed: - Fix ON CONFLICT DO UPDATE for tables with oids. When taking the UPDATE path in an INSERT .. ON CONFLICT .. UPDATE tables with oids were not supported. The tuple generated by the update target list was projected without space for an oid - a simple oversight. Reported-By: Peter Geoghegan Author: Andres Freund Backpatch: 9.5, where ON CONFLICT was introduced http://git.postgresql.org/pg/commitdiff/617db3a2d8ad6abc0d7be623f43edc943a6640a8 - Fix several bugs related to ON CONFLICT's EXCLUDED pseudo relation. Four related issues: 1) attnos/varnos/resnos for EXCLUDED were out of sync when a column after one dropped in the underlying relation was referenced. 2) References to whole-row variables (i.e. EXCLUDED.*) lead to errors. 3) It was possible to reference system columns in the EXCLUDED pseudo relations, even though they would not have valid contents. 4) References to EXCLUDED were rewritten by the RLS machinery, as EXCLUDED was treated as if it were the underlying relation. To fix the first two issues, generate the excluded targetlist with dropped columns in mind and add an entry for whole row variables. Instead of unconditionally adding a wholerow entry we could pull up the expression if needed, but doing it unconditionally seems simpler. The wholerow entry is only really needed for ruleutils/EXPLAIN support anyway. The remaining two issues are addressed by changing the EXCLUDED RTE to have relkind = composite. That fits with EXCLUDED not actually being a real relation, and allows to treat it differently in the relevant places. scanRTEForColumn now skips looking up system columns when the RTE has a composite relkind; fireRIRrules() already had a corresponding check, thereby preventing RLS expansion on EXCLUDED. Also add tests for these issues, and improve a few comments around excluded handling in setrefs.c. Reported-By: Peter Geoghegan, Geoff Winkless. Author: Andres Freund, Amit Langote, Peter Geoghegan. Discussion: CAEzk6fdzJ3xYQZGbcuYM2rBd2BuDkUksmK=mY9UYYDugg_GgZg@mail.gmail.com, CAM3SWZS+CauzbiCEcg-GdE6K6ycHE_Bz6Ksszy8AoixcMHOmsA@mail.gmail.com. Backpatch: 9.5, where ON CONFLICT was introduced http://git.postgresql.org/pg/commitdiff/ad227837924464eb9a7defbe34e95064a3b1bdc2 - Improve errhint() about replication slot naming restrictions. The existing hint talked about "may only contain letters", but the actual requirement is more strict: only lower case letters are allowed. Reported-By: Rushabh Lathia. Author: Rushabh Lathia. Discussion: AGPqQf2x50qcwbYOBKzb4x75sO_V3g81ZsA8+Ji9iN5t_khFhQ@mail.gmail.com. Backpatch: 9.4-, where replication slots were added http://git.postgresql.org/pg/commitdiff/920218cbc0a36f18793c20184b7ebb999d28e8c8 - Add missed CREATE EXTENSION ... CASCADE regression test adjustment. http://git.postgresql.org/pg/commitdiff/23fc0b485d6dfa49ff20fff6c5661be4b12a5bff - Add CASCADE support for CREATE EXTENSION. Without CASCADE, if an extension has an unfullfilled dependency on another extension, CREATE EXTENSION ERRORs out with "required extension ... is not installed". That is annoying, especially when that dependency is an implementation detail of the extension, rather than something the extension's user can make sense of. In addition to CASCADE this also includes a small set of regression tests around CREATE EXTENSION. Author: Petr Jelinek, editorialized by Michael Paquier, Andres Freund Reviewed-By: Michael Paquier, Andres Freund, Jeff Janes Discussion: 557E0520.3040800@2ndquadrant.com http://git.postgresql.org/pg/commitdiff/b67aaf21e8ef8273d5179a8832a109153d7adfe1 - Fix hstore_plpython test when python3 is used. Due to b67aaf21e8ef8 / CREATE EXTENSION ... CASCADE the test output contains the extension name in yet another place. Since that's variable depending on the python version... Add yet another name mangling stanza to regress-python3-mangle.mk. Author: Petr Jelinek http://git.postgresql.org/pg/commitdiff/86b1e6784b19b03606197cf3f69b10f5a653931d Álvaro Herrera pushed: - COPY: use pg_plan_query() instead of planner(). While at it, trim the includes list in copy.c. The planner headers cannot be removed, but there are a few others that are not of any use. http://git.postgresql.org/pg/commitdiff/590e2d12f0e17aed39a64050f2dddc0d9f56c1ba - Fix "sesssion" typo. It was introduced alongside replication origins, by commit 5aa2350426c, so backpatch to 9.5. Pointed out by Fujii Masao http://git.postgresql.org/pg/commitdiff/17f5831c8150349d22da2e88820b1be3d734315d - Code review for transaction commit timestamps. There are three main changes here: 1. No longer cause a start failure in a standby if the feature is disabled in postgresql.conf but enabled in the master. This reverts one part of commit 4f3924d9cd43; what we keep is the ability of the standby to activate/deactivate the module (which includes creating and removing segments as appropriate) during replay of such actions in the master. 2. Replay WAL records affecting commitTS even if the feature is disabled. This means the standby will always have the same state as the master after replay. 3. Have COMMIT PREPARE record the transaction commit time as well. We were previously only applying it in the normal transaction commit path. Author: Petr Jelínek Discussion: http://www.postgresql.org/message-id/CAHGQGwHereDzzzmfxEBYcVQu3oZv6vZcgu1TPeERWbDc+gQ06g@mail.gmail.com Discussion: http://www.postgresql.org/message-id/CAHGQGwFuzfO4JscM9LCAmCDCxp_MfLvN4QdB+xWsS-FijbjTYQ@mail.gmail.com Additionally, I cleaned up nearby code related to replication origins, which I found a bit hard to follow, and fixed a couple of typos. Backpatch to 9.5, where this code was introduced. Per bug reports from Fujii Masao and subsequent discussion. http://git.postgresql.org/pg/commitdiff/6b61955135e94b39d85571fdbb0c5a749af767f1 - Fix commit_ts for standby Module initialization was still not completely correct after commit 6b61955135e9, per crash report from Takashi Ohnishi. To fix, instead of trying to monkey around with the value of the GUC setting directly, add a separate boolean flag that enables the feature on a standby, but only for the startup (recovery) process, when it sees that its master server has the feature enabled. Discussion: http://www.postgresql.org/message-id/ca44c6c7f9314868bdc521aea4f77cbf@MP-MSGSS-MBX004.msg.nttdata.co.jp Also change the deactivation routine to delete all segment files rather than leaving the last one around. (This doesn't need separate WAL-logging, because on recovery we execute the same deactivation routine anyway.) In passing, clean up the code structure somewhat, particularly so that xlog.c doesn't know so much about when to activate/deactivate the feature. Thanks to Fujii Masao for testing and Petr Jelínek for off-list discussion. Back-patch to 9.5, where commit_ts was introduced. http://git.postgresql.org/pg/commitdiff/f12e814b88d8082804bbc8b827469d8068e7252c - Don't disable commit_ts in standby if enabled locally. Bug noticed by Fujii Masao http://git.postgresql.org/pg/commitdiff/e06b2e1d2ecde5cef469b9eac3dc5bf825a29a26 Stephen Frost pushed: - Ensure a few policies remain for pg_upgrade. To make sure that pg_dump/pg_restore function properly with RLS policies, arrange to have a few of them left around at the end of the regression tests. Back-patch to 9.5 where RLS was added. http://git.postgresql.org/pg/commitdiff/992d702bfafdb4258e65cbfa7353a631460a8389 - Include policies based on ACLs needed. When considering which policies should be included, rather than look at individual bits of the query (eg: if a RETURNING clause exists, or if a WHERE clause exists which is referencing the table, or if it's a FOR SHARE/UPDATE query), consider any case where we've determined the user needs SELECT rights on the relation while doing an UPDATE or DELETE to be a case where we apply SELECT policies, and any case where we've deteremind that the user needs UPDATE rights on the relation while doing a SELECT to be a case where we apply UPDATE policies. This simplifies the logic and addresses concerns that a user could use UPDATE or DELETE with a WHERE clauses to determine if rows exist, or they could use SELECT .. FOR UPDATE to lock rows which they are not actually allowed to modify through UPDATE policies. Use list_append_unique() to avoid adding the same quals multiple times, as, on balance, the cost of checking when adding the quals will almost always be cheaper than keeping them and doing busywork for each tuple during execution. Back-patch to 9.5 where RLS was added. http://git.postgresql.org/pg/commitdiff/7d8db3e8f37aec9d252353904e77381a18a2fa9f Tom Lane pushed: - Fix poor errno handling in libpq's version of our custom OpenSSL BIO. Thom Brown reported that SSL connections didn't seem to work on Windows in 9.5. Asif Naeem figured out that the cause was my_sock_read() looking at "errno" when it needs to look at "SOCK_ERRNO". This mistake was introduced in commit 680513ab79c7e12e402a2aad7921b95a25a4bcc8, which cloned the backend's custom SSL BIO code into libpq, and didn't translate the errno handling properly. Moreover, it introduced unnecessary errno save/restore logic, which was particularly confusing because it was incomplete; and it failed to check for all three of EINTR, EAGAIN, and EWOULDBLOCK in my_sock_write. (That might not be necessary; but since we're copying well-tested backend code that does do that, it seems prudent to copy it faithfully.) http://git.postgresql.org/pg/commitdiff/60f1e6bc130241c1f1da337d0a08a05ee8b982d8 - Fix plperl to handle non-ASCII error message texts correctly. We were passing error message texts to croak() verbatim, which turns out not to work if the text contains non-ASCII characters; Perl mangles their encoding, as reported in bug #13638 from Michal Leinweber. To fix, convert the text into a UTF8-encoded SV first. It's hard to test this without risking failures in different database encodings; but we can follow the lead of plpython, which is already assuming that no-break space (U+00A0) has an equivalent in all encodings we care about running the regression tests in (cf commit 2dfa15de5). Back-patch to 9.1. The code is quite different in 9.0, and anyway it seems too risky to put something like this into 9.0's final minor release. Alex Hunsaker, with suggestions from Tim Bunce and Tom Lane http://git.postgresql.org/pg/commitdiff/b631a46ed83b7eebf5cde16b41d842596cbcc69d - Small improvements in comments in async.c. We seem to have lost a line somewhere along the way in the comment block that discusses async.c's locks, because it suddenly refers to "both locks" without previously having mentioned more than one. Add a sentence to make that read more sanely. Also, refer to the "pos of the slowest backend" not the "tail of the slowest backend", since we have no per-backend value called "tail". http://git.postgresql.org/pg/commitdiff/6057f61b4d511d8321f295d827ced13809e5a1ae - Improve LISTEN startup time when there are many unread notifications. If some existing listener is far behind, incoming new listener sessions would start from that session's read pointer and then need to advance over many already-committed notification messages, which they have no interest in. This was expensive in itself and also thrashed the pg_notify SLRU buffers a lot more than necessary. We can improve matters considerably in typical scenarios, without much added cost, by starting from the furthest-ahead read pointer, not the furthest-behind one. We do have to consider only sessions in our own database when doing this, which requires an extra field in the data structure, but that's a pretty small cost. Back-patch to 9.0 where the current LISTEN/NOTIFY logic was introduced. Matt Newell, slightly adjusted by me http://git.postgresql.org/pg/commitdiff/07e4d03fb476587bd943b4ba7d51bf0bb559a631 - Fix errors in commit a04bb65f70dafdf462e0478ad19e6de56df89bfc. Not a lot of commentary needed here really. http://git.postgresql.org/pg/commitdiff/5884b92a841d83ceadb986843892d57c68ea32cd - Fix documentation error in commit 8703059c6b55c427100e00a09f66534b6ccbfaa1. Etsuro Fujita spotted a thinko in the README commentary. http://git.postgresql.org/pg/commitdiff/21995d3f6d69670b5bc5b3654970701f8abb6d9f - Fix pg_dump to handle inherited NOT VALID check constraints correctly. This case seems to have been overlooked when unvalidated check constraints were introduced, in 9.2. The code would attempt to dump such constraints over again for each child table, even though adding them to the parent table is sufficient. In 9.2 and 9.3, also fix contrib/pg_upgrade/Makefile so that the "make clean" target fully cleans up after a failed test. This evidently got dealt with at some point in 9.4, but it wasn't back-patched. I ran into it while testing this fix ... Per bug #13656 from Ingmar Brouns. http://git.postgresql.org/pg/commitdiff/8ab4a6bd3f1a300313eab04d52b0c9a4003f3e43 - Docs: add another example of creating a range type. The "floatrange" example is a bit too simple because float8mi can be used without any additional type conversion. Add an example that does have to account for that, and do some minor other wordsmithing. http://git.postgresql.org/pg/commitdiff/27fddec197857db4d54db50a63f311bb0ed7b6fd - Docs: add disclaimer about hazards of using regexps from untrusted sources. It's not terribly hard to devise regular expressions that take large amounts of time and/or memory to process. Recent testing by Greg Stark has also shown that machines with small stack limits can be driven to stack overflow by suitably crafted regexps. While we intend to fix these things as much as possible, it's probably impossible to eliminate slow-execution cases altogether. In any case we don't want to treat such things as security issues. The history of that code should already discourage prudent DBAs from allowing execution of regexp patterns coming from possibly-hostile sources, but it seems like a good idea to warn about the hazard explicitly. Currently, similar_escape() allows access to enough of the underlying regexp behavior that the warning has to apply to SIMILAR TO as well. We might be able to make it safer if we tightened things up to allow only SQL-mandated capabilities in SIMILAR TO; but that would be a subtly non-backwards-compatible change, so it requires discussion and probably could not be back-patched. Per discussion among pgsql-security list. http://git.postgresql.org/pg/commitdiff/558d4ada1851274fe4dd3618f3f6561b63857e8f - Add some more query-cancel checks to regular expression matching. Commit 9662143f0c35d64d7042fbeaf879df8f0b54be32 added infrastructure to allow regular-expression operations to be terminated early in the event of SIGINT etc. However, fuzz testing by Greg Stark disclosed that there are still cases where regex compilation could run for a long time without noticing a cancel request. Specifically, the fixempties() phase never adds new states, only new arcs, so it doesn't hit the cancel check I'd put in newstate(). Add one to newarc() as well to cover that. Some experimentation of my own found that regex execution could also run for a long time despite a pending cancel. We'd put a high-level cancel check into cdissect(), but there was none inside the core text-matching routines longest() and shortest(). Ordinarily those inner loops are very very fast ... but in the presence of lookahead constraints, not so much. As a compromise, stick a cancel check into the stateset cache-miss function, which is enough to guarantee a cancel check at least once per lookahead constraint test. Making this work required more attention to error handling throughout the regex executor. Henry Spencer had apparently originally intended longest() and shortest() to be incapable of incurring errors while running, so neither they nor their subroutines had well-defined error reporting behaviors. However, that was already broken by the lookahead constraint feature, since lacon() can surely suffer an out-of-memory failure --- which, in the code as it stood, might never be reported to the user at all, but just silently be treated as a non-match of the lookahead constraint. Normalize all that by inserting explicit error tests as needed. I took the opportunity to add some more comments to the code, too. Back-patch to all supported branches, like the previous patch. http://git.postgresql.org/pg/commitdiff/9fe8fe9c9e5d7fc099acfc96e976ee72b2b49865 - Fix potential infinite loop in regular expression execution. In cfindloop(), if the initial call to shortest() reports that a zero-length match is possible at the current search start point, but then it is unable to construct any actual match to that, it'll just loop around with the same start point, and thus make no progress. We need to force the start point to be advanced. This is safe because the loop over "begin" points has already tried and failed to match starting at "close", so there is surely no need to try that again. This bug was introduced in commit e2bd904955e2221eddf01110b1f25002de2aaa83, wherein we allowed continued searching after we'd run out of match possibilities, but evidently failed to think hard enough about exactly where we needed to search next. Because of the way this code works, such a match failure is only possible in the presence of backrefs --- otherwise, shortest()'s judgment that a match is possible should always be correct. That probably explains how come the bug has escaped detection for several years. The actual fix is a one-liner, but I took the trouble to add/improve some comments related to the loop logic. After fixing that, the submitted test case "()*\1" didn't loop anymore. But it reported failure, though it seems like it ought to match a zero-length string; both Tcl and Perl think it does. That seems to be from overenthusiastic optimization on my part when I rewrote the iteration match logic in commit 173e29aa5deefd9e71c183583ba37805c8102a72: we can't just "declare victory" for a zero-length match without bothering to set match data for capturing parens inside the iterator node. Per fuzz testing by Greg Stark. The first part of this is a bug in all supported branches, and the second part is a bug since 9.2 where the iteration rewrite happened. http://git.postgresql.org/pg/commitdiff/f2c4ffc3307cab6619a28e77da9211416c8b1d83 - Add recursion depth protections to regular expression matching. Some of the functions in regex compilation and execution recurse, and therefore could in principle be driven to stack overflow. The Tcl crew has seen this happen in practice in duptraverse(), though their fix was to put in a hard-wired limit on the number of recursive levels, which is not too appetizing --- fortunately, we have enough infrastructure to check the actually available stack. Greg Stark has also seen it in other places while fuzz testing on a machine with limited stack space. Let's put guards in to prevent crashes in all these places. Since the regex code would leak memory if we simply threw elog(ERROR), we have to introduce an API that checks for stack depth without throwing such an error. Fortunately that's not difficult. http://git.postgresql.org/pg/commitdiff/b63fc28776c5d2efdb4de326ad0f0b5b88f82220 - Add recursion depth protection to LIKE matching. Since MatchText() recurses, it could in principle be driven to stack overflow, although quite a long pattern would be needed. http://git.postgresql.org/pg/commitdiff/2e8cfcf4ea1140213eae56ce64a2c53182643578 - Update time zone data files to tzdata release 2015g. DST law changes in Cayman Islands, Fiji, Moldova, Morocco, Norfolk Island, North Korea, Turkey, Uruguay. New zone America/Fort_Nelson for Canadian Northern Rockies. http://git.postgresql.org/pg/commitdiff/241e6844ad7643783e9b4bc020d2abe6a9cbfed2 - Add missing "static" specifier. Per buildfarm (pademelon, at least, doesn't like this). http://git.postgresql.org/pg/commitdiff/bf686796a055a8bd4abc9765763eb675a0428473 - First-draft release notes for 9.4.5, 9.3.10, 9.2.14, 9.1.19, 9.0.23. http://git.postgresql.org/pg/commitdiff/01ef33701bf6d475deeb550c18a5c3fd698c9623 - Update 9.5 release notes through today. http://git.postgresql.org/pg/commitdiff/cf007a4bca59a1bca6951351e10ff227d2b0c10a - Fix some issues in new hashtable size calculations in nodeHash.c. Limit the size of the hashtable pointer array to not more than MaxAllocSize, per reports from Kouhei Kaigai and others of "invalid memory alloc request size" failures. There was discussion of allowing the array to get larger than that by using the "huge" palloc API, but so far no proof that that is actually a good idea, and at this point in the 9.5 cycle major changes from old behavior don't seem like the way to go. Fix a rather serious secondary bug in the new code, which was that it didn't ensure nbuckets remained a power of 2 when recomputing it for the multiple-batch case. Clean up sloppy division of labor between ExecHashIncreaseNumBuckets and its sole call site. http://git.postgresql.org/pg/commitdiff/a31e64d0652622d4627ffa2123163a43538a3f51 - Further twiddling of nodeHash.c hashtable sizing calculation. On reflection, the submitted patch didn't really work to prevent the request size from exceeding MaxAllocSize, because of the fact that we'd happily round nbuckets up to the next power of 2 after we'd limited it to max_pointers. The simplest way to enforce the limit correctly is to round max_pointers down to a power of 2 when it isn't one already. (Note that the constraint to INT_MAX / 2, if it were doing anything useful at all, is properly applied after that.) http://git.postgresql.org/pg/commitdiff/f2fc98fb8e060b2243aba4cfe8b206f54bcc5a4f - Improve contrib/pg_stat_statements' handling of garbage collection failure. If we can't read the query texts file (whether because out-of-memory, or for some other reason), give up and reset the file to empty, discarding all stored query texts, though not the statistics per se. We used to leave things alone and hope for better luck next time, but the problem is that the file is only going to get bigger and even harder to slurp into memory. Better to do something that will get us out of trouble. Likewise reset the file to empty for any other failure within gc_qtexts(). The previous behavior after a write error was to discard query texts but not do anything to truncate the file, which is just weird. Also, increase the maximum supported file size from MaxAllocSize to MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage collection successfully. Also, fix recalculation of mean_query_len within entry_dealloc() to match the calculation in gc_qtexts(). The previous coding overlooked the possibility of dropped texts (query_len == -1) and would underestimate the mean of the remaining entries in such cases, thus possibly causing excess garbage collection cycles. In passing, add some errdetail to the log entry that complains about insufficient memory to read the query texts file, which after all was Jim Nasby's original complaint. Back-patch to 9.4 where the current handling of query texts was introduced. Peter Geoghegan, rather editorialized upon by me http://git.postgresql.org/pg/commitdiff/8bbe4cbd9bd687eddfb8fd8f7bba488ff794d166 - Release notes for 9.5beta1, 9.4.5, 9.3.10, 9.2.14, 9.1.19, 9.0.23. http://git.postgresql.org/pg/commitdiff/16a70e3059885739f59ccdaa20f2e4a3b2a0a700 Andrew Dunstan pushed: - Fix compiler warning about unused function in non-readline case. Backpatch to all live branches to keep the code in sync. http://git.postgresql.org/pg/commitdiff/8a0aa686f41f68c03683a9e37ba783a20cf53a97 - Fix compiler warning for non-TIOCGWINSZ case Backpatch to 9.5 where the error was introduced. http://git.postgresql.org/pg/commitdiff/0557dc276f1022965f72dc8bcfc820dfd83a7dc2 - Disallow invalid path elements in jsonb_set Null path elements and, where the object is an array, invalid integer elements now cause an error. Incorrect behaviour noted by Thom Brown, patch from Dmitry Dolgov. Backpatch to 9.5 where jsonb_set was introduced http://git.postgresql.org/pg/commitdiff/1edd4ec831458e10b524d1473a7de5791aa8753e Tatsuo Ishii pushed: - Fix incorrect tps number calculation in "excluding connections establishing". The tolerance (larger than actual tps number) increases as the number of threads decreases. The bug has been there since the thread support was introduced in 9.0. Because back patching introduces incompatible behavior changes regarding the tps number, the fix is committed to master and 9.5 stable branches only. Problem spotted by me and fix proposed by Fabien COELHO. Note that his original patch included more than fixes (a code re-factoring) which is not related to the problem and I omitted the part. http://git.postgresql.org/pg/commitdiff/a16db3a07da9c47a5ef5b1d18f9c45bdb893b243 Fujii Masao pushed: - Fix mention of htup.h in storage.sgml. Previously it was documented that the details on HeapTupleHeaderData struct could be found in htup.h. This is not correct because it's now defined in htup_details.h. Back-patch to 9.3 where the definition of HeapTupleHeaderData struct was moved from htup.h to htup_details.h. Michael Paquier http://git.postgresql.org/pg/commitdiff/3123ee0db2982be545a0d403b543ed4518403ca1 - Fix incorrect tab-completion for GRANT and REVOKE. Previously "GRANT * ON * TO " was tab-completed to add an extra "TO", rather than with a list of roles. This is the bug that commit 2f88807 introduced unexpectedly. This commit fixes that incorrect tab-completion. Thomas Munro, reviewed by Jeff Janes. http://git.postgresql.org/pg/commitdiff/bf4817e4f090a0a72d1849a3d61b45e7f2feadda Noah Misch pushed: - Make BYPASSRLS behave like superuser RLS bypass. Specifically, make its effect independent from the row_security GUC, and make it affect permission checks pertinent to views the BYPASSRLS role owns. The row_security GUC thereby ceases to change successful-query behavior; it can only make a query fail with an error. Back-patch to 9.5, where BYPASSRLS was introduced. http://git.postgresql.org/pg/commitdiff/3cb0a7e75aaa9a7826c769068970ce2200e61023 - Document that row_security is a boolean GUC. Oversight in commit 537bd178c73b1d25938347b17e9e3e62898fc231. Back-patch to 9.5, like that commit. http://git.postgresql.org/pg/commitdiff/f78ae3747d8bdf07d864e465eb257ad5b7fd825a == Rejected Patches (for now) == No one was disappointed this week :-) == Pending Patches == Amir Rohan sent in another revision of a patch to better document base backups. David Rowley sent in two more revisions of a patch to allow GROUP BY before JOIN, which can immensely optimize performance of certain queries. Marti Raudsepp sent in a patch to skip ALTER x SET SCHEMA if the schema didn't change, rather than erroring out. Fabien COELHO sent in another revision of a patch to correct some TPS numbers in pgbench. Kaigai Kouhei sent in two more revisions of a patch to fix an infelicity between foreign join pushdown and EvalPlanQual. Alexander Shulgin sent in another revision of a patch to add an SHM table of contents to the explain DSM. Fabien COELHO sent in two more revisions of a patch to add more features to pgbench. Paul Ramsey and Michael Paquier traded patches to add remote extension support to the PostgreSQL FDW. Michael Paquier sent in another revision of a patch to add a test facility to check dump/restore with extensions. David Rowley sent in another revision of a patch to use foreign keys to improve join estimates. Joshua Elsasser sent in a flock of patches to add a single tar output format to pg_basebackup, fixing some bugs along the way. Adam Brightwell sent in a patch to document a possible information leak inherent in how tuple visibility works. Michael Paquier sent in another revision of a patch to handle timeline switches in pg_rewind. Tomas Vondra sent in two more revisions of a patch to allow index-only scans to use partial indexes. Matt Newell sent in two revisions of a patch to fix a LISTEN denial of service with aborted transaction. Nikolay Shaplov and Michael Paquier traded patches to add an ability to show tuple data to the pageinspect utility. Tom Lane sent in two revisions of a patch to make the postmaster die if $PGDATA disappears. Stephen Frost sent in another revision of a patch to reserve the namespaces starting with pg_ for roles. Michael Paquier sent in another revision of a patch to remove obsolete use of volatile in WAL-related files. SAWADA Masahiko sent in two more revisions of a patch to add a "frozen" bit to the visibility map. Dinesh Kumar sent in a patch to allow setting permissions on the output file COPY produces via the new syntax, COPY ... [[NO] READ ACCESS]. Alexander Korotkov sent in another revision of a patch to rework the access method interface. Michael Paquier sent in another revision of a patch to add in-core tests for recovery. Peter Geoghegan sent in two revisions of a patch to fix an issue that manifested as an error reporting issue in pg_stat_statements. Peter Geoghegan sent in a patch to extend SortSupport for text with strcoll() and strxfrm() caching. Peter Geoghegan sent in a patch to add SortSupport for the UUID type. Michael Paquier sent in two more revisions of a patch to add in-core tests for recovery. Marko Tiikkaja sent in a patch to add a CONTEXT line to index and constraint rebuilds.
pgsql-announce by date: