== PostgreSQL Weekly News - March 27 2016 == - Mailing list pgsql-announce
| From | David Fetter |
|---|---|
| Subject | == PostgreSQL Weekly News - March 27 2016 == |
| Date | |
| Msg-id | 20160327230616.GB29591@fetter.org Whole thread Raw |
| List | pgsql-announce |
== PostgreSQL Weekly News - March 27 2016 == == PostgreSQL Product News == Ora2PG 17.2 a migration system for moving from Oracle and MySQL to PostgreSQL, released. https://github.com/darold/ora2pg/blob/master/changelog PostGIS 2.2.2, the industry standard geographic information system for PostgreSQL, released. http://postgis.net/2016/03/22/postgis-2.2.2/ PyGreSQL 5.0, a Python driver for PostgreSQL, released. http://www.pygresql.org/contents/changelog.html Veil 9.5.1, a fine-grained access control system for PostgreSQL, released. Project page and download link below: https://github.com/marcmunro/veil http://pgfoundry.org/frs/shownotes.php?release_id=2119 == PostgreSQL Jobs for March == http://archives.postgresql.org/pgsql-jobs/2016-03/ == PostgreSQL Local == Nordic PGDay 2016 is a one day one track conference which will be held in Helsinki, Finland, on March 17, 2016. Registration is still open. http://2016.nordicpgday.org/ PGDay Paris 2016, on March 31 in Paris, has opened registration. http://www.pgday.paris/registration/ (scroll down for English) The 8th PostgreSQL Session will be held on April 6th, 2016, in Lyon, France. <AT> postgresql-sessions <DOT> org. The first meeting of PostgreSQL Israel will be taking place in Tel Aviv on April 7, 2016. http://www.meetup.com/PostgreSQL-Israel/events/229430729/ PGConf US 2016 will take place April 18-20, 2016 in NYC. Registration is open. http://www.pgconf.us/2016/ LinuxFest Northwest will take place April 23-24, 2016 at Bellingham Technical College in Bellingham, Washington, USA. The CfP is now open. http://www.linuxfestnorthwest.org/2016/present FOSS4G NA, will be held May 2-5, 2016 in Raleigh, North Carolina. The CfP is still open. https://2016.foss4g-na.org/cfp PGCon 2016 will be held May 17-21, 2016 in Ottawa. http://www.pgcon.org/ This year's Swiss PGDay will be held on June 24, 2016 at the University of Applied Sciences in Rapperswil (Switzerland). The CfP is open. http://www.pgday.ch/ "5432 ... Meet us!", will take place in Milan, Italy on June 28-29, 2016. The CfP is open until March 14, 2016. http://5432meet.us/ PG Day UK 2016 - Call for Papers is open until 5th April 2016. The conference will be 5th July 2016. Please submit proposals to Simon.Riggs@2ndquadrant.com <Simon AT 2ndquadrant DOT com> or via the website at: http://www.pgconf.uk/papers PgConf Silicon Valley 2016 will be held on November 14-16, 2016. http://www.pgconfsv.com/ == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm Pacific time. Please send English language ones to david@fetter.org, German language to pwn@pgug.de, Italian language to pwn@itpug.org. Spanish language to pwn@arpug.com.ar. == Applied Patches == Tom Lane pushed: - Teach the configure script to validate its --with-pgport argument. Previously, configure would take any string, including an empty string, leading to obscure compile failures in guc.c. It seems worth expending a few lines of code to ensure that the argument is a decimal number between 1 and 65535. Report and patch by Jim Nasby; reviews by Alex Shulgin, Peter Eisentraut, Ivan Kartyshov http://git.postgresql.org/pg/commitdiff/bf53d5c208a3bdce243a38666fc50f5418c78c3b - pg_stat_get_progress_info() should be marked STRICT. I didn't bother with a catversion bump. Report and patch by Thomas Munro http://git.postgresql.org/pg/commitdiff/2da75499879032d8d2f233ca42cc2efe48fd76ef - Use repalloc_huge() to enlarge a SPITupleTable's tuple pointer array. Commit 23a27b039d94ba35 widened the rows-stored counters to uint64, but that's academic unless we allow the tuple pointer array to exceed 1GB. (It might be a good idea to provide some other limit on how much storage a SPITupleTable can eat. On the other hand, there are plenty of other ways to drive a backend into swap hell.) Dagfinn Ilmari Mannsåker http://git.postgresql.org/pg/commitdiff/74a379b984d4df91acec2436a16c51caee3526af - Improve conversions from uint64 to Perl types. Perl's integers are pointer-sized, so can hold more than INT_MAX on LP64 platforms, and come in both signed (IV) and unsigned (UV). Floating point values (NV) may also be larger than double. Since Perl 5.19.4 array indices are SSize_t instead of I32, so allow up to SSize_t_max on those versions. The limit is not imposed just by av_extend's argument type, but all the array handling code, so remove the speculative comment. Dagfinn Ilmari Mannsåker http://git.postgresql.org/pg/commitdiff/f3f3aae4b7841f4dc51129691a7404a03eb55449 - Update PL/Perl's comment about hv_store(). Negative klen is documented since Perl 5.16, and 5.6 is no longer supported so no need to comment about it. Dagfinn Ilmari Mannsåker http://git.postgresql.org/pg/commitdiff/07341a2980a37ccbb3a51af2bd2f3c87953d8ea4 - Rethink representation of PathTargets. In commit 19a541143a09c067 I did not make PathTarget a subtype of Node, and embedded a RelOptInfo's reltarget directly into it rather than having a separately-allocated Node. In hindsight that was misguided micro-optimization, enabled by the fact that at that point we didn't have any Paths with custom PathTargets. Now that PathTarget processing has been fleshed out some more, it's easier to see that it's better to have PathTarget as an indepedent Node type, even if it does cost us one more palloc to create a RelOptInfo. So change it while we still can. This commit just changes the representation, without doing anything more interesting than that. http://git.postgresql.org/pg/commitdiff/307c78852f516042cebacaed411a0391bfeb2129 - Allow callers of create_foreignscan_path to specify nondefault PathTarget. Although the default choice of rel->reltarget should typically be sufficient for scan or join paths, it's not at all sufficient for the purposes PathTargets were invented for; in particular not for upper-relation Paths. So break API compatibility by adding a PathTarget argument to create_foreignscan_path(). To ease updating of existing code, accept a NULL value of the argument as selecting rel->reltarget. http://git.postgresql.org/pg/commitdiff/28048cbaa285b8ac46940e4b39f985d9885fc698 - Provide a planner hook at a suitable place for creating upper-rel Paths. In the initial revision of the upper-planner pathification work, the only available way for an FDW or custom-scan provider to inject Paths representing post-scan-join processing was to insert them during scan-level GetForeignPaths or similar processing. While that's not impossible, it'd require quite a lot of duplicative processing to look forward and see if the extension would be capable of implementing the whole query. To improve matters for custom-scan providers, provide a hook function at the point where the core code is about to start filling in upperrel Paths. At this point Paths are available for the whole scan/join tree, which should reduce the amount of redundant effort considerably. (An alternative design that was suggested was to provide a separate hook for each post-scan-join processing step, but that seems messy and not clearly more useful.) Following our time-honored tradition, there's no documentation for this hook outside the source code. As-is, this hook is only meant for custom scan providers, which we can't assume very much about. A followon patch will implement an FDW callback to let FDWs do the same thing in a somewhat more structured fashion. http://git.postgresql.org/pg/commitdiff/5864d6a4b62ada2ad60a8c456b4ee62972a9c10d - Add a GetForeignUpperPaths callback function for FDWs. This is basically like the just-added create_upper_paths_hook, but control is funneled only to the FDW responsible for all the baserels of the current query; so providing such a callback is much less likely to add useless overhead than using the hook function is. The documentation is a bit sketchy. We'll likely want to improve it, and/or adjust the call conventions, when we get some experience with actually using this callback. Hopefully somebody will find time to experiment with it before 9.6 feature freeze. http://git.postgresql.org/pg/commitdiff/101fd9349eddb7e9ed84a239145d5230a9bc7336 - Cope if platform declares mbstowcs_l(), but not locale_t, in <xlocale.h>. Previously, we included <xlocale.h> only if necessary to get the definition of type locale_t. According to notes in PGAC_TYPE_LOCALE_T, this is important because on some versions of glibc that file supplies an incompatible declaration of locale_t. (This info may be obsolete, because on my RHEL6 box that seems to be the *only* definition of locale_t; but there may still be glibc's in the wild for which it's a live concern.) It turns out though that on FreeBSD and maybe other BSDen, you can get locale_t from stdlib.h or locale.h but mbstowcs_l() and friends only from <xlocale.h>. This was leaving us compiling calls to mbstowcs_l() and friends with no visible prototype, which causes a warning and could possibly cause actual trouble, since it's not declared to return int. Hence, adjust the configure checks so that we'll include <xlocale.h> either if it's necessary to get type locale_t or if it's necessary to get a declaration of mbstowcs_l(). Report and patch by Aleksander Alekseev, somewhat whacked around by me. Back-patch to all supported branches, since we have been using mbstowcs_l() since 9.1. http://git.postgresql.org/pg/commitdiff/0e9b89986b7ced6daffdf14638a25a35c45423ff - Be more careful about out-of-range dates and timestamps. Tighten the semantics of boundary-case timestamptz so that we allow timestamps >= '4714-11-24 00:00+00 BC' and < 'ENDYEAR-01-01 00:00+00 AD' exactly, no more and no less, but it is allowed to enter timestamps within that range using non-GMT timezone offsets (which could make the nominal date 4714-11-23 BC or ENDYEAR-01-01 AD). This eliminates dump/reload failure conditions for timestamps near the endpoints. To do this, separate checking of the inputs for date2j() from the final range check, and allow the Julian date code to handle a range slightly wider than the nominal range of the datatypes. Also add a bunch of checks to detect out-of-range dates and timestamps that formerly could be returned by operations such as date-plus-integer. All C-level functions that return date, timestamp, or timestamptz should now be proof against returning a value that doesn't pass IS_VALID_DATE() or IS_VALID_TIMESTAMP(). Vitaly Burovoy, reviewed by Anastasia Lubennikova, and substantially whacked around by me http://git.postgresql.org/pg/commitdiff/a70e13a39eccf5fc944c66e0029004b6abcb3cae - Fix j2day() to behave sanely for negative Julian dates. Somebody had apparently once figured that casting to unsigned int would produce the right output for negative inputs, but that would only be true if 2^32 were a multiple of 7, which of course it ain't. We need to use a signed division and then correct the sign of the remainder. AFAICT, the only case where this would arise currently is when doing ISO-week calculations for dates in 4714BC, where we'd compute a negative Julian date representing 4714-01-04BC and then do some arithmetic with it. Since we don't even really document support for such dates, this is not of much consequence. But we may as well get it right. Per report from Vitaly Burovoy. http://git.postgresql.org/pg/commitdiff/5db51464311eb7fe4e90030c6a514ff61e9f1c00 - Fix "pg_bench -C -M prepared". This didn't work because when we dropped and re-established a database connection, we did not bother to reset session-specific state such as the statements-are-prepared flags. The st->prepared[] array certainly needs to be flushed, and I cleared a couple of other fields as well that couldn't possibly retain meaningful state for a new connection. In passing, fix some bogus comments and strange field order choices. Per report from Robins Tharakan. http://git.postgresql.org/pg/commitdiff/47211af17a2dbee38b53b2ea6de81499dbb2c7f5 - Fix assorted breakage in to_char()'s OF format option. In HEAD, fix incorrect field width for hours part of OF when tm_gmtoff is negative. This was introduced by commit 2d87eedc1d4468d3 as a result of falsely applying a pattern that's correct when + signs are omitted, which is not the case for OF. In 9.4, fix missing abs() call that allowed a sign to be attached to the minutes part of OF. This was fixed in 9.5 by 9b43d73b3f9bef27, but for inscrutable reasons not back-patched. In all three versions, ensure that the sign of tm_gmtoff is correctly reported even when the GMT offset is less than 1 hour. Add regression tests, which evidently we desperately need here. Thomas Munro and Tom Lane, per report from David Fetter http://git.postgresql.org/pg/commitdiff/55c3a04d60ccea9e999088fb847ceeb9fd4dd927 - Remove useless double calls of make_parsestate(). Aleksander Alekseev http://git.postgresql.org/pg/commitdiff/bd0ab28912d7502b237b8aeb95d052abe4ff6bc6 - Clean up some misplaced #includes. Random .h files have no business including postgres-fe.h (or postgres.h). If that wasn't the first #include done by the calling .c file, it's the .c file that's broken. Noted while prepping Kyotaro Horiguchi's psql lexer refactoring patch. http://git.postgresql.org/pg/commitdiff/3422fecccadb021b7b4cdbc73b2c29f66f031761 - Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me http://git.postgresql.org/pg/commitdiff/4e1d2a170836028370675922ea9a690648d3c18d - Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.) http://git.postgresql.org/pg/commitdiff/27199058d98ef7ff2f468af44654bc35bb70fe4a - Split psql's lexer into two separate .l files for SQL and backslash cases. This gets us to a point where psqlscan.l can be used by other frontend programs for the same purpose psql uses it for, ie to detect when it's collected a complete SQL command from input that is divided across line boundaries. Moreover, other programs can supply their own lexers for backslash commands of their own choosing. A follow-on patch will use this in pgbench. The end result here is roughly the same as in Kyotaro Horiguchi's 0001-Make-SQL-parser-part-of-psqlscan-independent-from-ps.patch, although the details of the method for switching between lexers are quite different. Basically, in this patch we share the entire PsqlScanState, YY_BUFFER_STATE stack, *and* yyscan_t between different lexers. The only thing we need to do to switch to a different lexer is to make sure the start_state is valid for the new lexer. This works because flex doesn't keep any other persistent state that depends on the specific lexing tables generated for a particular .l file. (We are assuming that both lexers are built with the same flex version, or at least versions that are compatible with respect to the contents of yyscan_t; but that doesn't seem likely to be a big problem in practice, considering how slowly flex changes.) Aside from being more efficient than Horiguchi-san's original solution, this avoids possible corner-case changes in semantics: the original code was capable of popping the input buffer stack while still staying in backslash-related parsing states. I'm not sure that that equates to any useful user-visible behaviors, but I'm not sure it doesn't either, so I'm loath to assume that we only need to consider the topmost buffer when parsing a backslash command. I've attempted to update the MSVC build scripts for the added .l file, but will rely on the buildfarm to see if I missed anything. Kyotaro Horiguchi and Tom Lane http://git.postgresql.org/pg/commitdiff/0ea9efbe9ec1bf07cc6ae070bdd54700af08e44d - Suppress FLEX_NO_BACKUP check for psqlscanslash.l. The existing infrastructure for FLEX_NO_BACKUP doesn't work reliably when two lexers are built in parallel in the same directory. We can probably fix that, but as a short-term workaround, just don't make the check for psqlscanslash.l. Per buildfarm. http://git.postgresql.org/pg/commitdiff/a3e39f83632935911bc159154a33e89495f4a676 - Use yylex_init not yylex_init_extra(). Older versions of flex don't have the latter. Per buildfarm. http://git.postgresql.org/pg/commitdiff/ff0a7e6167f475672d82d1cd7cd0d5e735154c4d - Fix missed update in _readForeignScan(). Blatant fail in 0bf3ae88af330496517722e391e7c975e6bad219. Caught by buildfarm member mandrill. http://git.postgresql.org/pg/commitdiff/07aed46a6b3994508e5674301c85ebf5807905ea - With ancient gcc, skip pg_attribute_printf() on function pointer. Buildfarm results show that the ability to attach pg_attribute_printf decoration to a function pointer appeared somewhere between gcc 2.95.3 and gcc 4.0.1. Guess that it was there in 4.0. http://git.postgresql.org/pg/commitdiff/b46d9beb658af7eb4e2a08dfa34206a117c9654f - Build backend/parser/scan.l and interfaces/ecpg/preproc/pgc.l standalone. Now that we know about the %top{} trick, we can revert to building flex lexers as separate .o files. This is worth doing for a couple of reasons besides sheer cleanliness. We can narrow the scope of the -Wno-error flag that's forced on scan.c. Also, since these grammar and lexer files are so large, splitting them into separate build targets should have some advantages in build speed, particularly in parallel or ccache'd builds. We have quite a few other .l files that could be changed likewise, but the above arguments don't apply to them, so the benefit of fixing them seems pretty minimal. Leave the rest for some other day. http://git.postgresql.org/pg/commitdiff/72b1e3a21f0540ffa5c1f8f474b6c52097a368bb - Typo fix. http://git.postgresql.org/pg/commitdiff/78e7c4439917b01afd645a2ec657008ba6c33d37 - Sync backend/parser/scan.l with bin/psql/psqlscan.l. Make some minor formatting adjustments to make it easier to diff these files and see that they indeed implement the same flex rules (at least to the extent that we want them to be the same). (Someday it'd be nice to make ecpg's pgc.l more easily diff'able too, but today is not that day.) Also run relevant parts of these files and psqlscanslash.l through pgindent. No actual behavioral changes here, just obsessive neatnik-ism. http://git.postgresql.org/pg/commitdiff/21c8ee79464a180ab0257abdfceae89274a46632 - Fix phony .PHONY. A couple makefiles had misspelled the magic .PHONY target as PHONY. http://git.postgresql.org/pg/commitdiff/d5351fcb03fc8e20651d5863b88b397a8be68d74 - SQL commands in pgbench scripts are now ended by semicolons, not newlines. To allow multiline SQL commands in scripts, adopt the same rules psql uses to decide what is the end of a SQL command, to wit, an unquoted semicolon not encased in parentheses. Do this by importing the same flex lexer that psql uses, since coping with stuff like dollar-quoted literals is hard to get right without going the full nine yards. This makes use of the infrastructure added in commit 0ea9efbe9ec1bf07 to support independently-written flex lexers scanning the same PsqlScanState input-buffer data structure. Since that infrastructure isn't very friendly to ad-hoc parsing code such as strtok(), improve exprscan.l so that it can parse either whitespace-separated words or expression tokens, on demand, and rewrite pgbench.c's backslash-command parsing code to always use the lexer to fetch tokens. It's still the case that pgbench backslash commands extend to the end of the line, no more and no less. That could be changed in a fairly localized way now, and there was some interest in doing so, but it seems like material for a separate patch. In passing, make some marginal cleanups in syntax error reporting, const-ify a few data structures that could use it, and run some of this code through pgindent. I can't tell whether the MSVC build scripts need to be taught explicitly about the changes here or not, but the buildfarm will soon tell us. Kyotaro Horiguchi and Tom Lane http://git.postgresql.org/pg/commitdiff/68ab8e8ba4a471d91b69f2f89782ba10a0fbef0c - Make pgbench's expression lexer reentrant. This is a necessary preliminary step for making it play with psqlscan.l given the way I set up the lexer input-buffer sharing mechanism in commit 0ea9efbe9ec1bf07. I've not tried to make it *actually* reentrant; there's still some static variables laying about. But flex thinks it's reentrant, and that's what counts. In support of that, fix exprparse.y to pass through the yyscan_t from the caller. Also do some minor code beautification, like not casting away const. http://git.postgresql.org/pg/commitdiff/429ee5a822db0e8faf669d77c810f1eeaaff1ab4 - Best-guess attempt at fixing MSVC build for 68ab8e8ba4a471d9. pgbench now needs to use src/bin/psql/psqlscan.l, but it's not very clear how to fit that into the MSVC build system. If this doesn't work I'm going to need some help from somebody who actually understands those scripts ... http://git.postgresql.org/pg/commitdiff/6f1f34c92b11593ec62ff3e12781eb96dc911821 - Use %option bison-bridge in psql/pgbench lexers. The point of this change is to use %pure-parser in pgbench's exprparse.y. The immediate reason is that it turns out very ancient versions of bison have a bug with the combination of a reentrant lexer and non-reentrant parser. We could consider dropping support for such ancient bisons; but considering that we might well need exprparse.y to be reentrant some day, it seems better to make it so right now than to move the portability goalposts. (AFAICT there's no particular performance consequence to this change, either, so there's no good reason not to do it.) Now, %pure-parser assumes that the called lexer is built with %option bison-bridge. Because we're assuming bitwise compatibility of yyscan_t (yyguts_t) data structures among all the psql/pgbench lexers, that requirement propagates back to psql's lexers as well. But it's just a few lines of change on that side too; and if psqlscan.l is to set the baseline for a possibly-large family of lexers, it should err on the side of including not omitting useful features. http://git.postgresql.org/pg/commitdiff/b6afae71aaf6d2df76d0a0a77c8b630220a01ec1 - lean up some Coverity complaints about commit 0bf3ae88af330496. The two get_tle_by_resno() calls introduced by this commit lacked any check for a NULL return, unlike any other calls of that function anywhere in our tree. Coverity quite properly complained about it. Also fix a misindented line in process_query_params(), which Coverity also complained about on the grounds that the bad indentation suggested possible programmer misinterpretation. http://git.postgresql.org/pg/commitdiff/92b7902deb3155f6975f33e8b6c8be4d9d066172 - Allow the delay in psql's \watch command to be a fractional second. Instead of just "2" seconds, allow eg. "2.5" seconds. Per request from Alvaro Herrera. No docs change since the docs didn't say you couldn't do this already. http://git.postgresql.org/pg/commitdiff/b283096534b9c514a92a70c98c033015b6792ba7 - Improve header output from psql's \watch command. Include the \pset title string if there is one, and shorten the prefab part of the header to be "timestamp (every Ns)". Per suggestion by David Johnston. Michael Paquier and Tom Lane http://git.postgresql.org/pg/commitdiff/dea2b5960a9460c02896ed361d35e92bce02801a - Fix EvalPlanQual bug when query contains both locked and not-locked rels. In commit afb9249d06f47d7a, we (probably I) made ExecLockRows assign null test tuples to all relations of the query while setting up to do an EvalPlanQual recheck for a newly-updated locked row. This was sheerest brain fade: we should only set test tuples for relations that are lockable by the LockRows node, and in particular empty test tuples are only sensible for inheritance child relations that weren't the source of the current tuple from their inheritance tree. Setting a null test tuple for an unrelated table causes it to return NULLs when it should not, as exhibited in bug #14034 from Bronislav Houdek. To add insult to injury, doing it the wrong way required two loops where one would suffice; so the corrected code is even a bit shorter and faster. Add a regression test case based on his example, and back-patch to 9.5 where the bug was introduced. http://git.postgresql.org/pg/commitdiff/71404af2a29ce4a3a5907cdc8b893ec2bc0285b4 - Fix unsafe use of strtol() on a non-null-terminated Text datum. jsonb_set() could produce wrong answers or incorrect error reports, or in the worst case even crash, when trying to convert a path-array element into an integer for use as an array subscript. Per report from Vitaly Burovoy. Back-patch to 9.5 where the faulty code was introduced (in commit c6947010ceb42143). Michael Paquier http://git.postgresql.org/pg/commitdiff/384dfbde19330541f7fb487f9352949aa06c812e - Code review for error reports in jsonb_set(). User-facing (even tested by regression tests) error conditions were thrown with elog(), hence had wrong SQLSTATE and were untranslatable. And the error message texts weren't up to project style, either. http://git.postgresql.org/pg/commitdiff/ea4b8bd6188ecb17ba37d93f57b8b020a964e66c - Move keywords.c/kwlookup.c into src/common/. Now that we have src/common/ for code shared between frontend and backend, we can get rid of (most of) the klugy ways that the keyword table and keyword lookup code were formerly shared between different uses. This is a first step towards a more general plan of getting rid of special-purpose kluges for sharing code in src/bin/. I chose to merge kwlookup.c back into keywords.c, as it once was, and always has been so far as keywords.h is concerned. We could have kept them separate, but there is noplace that uses ScanKeywordLookup without also wanting access to the backend's keyword list, so there seems little point. ecpg is still a bit weird, but at least now the trickiness is documented. I think that the MSVC build script should require no adjustments beyond what's done here ... but we'll soon find out. http://git.postgresql.org/pg/commitdiff/2c6af4f44228d76d3351fe26f68b00b55cdd239a - Avoid PGDLLIMPORT for simple local references in frontend programs. I was wondering if this would be an issue, and buildfarm member frogmouth says it is. http://git.postgresql.org/pg/commitdiff/c2d1eea9e750edb267e3f071a129e03d79ad198b - Create src/fe_utils/, and move stuff into there from pg_dump's dumputils. Per discussion, we want to create a static library and put the stuff into it that until now has been shared across src/bin/ directories by ad-hoc methods like symlinking a source file. This commit creates the library and populates it with a couple of files that contain the widely-useful portions of pg_dump's dumputils.c file. dumputils.c survives, because it has some stuff that didn't seem appropriate for fe_utils, but it's significantly smaller and is no longer referenced from any other directory. Follow-on patches will move more stuff into fe_utils. The Mkvcbuild.pm hacking here is just a best guess; we'll see how the buildfarm likes it. http://git.postgresql.org/pg/commitdiff/588d963b00e5e4385b6425418e3faa726f63f72e - Add missed inclusion requirement in Mkvcbuild.pm. Per buildfarm. http://git.postgresql.org/pg/commitdiff/0ecd3fedfcf3427ebeb73cc61b2fcf6ed67c43a2 - Suppress compiler warning for get_am_type_string(). Compilers that don't know that elog(ERROR) doesn't return complained that this function might fail to return a value. Per buildfarm. While at it, const-ify the function's declaration, since the intent is evidently to always return a constant string. http://git.postgresql.org/pg/commitdiff/a376960c8f8ec08783e1c529f36fbeb60236b378 - Move psql's print.c and mbprint.c into src/fe_utils. Just turning the crank ... http://git.postgresql.org/pg/commitdiff/d65bea26a867e3bbd053bf87b985b0e113256414 - Move psql's psqlscan.l into src/fe_utils. This completes (at least for now) the project of getting rid of ad-hoc linkages among the src/bin/ subdirectories. Everything they share is now in src/fe_utils/ and is included from a static library at link time. A side benefit is that we can restore the FLEX_NO_BACKUP check for psqlscanslash.l. We might need to think of another way to do that check if we ever need to build two lexers with that property in the same source directory, but there's no foreseeable reason to need that. http://git.postgresql.org/pg/commitdiff/c1156411ad0879a71956b64aa487babe7572685b - Link libpq after libpgfeutils to satisfy Windows linker. Some of the non-MSVC Windows buildfarm members seem to need this to avoid getting "undefined symbol" errors on libpgfeutils' references to libpq. I could understand that if libpq were a static library, but surely it is not? Oh well, at least the extra reference is no more harmful than it is for libpgcommon or libpgport. http://git.postgresql.org/pg/commitdiff/7caaeaf3607fae91318f24debce3dc017ca299a3 - Don't split up SRFs when choosing to postpone SELECT output expressions. In commit 9118d03a8cca3d97 we taught the planner to postpone evaluation of set-returning functions in a SELECT's targetlist until after any sort done to satisfy ORDER BY. However, if we postpone some SRFs this way while others do not get postponed (because they're sort or group key columns) we will break the traditional behavior by which all SRFs in the tlist run in-step during ExecTargetList(), so that you get the least common multiple of their periods not the product. Fix make_sort_input_target() so it will not split up SRF evaluation in such cases. There is still a hazard of similar odd behavior if there's a SRF in a grouping column and another one that isn't, but that was true before and we're just trying to preserve bug-compatibility with the traditional behavior. This whole area is overdue to be rethought and reimplemented, but we'll try to avoid changing behavior until then. Per report from Regina Obe. http://git.postgresql.org/pg/commitdiff/d543170f2fdd6d9845aaf91dc0f6be7a2bf0d9e7 - Fix DROP OPERATOR to reset oprcom/oprnegate links to the dropped operator. This avoids leaving dangling links in pg_operator; which while fairly harmless are also unsightly. While we're at it, simplify OperatorUpd, which went through heap_modify_tuple for no very good reason considering it had already made a tuple copy it could just scribble on. Roma Sokolov, reviewed by Tomas Vondra, additional hacking by Robert Haas and myself. http://git.postgresql.org/pg/commitdiff/c94959d4110a1965472956cfd631082a96f64a84 - In PL/Tcl, make database errors return additional info in the errorCode. Tcl has a convention for returning additional info about an error in a global variable named errorCode. Up to now PL/Tcl has ignored that, but this patch causes database errors caught by PL/Tcl to fill in errorCode with useful information from the ErrorData struct. Jim Nasby, reviewed by Pavel Stehule and myself http://git.postgresql.org/pg/commitdiff/fb8d2a7f57d87102f0a95025fbf1cad9c341739b - Improve PL/Tcl errorCode facility by providing decoded name for SQLSTATE. We don't really want to encourage people to write numeric SQLSTATEs in programs; that's unreadable and error-prone. Copy plpgsql's infrastructure for converting between SQLSTATEs and exception names shown in Appendix A, and modify examples in tests and documentation to do it that way. http://git.postgresql.org/pg/commitdiff/cd37bb78599dcf24cd22a124ce9174b5e2a76880 - Fix PL/Tcl for vpath builds. Commit cd37bb78599dcf24 works for in-tree builds, but not so much for VPATH. Per buildfarm. http://git.postgresql.org/pg/commitdiff/9f73a2f6d1c1305cf0dc749dbf631cffe26beda0 - Update time zone data files to tzdata release 2016c. DST law changes in Azerbaijan, Chile, Haiti, Palestine, and Russia (Altai, Astrakhan, Kirov, Sakhalin, Ulyanovsk regions). Historical corrections for Lithuania, Moldova, Russia (Kaliningrad, Samara, Volgograd). As of 2015b, the keepers of the IANA timezone database started to use numeric time zone abbreviations (e.g., "+04") instead of inventing abbreviations not found in the wild like "ASTT". This causes our rather old copy of zic to whine "warning: time zone abbreviation differs from POSIX standard" several times during "make install". This warning is harmless according to the IANA folk, and I don't see any problems with these abbreviations in some simple tests; but it seems like now would be a good time to update our copy of the tzcode stuff. I'll look into that soon. http://git.postgresql.org/pg/commitdiff/676265eb7b57ba5bfae859630b909e6045893b68 - Avoid a couple of zero-divide scenarios in the planner. cost_subplan() supposed that the given subplan must have plan_rows > 0, which as far as I can tell was true until recent refactoring of the code in createplan.c; but now that code allows the Result for a provably empty subquery to have plan_rows = 0. Rather than undo that change, put in a clamp to prevent zero divide. get_cheapest_fractional_path() likewise supposed that best_path->rows > 0. This assumption has been wrong for longer. It's actually harmless given IEEE float math, because a positive value divided by zero gives +Infinity and compare_fractional_path_costs() will do the right thing with that. Still, best not to assume that. final_cost_nestloop() also seems to have some risks in this area, so borrow the clamping logic already present in the mergejoin cost functions. Lastly, remove unnecessary clamp_row_est() in planner.c's calls to get_number_of_groups(). The only thing that function does with path_rows is pass it to estimate_num_groups() which already has an internal clamp, so we don't need the extra call; and if we did, the callers are arguably the wrong place for it anyway. First two items reported by Piotr Stefaniak, the others are products of my nosing around for similar problems. No back-patch since there's no evidence that problems arise in the back branches. http://git.postgresql.org/pg/commitdiff/76281aa9647e6a5dfc646514554d0f519e3b8a58 - Modernize zic's test for valid timezone abbreviations. We really need to sync all of our IANA-derived timezone code with upstream, but that's going to be a large patch and I certainly don't care to shove such a thing into stable branches immediately before a release. As a stopgap, copy just the tzcode2016c logic that checks validity of timezone abbreviations. This prevents getting multiple "time zone abbreviation differs from POSIX standard" bleats with tzdata 2014b and later. http://git.postgresql.org/pg/commitdiff/221619ad69b7e060041796a1974fbb0eeb9542d7 - First-draft release notes for 9.5.2. As usual, the release notes for other branches will be made by cutting these down, but put them up for community review first. http://git.postgresql.org/pg/commitdiff/29b6123ecb4113e366325245cec5a5c221dae691 Robert Haas pushed: - Improve check for overly-long extensible node name. The old code is bad for two reasons. First, it has an off-by-one error. Second, it won't help if you aren't running with assertions enabled. Per discussion, we want a check here in that case too. Author: KaiGai Kohei, adjusted by me. Reviewed-by: Petr Jelinek Discussion: 56E0D547.1030101@2ndquadrant.com http://git.postgresql.org/pg/commitdiff/3adf9ced17dfa84faa209127b4499e5a5b995806 - Update more comments for 96198d94cb7adc664bda341842dc8db671d8be72. Etsuro Fujita, reviewed (though not completely endorsed) by Ashutosh Bapat, and slightly expanded by me. http://git.postgresql.org/pg/commitdiff/6be84eeb8d5a72014b1b6e44d0646f69531a1ccf - Fix EXPLAIN ANALYZE SELECT INTO not to choose a parallel plan. We don't support any parallel write operations at present, so choosing a parallel plan causes us to error out. Also, add a new regression test that uses EXPLAIN ANALYZE SELECT INTO; if we'd had this previously, force_parallel_mode testing would have caught this issue. Mithun Cy and Robert Haas http://git.postgresql.org/pg/commitdiff/270b7daf5cb1e955f8771ec819eb810057d1779b - Add simple VACUUM progress reporting. There's a lot more that could be done here yet - in particular, this reports only very coarse-grained information about the index vacuuming phase - but even as it stands, the new pg_stat_progress_vacuum can tell you quite a bit about what a long-running vacuum is actually doing. Amit Langote and Robert Haas, based on earlier work by Vinayak Pokale and Rahila Syed. http://git.postgresql.org/pg/commitdiff/c16dc1aca5e01e6acaadfcf38f5fc964a381dc62 - Fix typos. Thomas Reiss http://git.postgresql.org/pg/commitdiff/2a90cb69e18e0c0637f3f93702e7b2f7808ff1b2 - postgres_fdw: make_tuple_from_result_row should set cur_attno for ctid. There's no reason for this function to do this for every other attribute number and omit it for CTID, especially since conversion_error_callback has code to handle that case. This seems to be an oversight in commit e690b9515072fd7767fdeca5c54166f6a77733bc. Etsuro Fujita http://git.postgresql.org/pg/commitdiff/4a46a99d8936328ab00e54bf8a2900832c4687e7 - Fix typos. Oskari Saarenmaa http://git.postgresql.org/pg/commitdiff/3aff33aa687e47d52f453892498b30ac98a296af - Add idle_in_transaction_session_timeout. Vik Fearing, reviewed by Stéphane Schildknecht and me, and revised slightly by me. http://git.postgresql.org/pg/commitdiff/c6dda1f48e573cc60fe25d8d470a05bfabed0252 - Fix typo. Amit Langote http://git.postgresql.org/pg/commitdiff/1c4f001b79878deb9475e5b32ff4ef87773c9f23 - Fix problems in commit c16dc1aca5e01e6acaadfcf38f5fc964a381dc62. Vinayak Pokale provided a patch for a copy-and-paste error in a comment. I noticed that I'd use the word "automatically" nearby where I meant to talk about things being "atomic". Rahila Syed spotted a misplaced counter update. Fix all that stuff. http://git.postgresql.org/pg/commitdiff/bc55cc0b6a4af048f46c7e46c9e9c1f24b7ee9b8 - Another comment update. I thought this was in my last commit, but I goofed. http://git.postgresql.org/pg/commitdiff/f2b74b01d4a18241bd7560e74d527f3ba42d4738 - Fix typos. Jim Nasby http://git.postgresql.org/pg/commitdiff/0218e8b3fa7ec72af441ac6f80927bff0d497334 - Update tuplesort.c comments for memory mangement improvements. I'm committing these changes separately so that it's clear what is Peter's original work versus what I changed. This is a followup to commit 0011c0091e886b874e485a46ff2c94222ffbf550, and these changes are all by me. http://git.postgresql.org/pg/commitdiff/c27033ff7c17b5100d02c454a0eebb95ec7b91cc - Various minor corrections of and improvements to comments. Aleksander Alekseev http://git.postgresql.org/pg/commitdiff/2d8a1e22b109680204cb015a30e5a733a233ed64 - Push scan/join target list beneath Gather when possible. This means that, for example, "SELECT expensive_func(a) FROM bigtab WHERE something" can compute expensive_func(a) in the workers rather than the leader if it happens to be parallel-safe, which figures to be a big win in some practical cases. Currently, we can only do this if the entire target list is parallel-safe. If we worked harder, we might be able to evaluate parallel-safe targets in the worker and any parallel-restricted targets in the leader, but that would be more complicated, and there aren't that many parallel-restricted functions that people are likely to use in queries anyway. I think. So just do the simple thing for the moment. Robert Haas, Amit Kapila, and Tom Lane http://git.postgresql.org/pg/commitdiff/992b5ba30dcafdc222341505b072a6b009b248a7 - Improve memory management for external sorts. Introduce a new memory context which stores tuple data, and reset it at the end of each merge pass; this helps avoid memory fragmentation and, consequently, overallocation. Also, for the final merge patch, eliminate memory context chunk header overhead entirely by allocating all of the memory used for buffering tuples during the merge in a single chunk. Since this modestly increases the number of tuples we can store, grow the memtuples array a bit so that we're less likely to run short of slots there. Peter Geoghegan. Review and testing of patches in this series by Jeff Janes, Greg Stark, Mithun Cy, and me. http://git.postgresql.org/pg/commitdiff/0011c0091e886b874e485a46ff2c94222ffbf550 - Directly modify foreign tables. postgres_fdw can now sent an UPDATE or DELETE statement directly to the foreign server in simple cases, rather than sending a SELECT FOR UPDATE statement and then updating or deleting rows one-by-one. Etsuro Fujita, reviewed by Rushabh Lathia, Shigeru Hanada, Kyotaro Horiguchi, Albe Laurenz, Thom Brown, and me. http://git.postgresql.org/pg/commitdiff/0bf3ae88af330496517722e391e7c975e6bad219 - Use INT64_FORMAT instead of %ld for int64. Commit 0011c0091e886b874e485a46ff2c94222ffbf550 introduced this mistake. Patch by me. Reported by Andres Freund, who also reviewed the patch. http://git.postgresql.org/pg/commitdiff/08a6d36dcbe4ebdcdf9d554e5b571748fe23001b - Support parallel aggregation. Parallel workers can now partially aggregate the data and pass the transition values back to the leader, which can combine the partial results to produce the final answer. David Rowley, based on earlier work by Haribabu Kommi. Reviewed by Álvaro Herrera, Tomas Vondra, Amit Kapila, James Sewell, and me. http://git.postgresql.org/pg/commitdiff/e06a38965b3bcdaa881e7e06892d4d8ab6c2c980 - Make max_parallel_degree PGC_USERSET. It was intended to be this way all along, just like other planner GUCs such as work_mem. But I goofed. http://git.postgresql.org/pg/commitdiff/ae507d9222989f2fd8455cf32d4dcb337ccf32bb - Partition the freelist for shared dynahash tables. Without this, contention on the freelist can become a pretty serious problem on large servers. Aleksander Alekseev, reviewed by Anastasia Lubennikova, Dilip Kumar, and me. http://git.postgresql.org/pg/commitdiff/44ca4022f3f9297bab5cbffdd97973dbba1879ed - postgres_fdw: Fix crash when pushing down multiple joins. A join clause might mention multiple relations on either side, so it need not be the case that a given joinrel's constituent relations are all on one side of the join clause or all on the other. Report by Rajkumar Raghuwanshi. Analysis and fix by Michael Paquier and Ashutosh Bapat. http://git.postgresql.org/pg/commitdiff/3151f16e1874db82ed85a005dac15368903ca9fb - Disable abbreviated keys for string-sorting in non-C locales. Unfortunately, every version of glibc thus far tested has bugs whereby strcoll() ordering does not match strxfrm() ordering as required by the standard. This can result in, for example, corrupted indexes. Disabling abbreviated keys in these cases slows down non-C-collation string sorting considerably, but there seems to be no practical alternative. Users who are confident that their libc implementations are solid in this regard can re-enable the optimization by compiling with TRUST_STRXFRM. Users who have built indexes using PostgreSQL 9.5 or PostgreSQL 9.5.1 should REINDEX if there is a possibility that they may have been affected by this problem. Report by Marc-Olaf Jaschke. Investigation mostly by Tom Lane, with help from Peter Geoghegan, Noah Misch, Stephen Frost, and me. Patch by me, reviewed by Peter Geoghegan and Tom Lane. http://git.postgresql.org/pg/commitdiff/3df9c374e279db37b00cd9c86219471d0cdaa97c - Use correct GetDatum function. Oops. http://git.postgresql.org/pg/commitdiff/59a02815e2d11e5a7b4848897c763175b6b42f8a - Improve documentation for combine functions. David Rowley http://git.postgresql.org/pg/commitdiff/a596db332b8c7f593a82af86f69353ba08f6214c Peter Eisentraut pushed: - Add missing include for self-containment http://git.postgresql.org/pg/commitdiff/be6de4c1215a8ad5607b1fcc7e9e6da1de780877 - UCS_to_EUC_JIS_2004.pl: Turn off "test" mode by default. It produces debugging output files that are of no further use, so we don't need that by default. http://git.postgresql.org/pg/commitdiff/f9e5ed61ed1e27390051e8eb10bd9ec4cb40d7b2 - Make spacing and punctuation consistent. http://git.postgresql.org/pg/commitdiff/9dbcb500cad06f2dd1c699e8dbcfbe23145e61fa - Add syslog_sequence_numbers parameter. Reviewed-by: Andreas Karlsson <andreas@proxel.se> http://git.postgresql.org/pg/commitdiff/f4c454e9ba5262c5614c7637983c83b8c327e231 - Add syslog_split_messages parameter. Reviewed-by: Andreas Karlsson <andreas@proxel.se> http://git.postgresql.org/pg/commitdiff/fc201dfd95059fd2fef9862a2fd09cfab42c9bf7 - Merge wal_level "archive" and "hot_standby" into new name "replica". The distinction between "archive" and "hot_standby" existed only because at the time "hot_standby" was added, there was some uncertainty about stability. This is now a long time ago. We would like to move forward with simplifying the replication configuration, but this distinction is in the way, because a primary server cannot tell (without asking a standby or predicting the future) which one of these would be the appropriate level. Pick a new name for the combined setting to make it clearer that it covers all (non-logical) backup and replication uses. The old values are still accepted but are converted internally. Reviewed-by: Michael Paquier <michael.paquier@gmail.com> Reviewed-by: David Steele <david@pgmasters.net> http://git.postgresql.org/pg/commitdiff/b555ed8102672cfedb06559952b8341756386d69 - Allow SSL server key file to have group read access if owned by root. We used to require the server key file to have permissions 0600 or less for best security. But some systems (such as Debian) have certificate and key files managed by the operating system that can be shared with other services. In those cases, the "postgres" user is made a member of a special group that has access to those files, and the server key file has permissions 0640. To accommodate that kind of setup, also allow the key file to have permissions 0640 but only if owned by root. From: Christoph Berg <myon@debian.org> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> http://git.postgresql.org/pg/commitdiff/9a83564c58b7f6363141a8f1d0c87c89a5ebab5d Álvaro Herrera pushed: - Fix typos in comments. http://git.postgresql.org/pg/commitdiff/5bcc413f80de4e0bc8f0df15d7d0ac2e8272dec5 - docs: Fix typo'd brin_summarize_new_values. I wrote "brin_summarize_new_pages" instead, in docs as well as in the commit message of commit ac443d1034d9. Bug: #14030 Reported-By: Chris Pacejo http://git.postgresql.org/pg/commitdiff/696684d878747a1a5cfae01887404629c40c09dd - pgbench: Allow changing weights for scripts. Previously, all scripts had the same probability of being chosen when multiple of them were specified via -b, -f, -N, -S. With this commit, -b and -f now search for an "@" in the script name and use the integer found after it as the drawing probability for that script. (One disadvantage is that if you have script whose names contain @, you are now forced to specify "@1" at the end; otherwise the name's @ is confused with a weight separator. We don't expect many pgbench script with @ in their names in the wild, so this shouldn't be too serious a problem.) While at it, rework the interface between addScript, process_file, process_builtin, and findBuiltin. It had gotten a bit out of hand with recent commits. Author: Fabien Coelho Reviewed-By: Andres Freund, Robert Haas, Álvaro Herrera, Michaël Paquier Discussion: http://www.postgresql.org/message-id/alpine.DEB.2.10.1603160721240.1666@sto http://git.postgresql.org/pg/commitdiff/7bafffea647e21864cb857ab5b6fe266f2d0976a - pgbench: Silence new compiler warnings. The original coding in 7bafffea647 and previous wasn't all that great anyway. Reported by Jeff Janes and Tom Lane http://git.postgresql.org/pg/commitdiff/1038bc91ca98865bd60bf63db46fc331f3099998 Stephen Frost pushed: - Avoid incorrectly indicating exclusion constraint wait. INSERT ... ON CONFLICT's precheck may have to wait on the outcome of another insertion, which may or may not itself be a speculative insertion. This wait is not necessarily associated with an exclusion constraint, but was always reported that way in log messages if the wait happened to involve a tuple that had no speculative token. Initially discovered through use of ON CONFLICT DO NOTHING, where spurious references to exclusion constraints in log messages were more likely. Patch by Peter Geoghegan. Reviewed by Julien Rouhaud. Back-patch to 9.5 where INSERT ... ON CONFLICT was added. http://git.postgresql.org/pg/commitdiff/fd658dbb300456b393536802d1145a9cea7b25d6 Teodor Sigaev pushed: - Improve script generating unaccent rules. Script now use the standard Unicode transliterator Latin-ASCII. Author: Leonard Benedetti http://git.postgresql.org/pg/commitdiff/9a206d063c410df7cd5da01b169b23bff413fef5 - fix typo in comment http://git.postgresql.org/pg/commitdiff/ce91b9209f221f9555d340b40e02aaf8cdc777e4 - Add files forgotten in f576b17cd6ba653bdace1f0da9a3b57f4984e460 http://git.postgresql.org/pg/commitdiff/e4b523e5b52a84010a34fc0796904ce39116e56c - Add word_similarity to pg_trgm contrib module. Patch introduces a concept of similarity over string and just a word from another string. Version of extension is not changed because 1.2 was already introduced in 9.6 release cycle, so, there wasn't a public version. Author: Alexander Korotkov, Artur Zakirov http://git.postgresql.org/pg/commitdiff/f576b17cd6ba653bdace1f0da9a3b57f4984e460 - Improve support of Hunspell: Allow to use non-ascii characters as affix flag. Non-numeric affix flags now are stored as string instead of numeric value of character. Allow to use 0 as affix flag in numeric encoded affixes That adds support for arabian, hungarian, turkish and brazilian portuguese languages. Author: Artur Zakirov with heavy editorization by me http://git.postgresql.org/pg/commitdiff/f4ceed6ceba31a72ed7a726fef05d211641f283c - GUC variable pg_trgm.similarity_threshold insead of set_limit(). Use GUC variable pg_trgm.similarity_threshold insead of set_limit()/show_limit() which was introduced when defining GUC varuables by modules was absent. Author: Artur Zakirov http://git.postgresql.org/pg/commitdiff/5871b88487cfd07966e2ce08609a4d6d5ee9718e - pg_trgm's set_limit() now uses SetConfigOption(). Deprecated set_limit() is modified to use SetConfigOption() to set similarity_threshold which is actually an instance of pg_trgm.similarity_threshold GUC variable. Previous coding directly sets similarity_threshold what could cause an inconsistency between states of actual variable and GUC representation. Per gripe from Tom Lane http://git.postgresql.org/pg/commitdiff/aa698d753566f68bdd54881d30b1a515b0327b0e - Fix a typo. Erik Rijkers http://git.postgresql.org/pg/commitdiff/61d2ebdbf9155d80f83281d8c5729c8f8f0949bc - Introduce parse_ident(). SQL-layer function to split qualified identifier into array parts. Author: Pavel Stehule with minor editorization by me and Jim Nasby http://git.postgresql.org/pg/commitdiff/3187d6de0e5a9e805b27c48437897e8c39071d45 - Improve docs of pg_trgm changes Artur Zakirov, per gripe from Jeff Janes http://git.postgresql.org/pg/commitdiff/f6bd0da63bf40bc324eec6fd47f3d3d301f44dab Andres Freund pushed: - Error out if waiting on socket readiness without a specified socket. Previously we just ignored such an attempt, but that seems to serve no purpose but making things harder to debug. Discussion: 20160114143931.GG10941@awork2.anarazel.de 20151230173734.hx7jj2fnwyljfqek@alap3.anarazel.de Reviewed-By: Robert Haas http://git.postgresql.org/pg/commitdiff/6bc4d95fcc2a432fc202cba03d5393be096f0422 - Only clear latch self-pipe/event if there is a pending notification. This avoids a good number of, individually quite fast, system calls in scenarios with many quick queries. Besides the aesthetic benefit of seing fewer superflous system calls with strace, it also improves performance by ~2% measured by pgbench -M prepared -c 96 -j 8 -S (scale 100). Without having benchmarked it, this patch also adjust the windows code, as that makes it easier to unify the unix/windows codepaths in a later patch. There's little reason to diverge in behaviour between the platforms. Discussion: CA+TgmoYc1Zm+Szoc_Qbzi92z2c1vRHZmjhfPn5uC=w8bXv6Avg@mail.gmail.com Reviewed-By: Robert Haas http://git.postgresql.org/pg/commitdiff/c4901a1e03a7730e4471fd1143f1caf79695493d - Remove unused, and dangerous, TestLatch() macro. The macro has not seen any in-tree use since latches had been introduced in 2746e5f, in 2010. http://git.postgresql.org/pg/commitdiff/fad0f9d8c9f6a8e99156b8f01cba54be39f31761 - Make it easier to choose the used waiting primitive in unix_latch.c. This allows for easier testing of the different primitives; in preparation for adding a new primitive. Discussion: 20160114143931.GG10941@awork2.anarazel.de Reviewed-By: Robert Haas http://git.postgresql.org/pg/commitdiff/c17966201c7de2a4c437bed6d83c6a7f2e7108f4 - Fix stupid omission in c4901a1e. Reported-By: Jeff Janes Discussion: CAMkU=1zGxREwoyaCrp_CHadEB+dPgpVyKBysCJ+6xP9gCOvAuw@mail.gmail.com http://git.postgresql.org/pg/commitdiff/6eb2be15b5d24b98d334a9dd637f0edb37e2eb7e - Second attempt at fixing MSVC build for 68ab8e8ba4a471d9. After the previous fix in 6f1f34c9 msvc ended up looking for psqlscan.c in the wrong directory. David's fix just forces the path to be adjusted. That's not a particularly pretty fix, but it hopefully will make the buildfarm green again. Author: David Rowley Discussion: CAKJS1f_9CCi_t+LEgV5GWoCj3wjavcMoDc5qfcf_A0UwpQoPoA@mail.gmail.com http://git.postgresql.org/pg/commitdiff/326d73c86fda407a810675c3b5a48e0a0cc992f5 - Introduce WaitEventSet API. Commit ac1d794 ("Make idle backends exit if the postmaster dies.") introduced a regression on, at least, large linux systems. Constantly adding the same postmaster_alive_fds to the OSs internal datastructures for implementing poll/select can cause significant contention; leading to a performance regression of nearly 3x in one example. This can be avoided by using e.g. linux' epoll, which avoids having to add/remove file descriptors to the wait datastructures at a high rate. Unfortunately the current latch interface makes it hard to allocate any persistent per-backend resources. Replace, with a backward compatibility layer, WaitLatchOrSocket with a new WaitEventSet API. Users can allocate such a Set across multiple calls, and add more than one file-descriptor to wait on. The latter has been added because there's upcoming postgres features where that will be helpful. In addition to the previously existing poll(2), select(2), WaitForMultipleObjects() implementations also provide an epoll_wait(2) based implementation to address the aforementioned performance problem. Epoll is only available on linux, but that is the most likely OS for machines large enough (four sockets) to reproduce the problem. To actually address the aforementioned regression, create and use a long-lived WaitEventSet for FE/BE communication. There are additional places that would benefit from a long-lived set, but that's a task for another day. Thanks to Amit Kapila, who helped make the windows code I blindly wrote actually work. Reported-By: Dmitry Vasilyev Discussion: CAB-SwXZh44_2ybvS5Z67p_CDz=XFn4hNAD=CnMEF+QqkXwFrGg@mail.gmail.com 20160114143931.GG10941@awork2.anarazel.de http://git.postgresql.org/pg/commitdiff/98a64d0bd713cb89e61bef6432befc4b7b5da59e - Combine win32 and unix latch implementations. Previously latches for windows and unix had been implemented in different files. A later patch introduce an expanded wait infrastructure, keeping the implementation separate would introduce too much duplication. This basically just moves the functions, without too much change. The reason to keep this separate is that it allows blame to continue working a little less badly; and to make review a tiny bit easier. Discussion: 20160114143931.GG10941@awork2.anarazel.de http://git.postgresql.org/pg/commitdiff/72e2d21c1249b674496f97cd6009c0bda62f6b4d - Properly declare FeBeWaitSet. Surprising that this worked on a number of systems. Reported by buildfarm member longfin. http://git.postgresql.org/pg/commitdiff/7fa0064092e135415a558dc3c4d7393d14ab6d8e - Change various Gin*Is* macros to return 0/1. Returning the direct result of bit arithmetic, in a macro intended to be used in a boolean manner, can be problematic if the return value is stored in a variable of type 'bool'. If bool is implemented using C99's _Bool, that can lead to comparison failures if the variable is then compared again with the expression (see ginStepRight() for an example that fails), as _Bool forces the result to be 0/1. That happens in some configurations of newer MSVC compilers. It's also problematic when storing the result of such an expression in a narrower type. Several gin macros have been declared in that style since gin's initial commit in 8a3631f8d86. There's a lot more macros like this, but this is the only one causing regression test failures; and I don't want to commit and backpatch a larger patch with lots of conflicts just before the next set of minor releases. Discussion: 20150811154237.GD17575@awork2.anarazel.de Backpatch: All supported branches http://git.postgresql.org/pg/commitdiff/af4472bcb88ab36b9abbe7fd5858e570a65a2d1a - Don't use !! but != 0/NULL to force boolean evaluation. I introduced several uses of !! to force bit arithmetic to be boolean, but per discussion the project prefers != 0/NULL. Discussion: CA+TgmoZP5KakLGP6B4vUjgMBUW0woq_dJYi0paOz-My0Hwt_vQ@mail.gmail.com http://git.postgresql.org/pg/commitdiff/1a7a43672bf2939dda93a27d498349e7a4aa3c14 Andrew Dunstan pushed: - Remove dependency on psed for MSVC builds. Modern Perl has removed psed from its core distribution, so it might not be readily available on some build platforms. We therefore replace its use with a Perl script generated by s2p, which is equivalent to the sed script. The latter is retained for non-MSVC builds to avoid creating a new hard dependency on Perl for non-Windows tarball builds. Backpatch to all live branches. Michael Paquier and me. http://git.postgresql.org/pg/commitdiff/5d0320105699c253fe19b8b42ae1bffb67785b02 Fujii Masao pushed: - Fix typo in docs. Jeff Janes http://git.postgresql.org/pg/commitdiff/112a2d0615bab27630829e486c8b0cd2fdd6980b Simon Riggs pushed: - Change comment to describe correct lock level used http://git.postgresql.org/pg/commitdiff/8320c625d983f4cf4c8f453165502d7cf3aa7cf3 Álvaro Herrera pushed: - Support CREATE ACCESS METHOD. This enables external code to create access methods. This is useful so that extensions can add their own access methods which can be formally tracked for dependencies, so that DROP operates correctly. Also, having explicit support makes pg_dump work correctly. Currently only index AMs are supported, but we expect different types to be added in the future. Authors: Alexander Korotkov, Petr Jelínek. Reviewed-By: Teodor Sigaev, Petr Jelínek, Jim Nasby. Commitfest-URL: https://commitfest.postgresql.org/9/353/ Discussion: https://www.postgresql.org/message-id/CAPpHfdsXwZmojm6Dx+TJnpYk27kT4o7Ri6X_4OSWcByu1Rm+VA@mail.gmail.com http://git.postgresql.org/pg/commitdiff/473b93287040b20017cc25a157cffdc5b978c254 == Rejected Patches (for now) == No one was disappointed this week :-) == Pending Patches == Etsuro Fujita sent in a patch to expose umid in foreign tables. Konstantin Knizhnik sent in another revision of a patch to extend ALTER INDEX to allow changing the predicate of a partial index. Stephen Frost sent in two more revisions of a patch to add default roles. Tomas Vondra sent in another revision of a patch to use foreign keys to improve join estimates. Peter Geoghegan sent in a patch to distrust external OpenSSL clients and clear err queue. Amit Kapila sent in another revision of a patch to prevent prepared statements from inappriately enabling parallelism. Álvaro Herrera sent in another revision of a patch to allow logical slots to follow timeline switches. Amit Kapila sent in another revision of a patch to group the status update for transactions. Kaigai Kouhei sent in another revision of a patch to rework custom scan serialization. Anastasia Lubennikova sent in another revision of a patch to implement covering unique indexes. Michaël Paquier sent in two more revisions of a patch to implement SCRAM authentication with infrastructure for making auth more pluggable. Corey Huinker sent in another revision of a patch to implement \gexec in psql. Robbie Harwood sent in two more revisions of a patch to add GSSAPI encryption support. David Rowley and Julien Rouhaud traded patches to enable setting parallel degree. Alexander Shulgin sent in another revision of a patch to add \errverbose psql command and support in libpq. Michaël Paquier sent in another revision of a patch to fix hot standby checkpoints. Ashutosh Bapat sent in two revisions of a patch to require user mappings for foreign joins if the tables involved would have required same. Oskari Saarenmaa sent in a patch to display backends for dropped roles in pg_stat_activity. Craig Ringer sent in two more revisions of a patch to implement failover slots. Michaël Paquier sent some follow-on patches to the ones now committed that make sure filesystem moves are successful before moving on. Constantin S. Pan sent in another revision of a patch to speed up GIN build with parallel workers. Thomas Reiss and Thom Brown traded patches to replace pg_stat_activity.waiting with things more descriptive. Vik Fearing sent in another revision of a patch to add an idle-in-transaction timeout. David Fetter sent in another revision of a patch to add weighted statistics (mean, stddev). David Rowley sent eight more revisions of a patch to enable parallel aggregation. Peter Geoghegan sent in a patch to refactor speculative insertion into unique indexes. Craig Ringer sent in two more revisions of a patch to fix an issue where logical decoding slots were not documented to be able to go backwards, but were able to from SQL. Craig Ringer sent in a patch to fix TAP to initialize $timed_out correctly if passed. Aleksander Alekseev sent in a patch to fix double variable initializations in policy.c. Amit Kapila and Robert Haas traded patches to push down target lists below gather nodes. Fabien COELHO and Álvaro Herrera traded patches to add pgbench stats per script, etc. Andres Freund and Thomas Munro traded patches to fix a performance regression. Artur Zakirov sent in a patch to allow numbers in email addresses in tsearch. Dmitry Dolgov sent in two more revisions of a patch to add a function which allows inserting into a JSONB object at an arbitrary point. Dmitry Ivanov sent in two more revisions of a patch to add phrase search to the tsearch extension. David Rowley and Haribabu Kommi traded patches for combining aggregates. Tomas Vondra sent in four more revisions of a patch to add multivariate statistics. Dilip Kumar and Robert Haas traded patches to help scale relation extension. Alexander Kuleshov sent in a patch to reduce three macros PG_CMD_PRINTF{1,2,3} to one. Mark Dilger and Kevin Grittner traded patches to remove gratuitous gender-specific wording from the code. Marisa Emerson sent in two more revisions of a patch to implement BSD Authentication. Haribabu Kommi sent in three more revisions of a patch to add pg_hba_lookup(). Stas Kelvich sent in four revisions of a patch to fix a bug in fd.c on OS X. Tomas Vondra sent in another revision of a patch to improve GROUP BY estimation. Kevin Grittner sent in another revision of a patch to implement "snapshot too old" configured by time. Teodor Sigaev sent in two more revisions of a patch to implement index lookups for OR clauses. Andrew Dunstan sent in two revisions of a patch to add enum support for btree_gi(n|st) supplied extensions. Tomas Vondra sent in another revision of a patch to fix an issue in pgstat. Robert Haas sent in a patch to fix an issue with the recently added memory management improvement for external sorts. Tomas Vondra and Fabien COELHO traded patches to fix outdated docs on pgbench. Petr Jelínek and Artur Zakirov traded patches to create generic logical WAL messages. Anastasia Lubennikova sent in another revision of a patch to compress away duplicates in a B-Tree index. Teodor Sigaev and Emre Hasegeli traded patches to add support for box type in SP-GiST index. Christoph Berg sent in a patch to create a pg_filedump for 9.5. Jeff Janes sent in a patch to allow zero weights in pgbench tests. Magnus Hagander sent in another revision of a patch to implement non-exclusive backups. Fabien COELHO sent in four more revisions of a patch to extend pgbench expressions with functions. Alexander Korotkov sent in three more revisions of a patch to move PinBuffer and UnpinBuffer to atomics. Tomas Vondra sent in another revision of a patch to enable index-only scans with partial indexes. Yuri Niyazov sent in a patch to improve the pg_upgrade documentation. Pavel Stěhule sent in another revision of a patch to add a \crosstabview to psql. Michaël Paquier and Ashutosh Bapat traded patches to fix some issues in the recently added FDW write pushdown patch. Christian Ullrich sent in two revisions of a patch to fix realm handling in SSPI auth. Anastasia Lubennikova sent in another revision of a patch to add covering + unique indexes. Julien Rouhaud sent in two more revisions of a patch to choose parallel degree. Jeff Janes sent in a patch to fix a typo in the vacuum progress documentation. Aleksander Alekseev sent in two more revisions of a patch to make PostgreSQL sanitizers-friendly. Abhijit Menon-Sen sent in a patch to implement ALTER FUNCTION x DEPENDS ON EXTENSION y. Artur Zakirov sent in another revision of a patch to add fuzzy substring searching with the pg_trgm extension. Grzegorz Sampolski sent in another revision of a patch to add an rhost entry to PAM auth. Constantin S. Pan sent in another revision of a patch to speed up GIN index builds with parallel workers. Craig Ringer sent in another revision of a patch to add timeline following for logical decoding. Michaël Paquier sent in two revisions of a patch to ensure that all messages from pg_rewind --debug get translated. Pavan Deolasee sent in two revisions of a patch to fix an issue where pg_xlogdump fails to handle WAL file with multi-page XLP_FIRST_IS_CONTRECORD data. Craig Ringer sent in two revisions of a patch to add a README for src/backend/replication/logical. Vitaly Burovoy and Michaël Paquier traded patches to fix an issue in searching path in jsonb_set when walking through JSONB array. Kyotaro HORIGUCHI sent in another revision of a patch to show dropped users' backends in pg_stat_activity. Fujii Masao and SAWADA Masahiko traded patches for support for N synchronous standby servers, N > 1. YUriy Zhuravlev sent in four revisions of a patch to add CINE to PREPARE. Roma Sokolov and Robert Haas traded patches to fix DROP OPERATOR to reset links to itself on commutator and negator. Amit Kapila sent in another revision of a patch to speed up CLOG access by increasing CLOG buffers. Stephen Frost sent in another revision of a patch to cause pg_dump to dump catalog ACLs. Aleksander Alekseev sent in another revision of a patch to fix lock contention for HASHHDR.mutex. Daniel Verité sent in another revision of a patch to fix pg_dump and COPY's handling of gigantic lines. Petr Jediný sent in another revision of a patch to add BRIN to the multicolumn indexes documentation. Gilles Darold sent in another revision of a patch to add pg_current_logfile(). Kaigai Kouhei sent in another revision of a patch to rework CustomScan serialization/deserialization. Tom Lane sent in a patch to fix SRF behavior in target lists. Michaël Paquier sent in another revision of a patch to add VS 2015 support in src/tools/msvc. Abhijit Menon-Sen sent in three more revisions of a patch to deal with extension dependencies that aren't quite 'e'. Thomas Munro sent in three more revisions of a patch to add "causal reads." Petr Jelínek sent in another revision of a patch to add a sequence access method and a gapless sequence implementation using same. Aleksander Alekseev sent in two revisions of a patch to fix some code duplication in heapam.c. Dagfinn Ilmari Mannsåker sent in three revisions of a patch to implement ALTER TYPE ... ALTER VALUE .. TO .. for enums. Robert Haas sent in a patch to measure waste memory for cache alignment correctness. Thomas Munro sent in a patch to fix a typo in a comment.
pgsql-announce by date: