== PostgreSQL Weekly News - July 06 2014 == - Mailing list pgsql-announce
From | David Fetter |
---|---|
Subject | == PostgreSQL Weekly News - July 06 2014 == |
Date | |
Msg-id | 20140707044417.GC15538@fetter.org Whole thread Raw |
List | pgsql-announce |
== PostgreSQL Weekly News - July 06 2014 == PGConf.EU 2014 in Madrid, Spain on October 21-24 is now open for registration. http://2014.pgconf.eu/registration/ PGDay.IT 2014 will take place in Prato on November the 7th 2014. The International Call For Papers is now open: http://2014.pgday.it/call-for-papers-en/ == PostgreSQL Product News == Database Designer for PostgreSQL 1.10.0, a CASE tool which works natively under the Windows OS family, released. http://microolap.com/products/database/postgresql-designer/download/ == PostgreSQL Jobs for July == http://archives.postgresql.org/pgsql-jobs/2014-07/threads.php == PostgreSQL Local == Char(14) and PGday UK will be held July 8 and 9, 2014. http://www.char14.info http://postgresqlusergroup.org.uk/ PgDay Portland, Oregon 2014 will be held Saturday September 6, 2014. https://wiki.postgresql.org/wiki/PDXPUGDay2014 Postgres Open 2014 will be in Chicago, IL, USA, September 17-19. Tickets and Tutorials now available for purchase. https://postgresopen.org/2014/tickets/ http://postgresopen.org/2014/callforpapers/ The sixth PGDay Cubano be held on 13 and 14 October 2014 in Habana. https://postgresql.uci.cu/?p=380 PostgreSQL Conference Europe 2014 will be held on October 21-24 in Madrid, Spain, at the Hotel Miguel Angel. http://2014.pgconf.eu/ == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm Pacific time. Please send English language ones to david@fetter.org, German language to pwn@pgug.de, Italian language to pwn@itpug.org. Spanish language to pwn@arpug.com.ar. == Applied Patches == Heikki Linnakangas pushed: - Fix and enhance the assertion of no palloc's in a critical section. The assertion failed if WAL_DEBUG or LWLOCK_STATS was enabled; fix that by using separate memory contexts for the allocations made within those code blocks. This patch introduces a mechanism for marking any memory context as allowed in a critical section. Previously ErrorContext was exempt as a special case. Instead of a blanket exception of the checkpointer process, only exempt the memory context used for the pending ops hash table. http://git.postgresql.org/pg/commitdiff/1c6821be31f91ab92547a8ed4246762c8cefb1b3 Andres Freund pushed: - Fix typos in the cluster_name commit. Thom Brown and Fujii Masao http://git.postgresql.org/pg/commitdiff/6647f59fd256927aaf9e35929b5c284d10d18839 - Check interrupts during logical decoding more frequently. When reading large amounts of preexisting WAL during logical decoding using the SQL interface we possibly could fail to check interrupts in due time. Similarly the same could happen on systems with a very high WAL volume while creating a new logical replication slot, independent of the used interface. Previously these checks where only performed in xlogreader's read_page callbacks, while waiting for new WAL to be produced. That's not sufficient though, if there's never a need to wait. Walsender's send loop already contains a interrupt check. Backpatch to 9.4 where the logical decoding feature was introduced. http://git.postgresql.org/pg/commitdiff/1cbc9480106241aaa8db112331e93d0a265b6db0 - Rename logical decoding's pg_llog directory to pg_logical. The old name wasn't very descriptive as of actual contents of the directory, which are historical snapshots in the snapshots/ subdirectory and mappingdata for rewritten tuples in mappings/. There's been a fair amount of discussion what would be a good name. I'm settling for pg_logical because it's likely that further data around logical decoding and replication will need saving in the future. Also add the missing entry for the directory into storage.sgml's list of PGDATA contents. Bumps catversion as the data directories won't be compatible. http://git.postgresql.org/pg/commitdiff/a36a8fa376631e59d006772bcd30f4f827d7e6b9 - Fix decoding of MULTI_INSERTs when rows other than the last are toasted. When decoding the results of a HEAP2_MULTI_INSERT (currently only generated by COPY FROM) toast columns for all but the last tuple weren't replaced by their actual contents before being handed to the output plugin. The reassembled toast datums where disregarded after every REORDER_BUFFER_CHANGE_(INSERT|UPDATE|DELETE) which is correct for plain inserts, updates, deletes, but not multi inserts - there we generate several REORDER_BUFFER_CHANGE_INSERTs for a single xl_heap_multi_insert record. To solve the problem add a clear_toast_afterwards boolean to ReorderBufferChange's union member that's used by modifications. All row changes but multi_inserts always set that to true, but multi_insert sets it only for the last change generated. Add a regression test covering decoding of multi_inserts - there was none at all before. Backpatch to 9.4 where logical decoding was introduced. Bug found by Petr Jelinek. http://git.postgresql.org/pg/commitdiff/1b86c81d2d255d3fb665ddc77c2bc3dfd751a1df Noah Misch pushed: - Don't prematurely free the BufferAccessStrategy in pgstat_heap(). This function continued to use it after heap_endscan() freed it. In passing, don't explicit create a strategy here. Instead, use the one created by heap_beginscan_strat(), if any. Back-patch to 9.2, where use of a BufferAccessStrategy here was introduced. http://git.postgresql.org/pg/commitdiff/9d0826c595e2a376f6645eb575a9d81d5f5ececf - Consistently pass an "unsigned char" to ctype.h functions. The isxdigit() calls relied on undefined behavior. The isascii() call was well-defined, but our prevailing style is to include the cast. Back-patch to 9.4, where the isxdigit() calls were introduced. http://git.postgresql.org/pg/commitdiff/333b7db8b39679acf0665b3fc4ad99cbc14fbba7 Bruce Momjian pushed: - pg_upgrade: update C comments about pg_dumpall. There were some C comments that hadn't been updated from the switch of using only pg_dumpall to using pg_dump and pg_dumpall, so update them. Also, don't bother using --schema-only for pg_dumpall --globals-only. Backpatch through 9.4 http://git.postgresql.org/pg/commitdiff/55863274d98556acf57013f64f545d9a1e640bba - pg_upgrade: no need to remove "members" files for pre-9.3 upgrades. Per analysis by Alvaro Herrera. Backpatch through 9.3 http://git.postgresql.org/pg/commitdiff/89cf2d5203085a3b74ab8ce1f323c93f8e410473 - pg_upgrade: preserve database and relation minmxid values. Also set these values for pre-9.3 old clusters that don't have values to preserve. Analysis by Alvaro Herrera. Backpatch through 9.3. http://git.postgresql.org/pg/commitdiff/a61daa14d56867e90dc011bbba52ef771cea6770 Tom Lane pushed: - Allow empty replacement strings in contrib/unaccent. This is useful in languages where diacritic signs are represented as separate characters; it's also one step towards letting unaccent be used for arbitrary substring substitutions. In passing, improve the user documentation for unaccent, which was sadly vague about some important details. Mohammad Alhashash, reviewed by Abhijit Menon-Sen http://git.postgresql.org/pg/commitdiff/97c40ce61465582b96944e41ed6ec06c2016b95c - Allow multi-character source strings in contrib/unaccent. This could be useful in languages where diacritic signs are represented as separate characters; more generally it supports using unaccent dictionaries for substring substitutions beyond narrowly conceived "diacritic removal". In any case, since the rule-file parser doesn't complain about multi-character source strings, it behooves us to do something unsurprising with them. http://git.postgresql.org/pg/commitdiff/1b2488731cc2c87cc9a4cb8d654e4d9981fdf9ac - Issue a WARNING about invalid rule file format in contrib/unaccent. We were already issuing a WARNING, albeit only elog not ereport, for duplicate source strings; so warning rather than just being stoically silent seems like the best thing to do here. Arguably both of these complaints should be upgraded to ERRORs, but that might be more behavioral change than people want. Note: the faulty line is already printed via an errcontext hook, so there's no need for more information than these messages provide. http://git.postgresql.org/pg/commitdiff/03a25cec8de3737924c9dd33bb868d4bc7a33ad5 - Fix inadequately-sized output buffer in contrib/unaccent. The output buffer size in unaccent_lexize() was calculated as input string length times pg_database_encoding_max_length(), which effectively assumes that replacement strings aren't more than one character. While that was all that we previously documented it to support, the code actually has always allowed replacement strings of arbitrary length; so if you tried to make use of longer strings, you were at risk of buffer overrun. To fix, use an expansible StringInfo buffer instead of trying to determine the maximum space needed a-priori. This would be a security issue if unaccent rules files could be installed by unprivileged users; but fortunately they can't, so in the back branches the problem can be labeled as improper configuration by a superuser. Nonetheless, a memory stomp isn't a nice way of reacting to improper configuration, so let's back-patch the fix. http://git.postgresql.org/pg/commitdiff/5a421a47eb1fc4398f42678c09e35aa72dc7cf18 - Improve handling of OOM score adjustment in sample Linux start script. Per a suggestion from Christoph Berg. http://git.postgresql.org/pg/commitdiff/f23425fa950fec3aff458de117037c9caadbc35c - Remove some useless code in the configure script. Almost ten years ago, commit e48322a6d6cfce1ec52ab303441df329ddbc04d1 broke the logic in ACX_PTHREAD by looping through all the possible flags rather than stopping with the first one that would work. This meant that $acx_pthread_ok was no longer meaningful after the loop; it would usually be "no", whether or not we'd found working thread flags. The reason nobody noticed is that Postgres doesn't actually use any of the symbols set up by the code after the loop. Rather than complicate things some more to make it work as designed, let's just remove all that dead code, and thereby save a few cycles in each configure run. http://git.postgresql.org/pg/commitdiff/2e8ce9ae46d15b2bfd34c2d53193da9858d3471a - Refactor CREATE/ALTER DATABASE syntax so options need not be keywords. Most of the existing option names are keywords anyway, but we can get rid of LC_COLLATE and LC_CTYPE as keywords known to the lexer/grammar. This immediately reduces the size of the grammar tables by about 8KB, and will save more when we add additional CREATE/ALTER DATABASE options in future. A side effect of the implementation is that the CONNECTION LIMIT option can now also be spelled CONNECTION_LIMIT. We choose not to document this, however. Vik Fearing, based on a suggestion by me; reviewed by Pavel Stehule http://git.postgresql.org/pg/commitdiff/15c82efd6994affd1d5654d13bc8911a9faff659 - Allow CREATE/ALTER DATABASE to manipulate datistemplate and datallowconn. Historically these database properties could be manipulated only by manually updating pg_database, which is error-prone and only possible for superusers. But there seems no good reason not to allow database owners to set them for their databases, so invent CREATE/ALTER DATABASE options to do that. Adjust a couple of places that were doing it the hard way to use the commands instead. Vik Fearing, reviewed by Pavel Stehule http://git.postgresql.org/pg/commitdiff/fbb1d7d73f8e23a3a61e702629c53cef48cb0918 - Add some errdetail to checkRuleResultList(). This function wasn't originally thought to be really user-facing, because converting a table to a view isn't something we expect people to do manually. So not all that much effort was spent on the error messages; in particular, while the code will complain that you got the column types wrong it won't say exactly what they are. But since we repurposed the code to also check compatibility of rule RETURNING lists, it's definitely user-facing. It now seems worthwhile to add errdetail messages showing exactly what the conflict is when there's a mismatch of column names or types. This is prompted by bug #10836 from Matthias Raffelsieper, which might have been forestalled if the error message had reported the wrong column type as being "record". Back-patch to 9.4, but not into older branches where the set of translatable error strings is supposed to be stable. http://git.postgresql.org/pg/commitdiff/7980ab30ecf36162699f138697e2ff5589d9063e - Improve support for composite types in PL/Python. Allow PL/Python functions to return arrays of composite types. Also, fix the restriction that plpy.prepare/plpy.execute couldn't handle query parameters or result columns of composite types. In passing, adopt a saner arrangement for where to release the tupledesc reference counts acquired via lookup_rowtype_tupdesc. The callers of PLyObject_ToCompositeDatum were doing the lookups, but then the releases happened somewhere down inside subroutines of PLyObject_ToCompositeDatum, which is bizarre and bug-prone. Instead release in the same function that acquires the refcount. Ed Behn and Ronan Dunklau, reviewed by Abhijit Menon-Sen http://git.postgresql.org/pg/commitdiff/8b6010b8350a1756cd85595705971df81b5ffc07 - Redesign API presented by nodeAgg.c for ordered-set and similar aggregates. The previous design exposed the input and output ExprContexts of the Agg plan node, but work on grouping sets has suggested that we'll regret doing that. Instead provide more narrowly-defined APIs that can be implemented in multiple ways, namely a way to get a short-term memory context and a way to register an aggregate shutdown callback. Back-patch to 9.4 where the bad APIs were introduced, since we don't want third-party code using these APIs and then having to change in 9.5. Andrew Gierth http://git.postgresql.org/pg/commitdiff/6f5034eda05c4946b65858fb8831d069f2873083 - Don't cache per-group context across the whole query in orderedsetaggs.c. Although nodeAgg.c currently uses the same per-group memory context for all groups of a query, that might change in future. Avoid assuming it. This costs us an extra AggCheckCallContext() call per group, but that's pretty cheap and is probably good from a safety standpoint anyway. Back-patch to 9.4 in case any third-party code copies this logic. Andrew Gierth http://git.postgresql.org/pg/commitdiff/ecd657974478fc713fdc3a625d648cd6a985e3e6 Robert Haas pushed: - Avoid copying index tuples when building an index. The previous code, perhaps out of concern for avoid memory leaks, formed the tuple in one memory context and then copied it to another memory context. However, this doesn't appear to be necessary, since index_form_tuple and the functions it calls take precautions against leaking memory. In my testing, building the tuple directly inside the sort context shaves several percent off the index build time. Rearrange things so we do that. Patch by me. Review by Amit Kapila, Tom Lane, Andres Freund. http://git.postgresql.org/pg/commitdiff/9f03ca915196dfc871804a1f8aad26207f601fd6 - Remove swpb-based spinlock implementation for ARMv5 and earlier. Per recent analysis by Andres Freund, this implementation is in fact unsafe, because ARMv5 has weak memory ordering, which means tha the CPU could move loads or stores across the volatile store performed by the default S_UNLOCK. We could try to fix this, but have no ARMv5 hardware to test on, so removing support seems better. We can still support ARMv5 systems on GCC versions new enough to have built-in atomics support for this platform, and can also re-add support for the old way if someone has hardware that can be used to test a fix. However, since the requirement to use a relatively-new GCC hasn't been an issue for ARMv6 or ARMv7, which lack the swpb instruction altogether, perhaps it won't be an issue for ARMv5 either. http://git.postgresql.org/pg/commitdiff/4893ccd02459fdb444f4f4b34c441eb899592879 Fujii Masao pushed: - Prevent psql from issuing BEGIN before ALTER SYSTEM when AUTOCOMMIT is off. The autocommit-off mode works by issuing an implicit BEGIN just before any command that is not already in a transaction block and is not itself a BEGIN or other transaction-control command, nor a command that cannot be executed inside a transaction block. This commit prevents psql from issuing such an implicit BEGIN before ALTER SYSTEM because it's not allowed inside a transaction block. Backpatch to 9.4 where ALTER SYSTEM was added. Report by Feike Steenbergen http://git.postgresql.org/pg/commitdiff/d97e98e8337e915e3404f7b2a8a6b966b6f5394f - Split out the description of page-level lock as new subsection in document. Michael Banck http://git.postgresql.org/pg/commitdiff/644d85351e6aa364d31d6c5402e4fd6a3e94e91d - Refactor pg_receivexlog main loop code, for readability. Previously the source codes for receiving the data and for polling the socket were included in pg_receivexlog main loop. This commit splits out them as separate functions. This is useful for improving the readability of main loop code and making the future pg_receivexlog-related patch simpler. http://git.postgresql.org/pg/commitdiff/74cbe966fe2d76de1d607d933c98c144dab58769 - Fix double-free bug of WAL streaming buffer in pg_receivexlog. This bug was introduced while refactoring in commit 74cbe96. http://git.postgresql.org/pg/commitdiff/4ccac92d08f22ef0f8df29ea6bd9d5e77bf1d246 Kevin Grittner pushed: - Smooth reporting of commit/rollback statistics. If a connection committed or rolled back any transactions within a PGSTAT_STAT_INTERVAL pacing interval without accessing any tables, the reporting of those statistics would be held up until the connection closed or until it ended a PGSTAT_STAT_INTERVAL interval in which it had accessed a table. This could result in under- reporting of transactions for an extended period, followed by a spike in reported transactions. While this is arguably a bug, the impact is minimal, primarily affecting, and being affected by, monitoring software. It might cause more confusion than benefit to change the existing behavior in released stable branches, so apply only to master and the 9.4 beta. Gurjeet Singh, with review and editing by Kevin Grittner, incorporating suggested changes from Abhijit Menon-Sen and Tom Lane. http://git.postgresql.org/pg/commitdiff/ac46de56eab9bb93c23c7f34070f3a16e7e76743 - Remove dead typeStruct variable from plpy_spi.c. Left behind by 8b6010b8350a1756cd85595705971df81b5ffc07. http://git.postgresql.org/pg/commitdiff/e254ff21d1286fe5c2b5ecdfa9bb6b15a7830fcd Peter Eisentraut pushed: - Use a separate temporary directory for the Unix-domain socket. Creating the Unix-domain socket in the build directory can run into name-length limitations. Therefore, create the socket file in the default temporary directory of the operating system. Keep the temporary data directory etc. in the build tree. http://git.postgresql.org/pg/commitdiff/f545d233ebce6971b6f9847680e48b679e707d22 - Support vpath builds in TAP tests http://git.postgresql.org/pg/commitdiff/0490db62871dc6888c52bd9465d9ade16ab8e795 == Rejected Patches (for now) == No one was disappointed this week :-) == Pending Patches == Noah Misch, Kyotaro HORIGUCHI and Etsuro Fujita traded patches around allowing foreign tables to be part of table inheritance hierarchies. Andres Freund sent in another revision of a patch to add a cluster_name GUC, which controls whether same is visible in ps output. Abhijit Menon-Sen and Marti Raudsepp traded patches to add a --stats option to pg_xlogdump. Pavel Stehule sent in two more revisions of a patch to allow psql to show only failed queries. Michael Paquier sent in another revision of a patch to extend MSVC scripts to support --with-extra-version. Heikki Linnakangas sent in another revision of a patch to change the WAL format and API. Michael Paquier and Ronan Dunklau traded patches for IMPORT FOREIGN SCHEMA. Tomas Vondra sent in three more revisions of a patch to make hash buckets grow appropriately. Kyotaro HORIGUCHI sent in a patch to break socket-blocking on termination. Kevin Grittner sent in another revision of a patch to add a C extension to test delta relations in AFTER triggers. Robert Haas and Andres Freund traded patches to simulate memory barriers on platforms where they are not available. Gurjeet Singh sent in another revision of a patch to send transaction commit/rollback stats to the stats collector unconditionally. Kyotaro HORIGUCHI sent in a patch to correct the documentation of ALTER USER SET local_preload_libraries. Ronan Dunklau sent in another revision of a patch to improve the functionality of arrays of composite types returned from PL/Python. Dilip Kumar sent in three more revisions of a patch to allow vacuumdb to use multiple cores in parallel. Michael Paquier sent two flocks of patches to fix an issue in WAL replay. Fujii Masao sent in another revision of a patch to make log_disconnections PGC_SUSET rather than PGC_SIGHUP. Michael Banck sent in a patch to add an additional documentation subsection for page-level locks in the explicit-locking section. Ian Lawrence Barwick sent in another revision of a patch to add a "RETURNING PRIMARY KEY" syntax extension to DML. Fabien COELHO sent in two more revisions of a patch to add a Gaussian distribution to pgbench. Peter Geoghegan sent in another revision of a patch to do better at HINTing an appropriate column within errorMissingColumn(). Tomas Vondra sent in another revision of a patch to tweak NTUP_PER_BUCKET. Tatsuo Ishii sent in another revision of a patch to add a shared_memory_type GUC. Abhijit Menon-Sen sent in two more revisions of a patch to introduce XLogLockBlockRangeForCleanup(). Rahila Syed sent in two more revisions of a patch to allow various compression algorithms for full page writes. Andrew (RhodiumToad) Gierth sent in two patch sets intended to be infrastructure for GROUPING SETS. Furuya Osamu sent in another revision of a patch add a synchronous mode to pg_receivexlog. Thomas Munro sent in a PoC patch to enable DISTINCT with btree skip scan (a.k.a. "loose index scan"). David Rowley sent in another revision of a patch to allow NOT IN to use ANTI joins. David Rowley sent in another revision of a patch to allow subquery LEFT JOIN removal where that would produce correct results. Andrew (RhodiumToad) Gierth sent in two revisions of a patch to fix a performance regression related to ScalarArrayOpExpr. Craig Ringer sent in a patch to improve bytea error messages. Emre Hasegeli sent in another revision of a patch to add selectivity estimation for inet operators. SAWADA Masahiko sent in another revision of a patch to add line number as prompt option to psql. Jeff Davis sent in another revision of a patch to make it possible for the LAG and LEAD window functions to ignore nulls.
pgsql-announce by date: