== PostgreSQL Weekly News - May 18 2014 == - Mailing list pgsql-announce
From | David Fetter |
---|---|
Subject | == PostgreSQL Weekly News - May 18 2014 == |
Date | |
Msg-id | 20140519032443.GA19829@fetter.org Whole thread Raw |
List | pgsql-announce |
== PostgreSQL Weekly News - May 18 2014 == PostgreSQL 9.4 Beta 1 released. TEST! http://www.postgresql.org/developer/beta PostgreSQL Conference Europe 2014 will be October 21-24 at Hotel Miguel Angel in Madrid, Spain. The CfP is open for talks in English and Spanish. http://2014.pgconf.eu/ == PostgreSQL Product News == pg_catcheck, a catalog integrity checker for PostgreSQL and Postgres Plus Advanced Server, released. https://github.com/EnterpriseDB/pg_catcheck PG Commander 1.3, a GUI made specifically for PostgreSQL on OSX, released. https://eggerapps.at/pgcommander/ psqlODBC 09.03.0300 released. http://www.postgresql.org/ftp/odbc/versions/ psycopg2 2.5.3, a Python connector for PostgreSQL, released. http://initd.org/psycopg/articles/2014/05/13/psycopg-253-released/ == PostgreSQL Jobs for May == http://archives.postgresql.org/pgsql-jobs/2014-05/threads.php == PostgreSQL Local == PGCon 2014, the world-wide developer conference for PostgreSQL, will be in Ottawa, Ontario, Canada May 20-24, 2014. http://www.pgcon.org/2014/ Char(14) and PGday UK will be held July 8 and 9, 2014. http://www.char14.info http://postgresqlusergroup.org.uk/ The sixth PGDay Cubano be held on 13 and 14 October 2014 in Habana. https://postgresql.uci.cu/?p=380 Postgres Open 2014 will be in Chicago, IL, USA, September 17-19. The CfP is open! http://postgresopen.org/2014/callforpapers/ PostgreSQL Conference Europe 2014 will be held on October 21-24 in Madrid, Spain, at the Hotel Miguel Angel. http://2014.pgconf.eu/ == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm Pacific time. Please send English language ones to david@fetter.org, German language to pwn@pgug.de, Italian language to pwn@itpug.org. Spanish language to pwn@arpug.com.ar. == Applied Patches == Heikki Linnakangas pushed: - Free PQresult on error in pg_receivexlog. The leak is fairly small and rare, but a leak nevertheless. Per Coverity report. Backpatch to 9.2, where pg_receivexlog was added. pg_basebackup shares the code, but it always exits on error, so there is no real leak. http://git.postgresql.org/pg/commitdiff/c890b488063e23f12b5ab01c9a6d254a034fcea2 - Initialize padding bytes in btree_gist varbit support. The code expands a varbit gist leaf key to a node key by copying the bit data twice in a varlen datum, as both the lower and upper key. The lower key was expanded to INTALIGN size, but the padding bytes were not initialized. That's a problem because when the lower/upper keys are compared, the padding bytes are used compared too, when the values are otherwise equal. That could lead to incorrect query results. REINDEX is advised for any btree_gist indexes on bit or bit varying data type, to fix any garbage padding bytes on disk. Per Valgrind, reported by Andres Freund. Backpatch to all supported versions. http://git.postgresql.org/pg/commitdiff/540ac7cea919623f691b20892ccc50e5e33b5009 - Fix harmless access to uninitialized memory. When cache invalidations arrive while ri_LoadConstraintInfo() is busy filling a new cache entry, InvalidateConstraintCacheCallBack() compares the - not yet initialized - oidHashValue field with the to-be-invalidated hash value. To fix, check whether the entry is already marked as invalid. Andres Freund http://git.postgresql.org/pg/commitdiff/f35aef415aa755c4e99f8c0ef83f9d14dbc48bb4 - Silence warnings about redefining popen on Mingw-w64. Mingw-w64 headers map popen/pclose to _popen and _pclose, but we want to use our popen wrapper rather than the Mingw-w64. #undef the Mingw's version. http://git.postgresql.org/pg/commitdiff/a82a17475d8a8a91df5fd1448b0f93772f590ecd - Misc message style and doc fixes. Euler Taveira http://git.postgresql.org/pg/commitdiff/ff810b4928882bfdc4ebe1ce603c47830aba3132 - Fix race condition in preparing a transaction for two-phase commit. To lock a prepared transaction's shared memory entry, we used to mark it with the XID of the backend. When the XID was no longer active according to the proc array, the entry was implicitly considered as not locked anymore. However, when preparing a transaction, the backend's proc array entry was cleared before transfering the locks (and some other state) to the prepared transaction's dummy PGPROC entry, so there was a window where another backend could finish the transaction before it was in fact fully prepared. To fix, rewrite the locking mechanism of global transaction entries. Instead of an XID, just have simple locked-or-not flag in each entry (we store the locking backend's backend id rather than a simple boolean, but that's just for debugging purposes). The backend is responsible for explicitly unlocking the entry, and to make sure that that happens, install a callback to unlock it on abort or process exit. Backpatch to all supported versions. http://git.postgresql.org/pg/commitdiff/bb38fb0d43c8d7ff54072bfd8bd63154e536b384 - Handle duplicate XIDs in txid_snapshot. The proc array can contain duplicate XIDs, when a transaction is just being prepared for two-phase commit. To cope, remove any duplicates in txid_current_snapshot(). Also ignore duplicates in the input functions, so that if e.g. you have an old pg_dump file that already contains duplicates, it will be accepted. Report and fix by Jan Wieck. Backpatch to all supported versions. http://git.postgresql.org/pg/commitdiff/8f9b9590d79fc1fc1ad08b207401acfdbb0bfac7 - Fix a couple of bugs in pg_recvlogical output to stdout. Don't close stdout on SIGHUP. Also, when a SIGHUP is received, close the file immediately, rather than only after receiving some more data from the server. Rename a variable, to avoid mentally dealing with double negatives (not unsynced means synced). http://git.postgresql.org/pg/commitdiff/00c26b6a60e7ceed29ddae34b0a69fe945c08f81 - Initialize tsId and dbId fields in WAL record of COMMIT PREPARED. Commit dd428c79 added dbId and tsId to the xl_xact_commit struct but missed that prepared transaction commits reuse that struct. Fix that. Because those fields were left unitialized, replaying a commit prepared WAL record in a hot standby node would fail to remove the relcache init file. That can lead to "could not open file" errors on the standby. Relcache init file only needs to be removed when a system table/index is rewritten in the transaction using two phase commit, so that should be rare in practice. In HEAD, the incorrect dbId/tsId values are also used for filtering in logical replication code, causing the transaction to always be filtered out. Analysis and fix by Andres Freund. Backpatch to 9.0 where hot standby was introduced. http://git.postgresql.org/pg/commitdiff/07a4a93a0e35a778c77ffbbbc18de29e859e18f0 - Open output file before sleeping in pg_recvlogical. Let's complain about e.g an invalid path or permission problem sooner rather than later. Before this patch, we would only try to open the output file after receiving the first decoded message from the server. http://git.postgresql.org/pg/commitdiff/e7873b74d9c1bb7a325960f52043db6de88e6e66 - Fix thinko in logical decoding of commit-prepared records. The decoding of prepared transaction commits accidentally used the XID of the transaction performing the COMMIT PREPARED, not the XID of the prepared transaction. Before bb38fb0d43c8d that lead to those transactions not being decoded, afterwards to a assertion failure. http://git.postgresql.org/pg/commitdiff/03e2b1017c0d360cae0b4d0a8bf5d6924536c834 - Fix test_decoding test case's check that slot has been dropped. pg_stat_replication shows connected replication clients. The ddl test case never has any replication clients connected, so querying pg_stat_replication is pointless. To check that a slot has been dropped correctly, query pg_replication_slots instead. Andres Freund http://git.postgresql.org/pg/commitdiff/afd0fcbc5d421454e7d55d949cca0721c9c135c3 - Add test case for logical decoding of prepared transactions. Andres Freund http://git.postgresql.org/pg/commitdiff/d900e192a3ef2dd73fae4522c35aadce6d72a534 - Update README, we don't do post-recovery cleanup actions anymore. transam/README explained how B-tree incomplete splits were tracked and fixed after recovery, as an example of handling complex actions that need multiple WAL records, but that's not how it works anymore. Explain the new paradigm. http://git.postgresql.org/pg/commitdiff/a3655dd4a5cee3917a7d1766e07e36013e7e8835 Fujii Masao pushed: - Fix typo in test_shm_mq document. Amit Langote http://git.postgresql.org/pg/commitdiff/a3632771bb66eec74c7e73c0d7a5f8cdd6719be2 Tom Lane pushed: - Ignore config.pl and buildenv.pl in src/tools/msvc. config.pl and buildenv.pl can be used to customize build settings when using MSVC. They should never get committed into the common source tree. Back-patch to 9.0; it looks like the rules were different in 8.4. Michael Paquier http://git.postgresql.org/pg/commitdiff/73011f35eca7f4a3973289d5813648abbc1074a0 - Be more wary in choice of timezone names to test make_timestamptz with. America/Metlakatla hasn't been in the IANA database all that long, so some installations might not have it. It does seem worthwhile to test with a fractional-minute GMT offset, but we can get that from almost any pre-1900 date; I chose Europe/Paris, whose LMT offset from Greenwich should be pretty darn well established. Also, assuming that Mars/Mons_Olympus will never be in the IANA database seems less than future-proof, so let's use a more fanciful location for the bad-zone-name check. Per complaint from Christoph Berg. http://git.postgresql.org/pg/commitdiff/66b737cd9a72e10df4de1867ae7b675bc48d478c - Make initdb throw error for bad locale values. Historically we've printed a complaint for a bad locale setting, but then fallen back to the environment default. Per discussion, this is not such a great idea, because rectifying an erroneous locale choice post-initdb (perhaps long after data has been loaded) could be enormously expensive. Better to complain and give the user a chance to double-check things. The behavior was particularly bad if the bad setting came from environment variables rather than a bogus command-line switch: in that case not only was there a fallback to C/SQL_ASCII, but the printed complaint was quite unhelpful. It's hard to be entirely sure what variables setlocale looked at, but we can at least give a hint where the problem might be. Per a complaint from Tomas Vondra. http://git.postgresql.org/pg/commitdiff/31a263237fa3eb6f36d58335fbdb82bab3136db3 - Code review for recent changes in relcache.c. rd_replidindex should be managed the same as rd_oidindex, and rd_keyattr and rd_idattr should be managed like rd_indexattr. Omissions in this area meant that the bitmapsets computed for rd_keyattr and rd_idattr would be leaked during any relcache flush, resulting in a slow but permanent leak in CacheMemoryContext. There was also a tiny probability of relcache entry corruption if we ran out of memory at just the wrong point in RelationGetIndexAttrBitmap. Otherwise, the fields were not zeroed where expected, which would not bother the code any AFAICS but could greatly confuse anyone examining the relcache entry while debugging. Also, create an API function RelationGetReplicaIndex rather than letting non-relcache code be intimate with the mechanisms underlying caching of that value (we won't even mention the memory leak there). Also, fix a relcache flush hazard identified by Andres Freund: RelationGetIndexAttrBitmap must not assume that rd_replidindex stays valid across index_open. The aspects of this involving rd_keyattr date back to 9.3, so back-patch those changes. http://git.postgresql.org/pg/commitdiff/b23b0f5588d2e2f15edff66e072e339a8c9616a7 - Fix version check for pg_upgrade line type crosscheck. Per buildfarm. http://git.postgresql.org/pg/commitdiff/4456763c78d2a94c42e67ce788fbe94a07aa12dc - In initdb, ensure stdout/stderr buffering behavior is what we expect. Since this program may print to either stdout or stderr, the relative ordering of its messages depends on the buffering behavior of those files. Force stdout to be line-buffered and stderr to be unbuffered, ensuring that the behavior will match standard Unix interactive behavior, even when stdout and stderr are rerouted to a file. Per complaint from Tomas Vondra. The particular case he pointed out is new in HEAD, but issues of the same sort could arise in any branch with other error messages, so back-patch to all branches. I'm unsure whether we might not want to do this in other client programs as well. For the moment, just fix initdb. http://git.postgresql.org/pg/commitdiff/2dc4f011fd61501cce507be78c39a2677690d44b - Fix unportable setvbuf() usage in initdb. In yesterday's commit 2dc4f011fd61501cce507be78c39a2677690d44b, I tried to force buffering of stdout/stderr in initdb to be what it is by default when the program is run interactively on Unix (since that's how most manual testing is done). This tripped over the fact that Windows doesn't support _IOLBF mode. We dealt with that a long time ago in syslogger.c by falling back to unbuffered mode on Windows. Export that solution in port.h and use it in initdb. Back-patch to 8.4, like the previous commit. http://git.postgresql.org/pg/commitdiff/f62d417825d6a8e26768a130a5f4c9e454d9dbe4 - Fix valgrind warning for btree_gist indexes on macaddr. The macaddr opclass stores two macaddr structs (each of size 6) in an index column that's declared as being of type gbtreekey16, ie 16 bytes. In the original coding this led to passing a palloc'd value of size 12 to the index insertion code, so that data would be fetched past the end of the allocated value during index tuple construction. This makes valgrind unhappy. In principle it could result in a SIGSEGV, though with the current implementation of palloc there's no risk since the 12-byte request size would be rounded up to 16 bytes anyway. To fix, add a field to struct gbtree_ninfo showing the declared size of the index datums, and use that in the palloc requests; and use palloc0 to be sure that any wasted bytes are cleanly initialized. Per report from Andres Freund. No back-patch since there's no current risk of a real problem. http://git.postgresql.org/pg/commitdiff/82bbb60c30dbff0633da34387ccab58d843379b5 - Fix a second cause of undersized pallocs for btree_gist indexes on macaddr. gbt_macad_union also allocated 12-byte structs where we really need 16. Per report from Andres Freund. No back-patch since there's no current risk of a real problem. http://git.postgresql.org/pg/commitdiff/39586bc1e971c136c4487b26eb7334dbbe23b102 - Suppress some more valgrind whining about btree_gist. A couple of functions didn't bother to zero out pad bytes in datums that would ultimately go to disk. Harmless, but valgrind doesn't know that. http://git.postgresql.org/pg/commitdiff/af215d8190e6ab170c02c24afd1be81f5a147481 - Make sure chr(int) can't create invalid UTF8 sequences. Several years ago we changed chr(int) so that if the database encoding is UTF8, it would interpret its argument as a Unicode code point and expand it into the appropriate multibyte sequence. However, we weren't sufficiently careful about checking validity of the input. According to RFC3629, UTF8 disallows code points above U+10FFFF (note that the predecessor standard RFC2279 was more liberal). Also, both versions of the UTF8 spec agree that Unicode surrogate-pair codes should never appear in UTF8. Because our encoding validity checks follow RFC3629, our failure to enforce these restrictions in chr() means it could be used to produce text strings that will be rejected when the database is dumped and reloaded. To ensure consistency with the input functions, let's actually apply pg_utf8_islegal() to the proposed output of chr(). Per discussion, this seems like too much of a behavioral change to back-patch, but it's not too late to squeeze it into 9.4. http://git.postgresql.org/pg/commitdiff/7894ac5004d5ec158f46296540d3cf49386d30e4 - Fix unaligned accesses in DecodeUpdate(). The xl_heap_header_len structures in an XLOG_HEAP_UPDATE record aren't necessarily aligned adequately. The regular replay function for these records is aware of that, but decode.c didn't get the memo. I'm not sure why the buildfarm failed to catch this; the test_decoding test certainly blows up real good on my old HPPA box. Also, I'm pretty sure that the address arithmetic was wrong for the case of XLOG_HEAP_CONTAINS_OLD and not XLOG_HEAP_CONTAINS_NEW_TUPLE, though this apparently can't happen when logical decoding is active. http://git.postgresql.org/pg/commitdiff/6c42b2b10af3f717030966b9f05867f3e065becc - Fix a bunch of functions that were declared static then defined not-static. Per testing with a compiler that whines about this. http://git.postgresql.org/pg/commitdiff/c1907f0cc49e38df9853b7547c9afce5204e4784 - Make fallback implementation of pg_memory_barrier() work. The fallback implementation involves acquiring and releasing a spinlock variable that is otherwise unreferenced --- not even to the extent of initializing it. This accidentally fails to fail on platforms where spinlocks should be initialized to zeroes, but elsewhere it results in a "stuck spinlock" failure during startup. I griped about this last July, and put in a hack that worked for gcc on HPPA, but didn't get around to fixing the general case. Per the discussion back then, the best thing to do seems to be to initialize dummy_spinlock in main.c. http://git.postgresql.org/pg/commitdiff/44cd47c1d49655c5dd9648bde8e267617c3735b4 - Fix two ancient memory-leak bugs in relcache.c. RelationCacheInsert() ignored the possibility that hash_search(HASH_ENTER) might find a hashtable entry already present for the same OID. However, that can in fact occur during recursive relcache load scenarios. When it did happen, we overwrote the pointer to the pre-existing Relation, causing a session-lifespan leakage of that entire structure. As far as is known, the pre-existing Relation would always have reference count zero by the time we arrive back at the outer insertion, so add code that deletes the pre-existing Relation if so. If by some chance its refcount is positive, elog a WARNING and allow the pre-existing Relation to be leaked as before. Also, AttrDefaultFetch() was sloppy about leaking the cstring form of the pg_attrdef.adbin value it's copying into the relcache structure. This is only a query-lifespan leakage, and normally not very significant, but it adds up during CLOBBER_CACHE testing. These bugs are of very ancient vintage, but I'll refrain from back-patching since there's no evidence that these leaks amount to anything in ordinary usage. http://git.postgresql.org/pg/commitdiff/078b2ed291c758e7125d72c3a235f128d40a232b - Ooops, I broke initdb with that last patch. That's what I get for not fully retesting the final version of the patch. The replace_allowed cross-check needs an additional special case for bootstrapping. http://git.postgresql.org/pg/commitdiff/0c19aaba22c4d904f7259ef11f7d6db7b5dd36a0 Bruce Momjian pushed: - doc: 9.4 release note adjustments. Report by Nicolas Barbier, Tatsuo Ishii, MauMau http://git.postgresql.org/pg/commitdiff/6de9a6c015d543c41933171cd6cf5d357597f19f - docs: use structfield instead of structname in 9.4 release notes. Where appropriate http://git.postgresql.org/pg/commitdiff/9af8bd51a10edaf454c6b5246d79eded73528fd2 - docs: 9.4 release notes adjustments. Patch by Andres Freund, slight adjustments by me http://git.postgresql.org/pg/commitdiff/062f53518927f9bfe1820578ce79d3180b1aa2ca - doc: auto-updatable view adjustments for 9.4 release notes. Report by Dean Rasheed http://git.postgresql.org/pg/commitdiff/e513087f6401a64b0d9f0b20cd2a51a7938ac645 - docs: mention windows quoting change in 9.4 release notes. Report by Heikki Linnakangas http://git.postgresql.org/pg/commitdiff/eb6144bb447ecbb036bfa08da85940cd50d49410 - doc: fix typo in 9.4 release note comments. Patch by Sergey Muraviov http://git.postgresql.org/pg/commitdiff/eefd90c4a95449f02b568d5370cfe6bcd1bf13e9 - doc: fix 9.4 release notes typo. Report by Dean Rasheed http://git.postgresql.org/pg/commitdiff/ac53295d667e7727d7b70ddf11d82c067870501f - pg_upgrade: error out on 'line' data type usage. The data type internal format changed in 9.4. Also mention this in the 9.4 release notes. http://git.postgresql.org/pg/commitdiff/bb797b640449d67f722d27660c718b68a692f8f9 - pg_update: slight major version adjustment http://git.postgresql.org/pg/commitdiff/2e6602dfc25d1f8e8a2e1df48c96440b02aa19c1 Noah Misch pushed: - Add Valgrind suppression for reorderbuffer padding bytes. Andres Freund http://git.postgresql.org/pg/commitdiff/5a90ac29c44f0dabf1e3a7b0443ba772cf206e2e Peter Eisentraut pushed: - pg_ctl: Write error messages to stderr http://git.postgresql.org/pg/commitdiff/c424c049189f758920b1a3bb961bc05c93a900df - doc: Clarify what files pg_basebackup omits from data directory http://git.postgresql.org/pg/commitdiff/4773c70aa4c938c367ee179f7301f70a4efabc0e - Fix whitespace http://git.postgresql.org/pg/commitdiff/2f8e68bd7ea3523977567369056c4b3e67e44fdc == Rejected Patches (for now) == No one was disappointed this week :-) == Pending Patches == Kaigai Kouhei sent in two revisions of a patch to fix an error in the regression tests for SEPgsql. Thomas Munro and Craig Ringer traded patches to implement SKIP LOCKED DATA. Sergey Muraviov sent in three more revisions of a patch to fix psql's wrapping in extended mode with the default pager. Amit Kapila sent in two revisions of a patch to help scale shared buffer eviction by reducing contention around BufFreelistLock. Emre Hasegeli sent in another revision of a patch to add a selectivity estimation for inet operators. Jeff Janes sent in a patch to fix pg_dump warnings in the MinGW build. Jeff Janes sent in a patch to fix an issue with autovacuum scheduling starvation and frenzy. Craig Ringer sent in a patch to fix an issue where dynamic background workers with BGW_NEVER_RESTART worker restarted on FatalError. Steve Singer and Andres Freund traded patches to fix an issue where CREATE REPLICATION SLOT would fail on a timeout. Andres Freund sent in two revisions of a patch to fix an issue where %d in log_line_prefix doesn't work for bg/autovacuum workers. Heikki Linnakangas sent in another revision of a patch to change the WAL format with corresponding changes to the API. Marko Kreen sent in a patch to clarify the crypto documentation for SSL. Heikki Linnakangas sent in a patch to fix a checksum error in recovery with btree index. David Rowley sent in another revision of a patch to allow join removals for more join types. Fabrízio de Royes Mello sent in a patch to fix an issue where psql's \db+ does not have a size column. Fabrízio de Royes Mello sent in a patch to fix a wrong name in the documentation of the pg_stat_replication view.
pgsql-announce by date: