== PostgreSQL Weekly News - December 01 2013 == - Mailing list pgsql-announce
From | David Fetter |
---|---|
Subject | == PostgreSQL Weekly News - December 01 2013 == |
Date | |
Msg-id | 20131202073647.GA2336@fetter.org Whole thread Raw |
List | pgsql-announce |
== PostgreSQL Weekly News - December 01 2013 == == PostgreSQL Product News == pg_monz, a Zabbix template for monitoring PostgreSQL, released. http://pg-monz.github.io/pg_monz/index-en.html Pyrseas 0.7.0, a toolkit for PostgreSQL version control, released. https://github.com/pyrseas/Pyrseas == PostgreSQL Jobs for December == http://archives.postgresql.org/pgsql-jobs/2013-12/threads.php == PostgreSQL Local == FOSDEM PGDay, a one day conference held before FOSDEM in Brussels, Belgium, will be on Jan 31st, 2014. Details: http://fosdem2014.pgconf.eu/ CfP is open: http://fosdem2014.pgconf.eu/callforpapers/ == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm Pacific time. Please send English language ones to david@fetter.org, German language to pwn@pgug.de, Italian language to pwn@itpug.org. Spanish language to pwn@arpug.com.ar. == Applied Patches == Michael Meskes pushed: - Fix ecpg parsing of sizeof(). The last fix used the wrong non-terminal to define valid types. http://git.postgresql.org/pg/commitdiff/ef8b3b00b55ef1feedb352a8a26f94845b6437dc - More improvement to comment parsing in ecpg. ECPG is not supposed to allow and output nested comments in C. These comments are only allowed in the SQL parts and must not be written into the C file. Also the different handling of different comments is documented. http://git.postgresql.org/pg/commitdiff/05b476c2983a8d38bf6a0254553b4a0776896aa7 - ECPG: Make the preprocessor emit ';' if the variable type for a list of variables is varchar. This fixes this test case: int main(void) { exec sql begin declare section; varchar a[50], b[50]; exec sql end declare section; return 0; } Since varchars are internally turned into custom structs and the type name is emitted for these variable declarations, the preprocessed code previously had: struct varchar_1 { ... } a _,_ struct varchar_2 { ... } b ; The comma in the generated C file was a syntax error. There are no regression test changes since it's not exercised. Patch by Boszormenyi Zoltan <zb@cybertec.at> http://git.postgresql.org/pg/commitdiff/b46fa321003e4d07d881c2583eb1126e8fadabdc - ECPG: Simplify free_variable(). Patch by Boszormenyi Zoltan <zb@cybertec.at> http://git.postgresql.org/pg/commitdiff/f641fc86fbe72a84a9d0a32279c4f9941bde6b6a - ECPG: Add EXEC SQL CLOSE C to the tests. Patch by Boszormenyi Zoltan <zb@cybertec.at> http://git.postgresql.org/pg/commitdiff/1ec4c56e76714d3894cfc301212ec98674720e3b - ECPG: Free the malloc()'ed variables in the test so it comes out clean on Valgrind runs. Patch by Boszormenyi Zoltan <zb@cybertec.at> http://git.postgresql.org/pg/commitdiff/db58e8ff7cec9a14418d6dccd33bbe186d10c1f4 - ECPG: Fix offset to NULL/size indicator array. Patch by Boszormenyi Zoltan <zb@cybertec.at> http://git.postgresql.org/pg/commitdiff/d2542f9270e9587f54a63fd6ffbdd124abc6f80c - Documentation fix for ecpg. The latest fixes removed a limitation that was still in the docs, so Zoltan updated the docs, too. http://git.postgresql.org/pg/commitdiff/2390f2b2d3ef708c282846457b6fd837a0c08061 - ECPG: Fix searching for quoted cursor names case-sensitively. Patch by Böszörményi Zoltán <zb@cybertec.at> http://git.postgresql.org/pg/commitdiff/51867a0f9b5c6d15c541e19c202520e95c5950a0 Bruce Momjian pushed: - Change SET LOCAL/CONSTRAINTS/TRANSACTION and ABORT behavior. Change SET LOCAL/CONSTRAINTS/TRANSACTION behavior outside of a transaction block from error (post-9.3) to warning. (Was nothing in <= 9.3.) Also change ABORT outside of a transaction block from notice to warning. http://git.postgresql.org/pg/commitdiff/a6542a4b6870a019cd952d055d2e7af2da2fe102 - pg_buffercache docs: adjust order of fields. Adjust order of fields to match view order. Jaime Casanova http://git.postgresql.org/pg/commitdiff/9ef780d4d7b4466c8ba9ab5cdcd7d326a35ecbe1 - pg_upgrade: Handle default_transaction_read_only settings. Setting default_transaction_read_only=true could prevent pg_upgrade from completing, so prepend default_transaction_read_only=false to PGOPTIONS. http://git.postgresql.org/pg/commitdiff/e7d56aee2d1805cd9d280c0e6f8c8af80a13aa3e Jeff Davis pushed: - Add missing entry for session_preload_libraries in sample config. The omission was apparently an oversight in the original patch. http://git.postgresql.org/pg/commitdiff/7cc0ba9f17fdae610187411439e8c01b26da15ac Heikki Linnakangas pushed: - Handle domains over arrays like plain arrays in PL/python. Domains over arrays are now converted to/from python lists when passed as arguments or return values. Like regular arrays. This has some potential to break applications that rely on the old behavior that they are passed as strings, but in practice there probably aren't many such applications out there. Rodolfo Campero http://git.postgresql.org/pg/commitdiff/37364c63115a52b4dc7ea280aa5b358abd4a9c38 - Oops, forgot to "git add" last minute changes to regression test. http://git.postgresql.org/pg/commitdiff/4c83e0353f2c396e9e89fee564d9a5b868d5c3b2 - Don't update relfrozenxid if any pages were skipped. Vacuum recognizes that it can update relfrozenxid by checking whether it has processed all pages of a relation. Unfortunately it performed that check after truncating the dead pages at the end of the relation, and used the new number of pages to decide whether all pages have been scanned. If the new number of pages happened to be smaller or equal to the number of pages scanned, it incorrectly decided that all pages were scanned. This can lead to relfrozenxid being updated, even though some pages were skipped that still contain old XIDs. That can lead to data loss due to xid wraparounds with some rows suddenly missing. This likely has escaped notice so far because it takes a large number (~2^31) of xids being used to see the effect, while a full-table vacuum before that would fix the issue. The incorrect logic was introduced by commit b4b6923e03f4d29636a94f6f4cc2f5cf6298b8c8. Backpatch this fix down to 8.4, like that commit. Andres Freund, with some modifications by me. http://git.postgresql.org/pg/commitdiff/82b43f7df2036d06b4410721f77512969846b6d0 - Fix plpython3 expected output. I neglected this in the previous commit that updated the plpython2 output, which I forgot to "git add" earlier. As pointed out by Rodolfo Campero and Marko Kreen. http://git.postgresql.org/pg/commitdiff/4118f7e8ede8a3616189b53983aea293fd0b3cb1 - More GIN refactoring. Separate the insertion payload from the more static portions of GinBtree. GinBtree now only contains information related to searching the tree, and the information of what to insert is passed separately. Add root block number to GinBtree, instead of passing it around all the functions as argument. Split off ginFinishSplit() from ginInsertValue(). ginFinishSplit is responsible for finding the parent and inserting the downlink to it. http://git.postgresql.org/pg/commitdiff/ce5326eed386959aac7a322880896ddeade7fd52 - Get rid of the post-recovery cleanup step of GIN page splits. Replace it with an approach similar to what GiST uses: when a page is split, the left sibling is marked with a flag indicating that the parent hasn't been updated yet. When the parent is updated, the flag is cleared. If an insertion steps on a page with the flag set, it will finish split before proceeding with the insertion. The post-recovery cleanup mechanism was never totally reliable, as insertion to the parent could fail e.g because of running out of memory or disk space, leaving the tree in an inconsistent state. This also divides the responsibility of WAL-logging more clearly between the generic ginbtree.c code, and the parts specific to entry and posting trees. There is now a common WAL record format for insertions and deletions, which is written by ginbtree.c, followed by tree-specific payload, which is returned by the placetopage- and split- callbacks. http://git.postgresql.org/pg/commitdiff/631118fe1e8f66fd15eb0013dd9940d9146a5096 - Fix gin_desc routine to match the WAL format. In the GIN incomplete-splits patch, I used BlockIdDatas to store the block number of left and right children, when inserting a downlink after a split to an internal page posting list page. But gin_desc thought they were stored as BlockNumbers. http://git.postgresql.org/pg/commitdiff/97c19e6c38ebc5e300539f6985ef3e05398c8f46 - Another gin_desc fix. The number of items inserted was incorrectly printed as if it was a boolean. http://git.postgresql.org/pg/commitdiff/2fe69cacff4e92201cb3e3de7748f3e1d51d9e25 Peter Eisentraut pushed: - doc: Add id to index in XSLT build. That way, the HTML file name of the index will be the same as currently for the DSSSL build. http://git.postgresql.org/pg/commitdiff/3803ff985c7940af32912b158322cce399952e76 - Implement information_schema.parameters.parameter_default column. Reviewed-by: Ali Dar <ali.munir.dar@gmail.com> Reviewed-by: Amit Khandekar <amit.khandekar@enterprisedb.com> Reviewed-by: Rodolfo Campero <rodolfo.campero@anachronics.com> http://git.postgresql.org/pg/commitdiff/85ed91ee7da8f560c3443733cbda2862cd25facd - doc: Put data types in alphabetical order. From: Andreas Karlsson <andreas@proxel.se> http://git.postgresql.org/pg/commitdiff/a607b6903e90383e408d5085ca9b93a8a22392c1 - doc: Set chunk.first.sections in XSLT, for consistency with DSSSL output http://git.postgresql.org/pg/commitdiff/b6ecde8ed4799720eaa1c015e325012dbad095fa - doc: Allow selecting web site CSS style sheet in XSLT HTML build http://git.postgresql.org/pg/commitdiff/384eb1d40db6fe97678cb00e57365e785cb85eff - doc: Enhance documentation of ssl_ciphers setting a bit http://git.postgresql.org/pg/commitdiff/50107ee7a58f36568997db84df641fde122f3132 - doc: Simplify handling of variablelists in XSLT build. The previously used custom template is no longer necessary because parameters provided by the standard style sheet can achieve the same outcome. http://git.postgresql.org/pg/commitdiff/1eafea5d1bc5cf934eaf96e65115841fe3829986 - Remove use of obsolescent Autoconf macros. Remove the use of the following macros, which are obsolescent according to the Autoconf documentation: AC_C_CONST AC_C_STRINGIZE AC_C_VOLATILE AC_FUNC_MEMCMP http://git.postgresql.org/pg/commitdiff/34fa72ec9cfc1f5c4b11b67b2ce7eac6caca9fa5 - doc: Disable preface.autolabel in XSLT. The makes the output more consistent with the existing DSSSL setup. http://git.postgresql.org/pg/commitdiff/3c81b5c1d28faaacbd56583ebf00dd5f3db57c31 - Translation updates http://git.postgresql.org/pg/commitdiff/3e3520cf7a70aa85f55a7be37924aaa1809a4be3 Fujii Masao pushed: - Fix typo in release note. Backpatch to 9.1. Josh Kupershmidt http://git.postgresql.org/pg/commitdiff/551c78281b47b8ab12cfc142b8da68db8473e591 - Add --xlogdir option to pg_basebackup, for specifying the pg_xlog directory. Haribabu kommi, slightly modified by me. http://git.postgresql.org/pg/commitdiff/d1b88f6b3660753771895a4441b6bb4ccc9bcaac Tom Lane pushed: - Minor corrections in lmgr/README. Correct an obsolete statement that no backend touches another backend's PROCLOCK lists. This was probably wrong even when written (the deadlock checker looks at everybody's lists), and it's certainly quite wrong now that fast-path locking can require creation of lock and proclock objects on behalf of another backend. Also improve some statements in the hot standby explanation, and do one or two other trivial bits of wordsmithing/ reformatting. http://git.postgresql.org/pg/commitdiff/8c84803e14b1ba2025c37dcc1ff1c41dd6264fa0 - Fix stale-pointer problem in fast-path locking logic. When acquiring a lock in fast-path mode, we must reset the locallock object's lock and proclock fields to NULL. They are not necessarily that way to start with, because the locallock could be left over from a failed lock acquisition attempt earlier in the transaction. Failure to do this led to all sorts of interesting misbehaviors when LockRelease tried to clean up no-longer-related lock and proclock objects in shared memory. Per report from Dan Wood. In passing, modify LockRelease to elog not just Assert if it doesn't find lock and proclock objects for a formerly fast-path lock, matching the code in FastPathGetRelationLockEntry and LockRefindAndRelease. This isn't a bug but it will help in diagnosing any future bugs in this area. Also, modify FastPathTransferRelationLocks and FastPathGetRelationLockEntry to break out of their loops over the fastpath array once they've found the sole matching entry. This was inconsistently done in some search loops and not others. Improve assorted related comments, too. Back-patch to 9.2 where the fast-path mechanism was introduced. http://git.postgresql.org/pg/commitdiff/7db285afc9cf51763c5f49c453f8d189dca17b9d - Fix latent(?) race condition in LockReleaseAll. We have for a long time checked the head pointer of each of the backend's proclock lists and skipped acquiring the corresponding locktable partition lock if the head pointer was NULL. This was safe enough in the days when proclock lists were changed only by the owning backend, but it is pretty questionable now that the fast-path patch added cases where backends add entries to other backends' proclock lists. However, we don't really wish to revert to locking each partition lock every time, because in simple transactions that would add a lot of useless lock/unlock cycles on already-heavily-contended LWLocks. Fortunately, the only way that another backend could be modifying our proclock list at this point would be if it was promoting a formerly fast-path lock of ours; and any such lock must be one that we'd decided not to delete in the previous loop over the locallock table. So it's okay if we miss seeing it in this loop; we'd just decide not to delete it again. However, once we've detected a non-empty list, we'd better re-fetch the list head pointer after acquiring the partition lock. This guards against possibly fetching a corrupt-but-non-null pointer if pointer fetch/store isn't atomic. It's not clear if any practical architectures are like that, but we've never assumed that before and don't wish to start here. In any case, the situation certainly deserves a code comment. While at it, refactor the partition traversal loop to use a for() construct instead of a while() loop with goto's. Back-patch, just in case the risk is real and not hypothetical. http://git.postgresql.org/pg/commitdiff/da8a7160893c463690146c7b30220b90c593ec15 - Fix assorted race conditions in the new timeout infrastructure. Prevent handle_sig_alarm from losing control partway through due to a query cancel (either an asynchronous SIGINT, or a cancel triggered by one of the timeout handler functions). That would at least result in failure to schedule any required future interrupt, and might result in actual corruption of timeout.c's data structures, if the interrupt happened while we were updating those. We could still lose control if an asynchronous SIGINT arrives just as the function is entered. This wouldn't break any data structures, but it would have the same effect as if the SIGALRM interrupt had been silently lost: we'd not fire any currently-due handlers, nor schedule any new interrupt. To forestall that scenario, forcibly reschedule any pending timer interrupt during AbortTransaction and AbortSubTransaction. We can avoid any extra kernel call in most cases by not doing that until we've allowed LockErrorCleanup to kill the DEADLOCK_TIMEOUT and LOCK_TIMEOUT events. Another hazard is that some platforms (at least Linux and *BSD) block a signal before calling its handler and then unblock it on return. When we longjmp out of the handler, the unblock doesn't happen, and the signal is left blocked indefinitely. Again, we can fix that by forcibly unblocking signals during AbortTransaction and AbortSubTransaction. These latter two problems do not manifest when the longjmp reaches postgres.c, because the error recovery code there kills all pending timeout events anyway, and it uses sigsetjmp(..., 1) so that the appropriate signal mask is restored. So errors thrown outside any transaction should be OK already, and cleaning up in AbortTransaction and AbortSubTransaction should be enough to fix these issues. (We're assuming that any code that catches a query cancel error and doesn't re-throw it will do at least a subtransaction abort to clean up; but that was pretty much required already by other subsystems.) Lastly, ProcSleep should not clear the LOCK_TIMEOUT indicator flag when disabling that event: if a lock timeout interrupt happened after the lock was granted, the ensuing query cancel is still going to happen at the next CHECK_FOR_INTERRUPTS, and we want to report it as a lock timeout not a user cancel. Per reports from Dan Wood. Back-patch to 9.3 where the new timeout handling infrastructure was introduced. We may at some point decide to back-patch the signal unblocking changes further, but I'll desist from that until we hear actual field complaints about it. http://git.postgresql.org/pg/commitdiff/16e1b7a1b7f7ffd8a18713e83c8cd72c9ce48e07 - Be sure to release proc->backendLock after SetupLockInTable() failure. The various places that transferred fast-path locks to the main lock table neglected to release the PGPROC's backendLock if SetupLockInTable failed due to being out of shared memory. In most cases this is no big deal since ensuing error cleanup would release all held LWLocks anyway. But there are some hot-standby functions that don't consider failure of FastPathTransferRelationLocks to be a hard error, and in those cases this oversight could lead to system lockup. For consistency, make all of these places look the same as FastPathTransferRelationLocks. Noted while looking for the cause of Dan Wood's bugs --- this wasn't it, but it's a bug anyway. http://git.postgresql.org/pg/commitdiff/8b151558c837ea5fccccd84adde251eeddde81f0 - Fix assorted issues in pg_ctl's pgwin32_CommandLine(). Ensure that the invocation command for postgres or pg_ctl runservice double-quotes the executable's pathname; failure to do this leads to trouble when the path contains spaces. Also, ensure that the path ends in ".exe" in both cases and uses backslashes rather than slashes as directory separators. The latter issue is reported to confuse some third-party tools such as Symantec Backup Exec. Also, rewrite the function to avoid buffer overrun issues by using a PQExpBuffer instead of a fixed-size static buffer. Combinations of very long executable pathnames and very long data directory pathnames could have caused trouble before, for example. Back-patch to all active branches, since this code has been like this for a long while. Naoya Anzai and Tom Lane, reviewed by Rajeev Rastogi http://git.postgresql.org/pg/commitdiff/79193c75f8c78ff494f32cd7b7ee79d7166c225b - Editorial corrections to the October 2013 minor-release notes. This is mostly to fix incorrect migration instructions: since the preceding minor releases advised reindexing some GIST indexes, it's important that we back-link to that advice rather than earlier instances. Also improve some bug descriptions and fix a few typos. No back-patch yet; these files will get copied into the back branches later in the release process. http://git.postgresql.org/pg/commitdiff/47960354024eac823981d7f264a0cf804b0c42c9 - Update time zone data files to tzdata release 2013h. DST law changes in Argentina, Brazil, Jordan, Libya, Liechtenstein, Morocco, Palestine. New timezone abbreviations WIB, WIT, WITA for Indonesia. http://git.postgresql.org/pg/commitdiff/335470251d69c5c63a864c706db02fe173576834 - Draft release notes for 9.3.2. I'm putting these up for review before I start to extract the relevant subsets for the older branches. It'll be easier to make any suggested wording improvements at this stage. http://git.postgresql.org/pg/commitdiff/23e796de15567e9d31e8f9e8661828179f24a7be Kevin Grittner pushed: - Minor correction of READ COMMITTED isolation level docs. Per report from AK. http://git.postgresql.org/pg/commitdiff/89ba815092f9ab0b153aa67328a7c18431d809d7 - Fix pg_dumpall to work for databases flagged as read-only. pg_dumpall's charter is to be able to recreate a database cluster's contents in a virgin installation, but it was failing to honor that contract if the cluster had any ALTER DATABASE SET default_transaction_read_only settings. By including a SET command for the connection for each connection opened by pg_dumpall output, errors are avoided and the source cluster is successfully recreated. There was discussion of whether to also set this for the connection applying pg_dump output, but it was felt that it was both less appropriate in that context, and far easier to work around. Backpatch to all supported branches. http://git.postgresql.org/pg/commitdiff/4bd371f6f886446b6c0886f088992e6f3d4ab4a2 Alvaro Herrera pushed: - Cope with heap_fetch failure while locking an update chain. The reason for the fetch failure is that the tuple was removed because it was dead; so the failure is innocuous and can be ignored. Moreover, there's no need for further work and we can return success to the caller immediately. EvalPlanQualFetch is doing something very similar to this already. Report and test case from Andres Freund in 20131124000203.GA4403@alap2.anarazel.de http://git.postgresql.org/pg/commitdiff/e518fa7adfc28e0d0a99fe4e5711386d9a5c6532 - Compare Xmin to previous Xmax when locking an update chain. Not doing so causes us to traverse an update chain that has been broken by concurrent page pruning. All other code that traverses update chains uses this check as one of the cases in which to stop iterating, so replicate it here too. Failure to do so leads to erroneous CLOG, subtrans or multixact lookups. Per discussion following the bug report by J Smith in CADFUPgc5bmtv-yg9znxV-vcfkb+JPRqs7m2OesQXaM_4Z1JpdQ@mail.gmail.com as diagnosed by Andres Freund. http://git.postgresql.org/pg/commitdiff/e4828e9ccba731178dd77aed078db7ceb0e1e8d1 - Use a more granular approach to follow update chains. Instead of simply checking the KEYS_UPDATED bit, we need to check whether each lock held on the future version of the tuple conflicts with the lock we're trying to acquire. Per bug report #8434 by Tomonari Katsumata http://git.postgresql.org/pg/commitdiff/247c76a989097f1b4ab6fae898f24e75aa27fc1b - Don't try to set InvalidXid as page pruning hint. If a transaction updates/deletes a tuple just before aborting, and a concurrent transaction tries to prune the page concurrently, the pruner may see HeapTupleSatisfiesVacuum return HEAPTUPLE_DELETE_IN_PROGRESS, but a later call to HeapTupleGetUpdateXid() return InvalidXid. This would cause an assertion failure in development builds, but would be otherwise Mostly Harmless. Fix by checking whether the updater Xid is valid before trying to apply it as page prune point. Reported by Andres in 20131124000203.GA4403@alap2.anarazel.de http://git.postgresql.org/pg/commitdiff/c235a6a589bbd0a24e54fdb0df28979c9fb09463 - Unbreak buildfarm. I removed an intermediate commit before pushing and forgot to test the resulting tree :-( http://git.postgresql.org/pg/commitdiff/d51a8c52bae6f95af100572a1df94cc3362d2059 - Don't TransactionIdDidAbort in HeapTupleGetUpdateXid. It is dangerous to do so, because some code expects to be able to see what's the true Xmax even if it is aborted (particularly while traversing HOT chains). So don't do it, and instead rely on the callers to verify for abortedness, if necessary. Several race conditions and bugs fixed in the process. One isolation test changes the expected output due to these. This also reverts commit c235a6a589b, which is no longer necessary. Backpatch to 9.3, where this function was introduced. Andres Freund http://git.postgresql.org/pg/commitdiff/1ce150b7bb14105ddc190c5f1acf2ae1a9b2854a - Truncate pg_multixact/'s contents during crash recovery. Commit 9dc842f08 of 8.2 era prevented MultiXact truncation during crash recovery, because there was no guarantee that enough state had been setup, and because it wasn't deemed to be a good idea to remove data during crash recovery anyway. Since then, due to Hot-Standby, streaming replication and PITR, the amount of time a cluster can spend doing crash recovery has increased significantly, to the point that a cluster may even never come out of it. This has made not truncating the content of pg_multixact/ not defensible anymore. To fix, take care to setup enough state for multixact truncation before crash recovery starts (easy since checkpoints contain the required information), and move the current end-of-recovery actions to a new TrimMultiXact() function, analogous to TrimCLOG(). At some later point, this should probably done similarly to the way clog.c is doing it, which is to just WAL log truncations, but we can't do that for the back branches. Back-patch to 9.0. 8.4 also has the problem, but since there's no hot standby there, it's much less pressing. In 9.2 and earlier, this patch is simpler than in newer branches, because multixact access during recovery isn't required. Add appropriate checks to make sure that's not happening. Andres Freund http://git.postgresql.org/pg/commitdiff/1df0122daa6510eed4146033379a5055f66f5a8e - Fix full-table-vacuum request mechanism for MultiXactIds. While autovacuum dutifully launched anti-multixact-wraparound vacuums when the multixact "age" was reached, the vacuum code was not aware that it needed to make them be full table vacuums. As the resulting partial-table vacuums aren't capable of actually increasing relminmxid, autovacuum continued to launch anti-wraparound vacuums that didn't have the intended effect, until age of relfrozenxid caused the vacuum to finally be a full table one via vacuum_freeze_table_age. To fix, introduce logic for multixacts similar to that for plain TransactionIds, using the same GUCs. Backpatch to 9.3, where permanent MultiXactIds were introduced. Andres Freund, some cleanup by Álvaro http://git.postgresql.org/pg/commitdiff/f54106f77e6d71cbb3fa0924095e5142341fde2b - Replace hardcoded 200000000 with autovacuum_freeze_max_age. Parts of the code used autovacuum_freeze_max_age to determine whether anti-multixact-wraparound vacuums are necessary, while others used a hardcoded 200000000 value. This leads to problems when autovacuum_freeze_max_age is set to a non-default value. Use the latter everywhere. Backpatch to 9.3, where vacuuming of multixacts was introduced. Andres Freund http://git.postgresql.org/pg/commitdiff/76a31c689c627268067175b5d0687ce0dac9a4f4 - Fix a couple of bugs in MultiXactId freezing. Both heap_freeze_tuple() and heap_tuple_needs_freeze() neglected to look into a multixact to check the members against cutoff_xid. This means that a very old Xid could survive hidden within a multi, possibly outliving its CLOG storage. In the distant future, this would cause clog lookup failures: ERROR: could not access status of transaction 3883960912 DETAIL: Could not open file "pg_clog/0E78": No such file or directory. This mostly was problematic when the updating transaction aborted, since in that case the row wouldn't get pruned away earlier in vacuum and the multixact could possibly survive for a long time. In many cases, data that is inaccessible for this reason way can be brought back heuristically. As a second bug, heap_freeze_tuple() didn't properly handle multixacts that need to be frozen according to cutoff_multi, but whose updater xid is still alive. Instead of preserving the update Xid, it just set Xmax invalid, which leads to both old and new tuple versions becoming visible. This is pretty rare in practice, but a real threat nonetheless. Existing corrupted rows, unfortunately, cannot be repaired in an automated fashion. Existing physical replicas might have already incorrectly frozen tuples because of different behavior than in master, which might only become apparent in the future once pg_multixact/ is truncated; it is recommended that all clones be rebuilt after upgrading. Following code analysis caused by bug report by J Smith in message CADFUPgc5bmtv-yg9znxV-vcfkb+JPRqs7m2OesQXaM_4Z1JpdQ@mail.gmail.com and privately by F-Secure. Backpatch to 9.3, where freezing of MultiXactIds was introduced. Analysis and patch by Andres Freund, with some tweaks by Álvaro. http://git.postgresql.org/pg/commitdiff/2393c7d102368717283d7121a6ea8164e902b011 Robert Haas pushed: - Refine our definition of what constitutes a system relation. Although user-defined relations can't be directly created in pg_catalog, it's possible for them to end up there, because you can create them in some other schema and then use ALTER TABLE .. SET SCHEMA to move them there. Previously, such relations couldn't afterwards be manipulated, because IsSystemRelation()/IsSystemClass() rejected all attempts to modify objects in the pg_catalog schema, regardless of their origin. With this patch, they now reject only those objects in pg_catalog which were created at initdb-time, allowing most operations on user-created tables in pg_catalog to proceed normally. This patch also adds new functions IsCatalogRelation() and IsCatalogClass(), which is similar to IsSystemRelation() and IsSystemClass() but with a slightly narrower definition: only TOAST tables of system catalogs are included, rather than *all* TOAST tables. This is currently used only for making decisions about when invalidation messages need to be sent, but upcoming logical decoding patches will find other uses for this information. Andres Freund, with some modifications by me. http://git.postgresql.org/pg/commitdiff/8e18d04d4daf34b8a557e2dc553a7754b255cd9a == Rejected Patches (for now) == No one was disappointed this week :-) == Pending Patches == Heikki Linnakangas sent in a patch to critique an optimization for fixed-length arrays in PL/pgsql. Kyotaro HORIGUCHI sent in new revisions of two ways to optimize certain UNION ALL queries. Rajeev Rastogi sent in three more revisions of a patch to ensure that COPY FROM ... STDIN generates a count tag. Stephen Frost and Dimitri Fontaine traded patches to implement EXTENSION templates. SAKAMOTO Masahiko sent in three more revisions of a patch to enable logging WAL when updating hint bits. Peter Eisentraut sent in another revision of a patch to add EVENT triggers to PL/Perl. Gurjeet Singh sent in a patch to remove "Nothing to be done for 'all'" messages. Heikki Linnakangas sent in another revision of a patch to implement INSERT...ON DUPLICATE KEY LOCK FOR UPDATE. Kyotaro HORIGUCHI and Etsuro Fujita traded patches to improve the path generation for certain indexed queries. Rajeev Rastogi and Amit Kapila traded revisions of a patch to split out pg_resetxlog output into pre- and post-sections. Haribabu Kommi sent in two more revisions of a patch to ensure that a heavily modified table not bloat in the case where autovacuum is having a hard time keeping up. Pavel Stehule sent in another revision of a patch to detect orphan locks. Gurjeet Singh sent in another revision of a patch to shave a few instructions from backend child startup. Marko Kreen sent in a patch to fix the regression tests for PL/Python3 per the addition of domains over arrays. Tom Dunstan sent in a patch to allow foreign keys to have WHERE clauses. Bruce Momjian sent in a patch to fix an issue in PostgreSQL 9.2 where during index-only scans, there were abnormal heap fetches after VACUUM FULL. Pavel Stehule sent in another revision of a patch to implement new border styles for psql. Pavel Stehule and Dean Rasheed traded patches to expand the DROP ... IF EXISTS construct to more database objects. Alvaro Herrera sent in a patch to avoid a useless palloc(). KaiGai Kohei sent in another flock of patches implementing and using custom plan nodes. Robert Haas sent in another revision of a patch to implement logical changeset logging. Gregory Stark sent in another revision of a WIP patch to optimize away some overflow checks. Heikki Linnakangas sent in two more revisions of a patch to implement GIN packed posting lists. Fabrízio de Royes Mello sent in another revision of a patch to implement time-delayed standbys. Bruce Momjian sent in a patch to fix an issue where pg_upgrades segfaults when given an invalid PGSERVICE value. Bruce Momjian sent in a patch to document some behavior around client certificates. Peter Geoghegan sent in two more revisions of a patch to add some features to pg_stat_statements.
pgsql-announce by date: