== PostgreSQL Weekly News - December 28 2014 == - Mailing list pgsql-announce

== PostgreSQL Weekly News - December 28 2014 ==

PGCon 2015 (June 16-20) call for papers is out.  Talk submission ends
January 19, 2014.  Details at
http://www.pgcon.org/2015/papers.php

== PostgreSQL Product News ==

Bucardo 5.3.0, a multiple source replication system for Postgres, released:
https://mail.endcrypt.com/pipermail/bucardo-announce/2014/000030.html

== PostgreSQL Jobs for December ==

http://archives.postgresql.org/pgsql-jobs/2014-12/threads.php

== PostgreSQL Local ==

FOSDEM PGDay (January 30, 2015) and the FOSDEM PostgreSQL
Dev Room (January 31-February 1, 2015)
http://fosdem2015.pgconf.eu/

Prague PostgreSQL Developer Day (P2D2) 2015 will be in Prague, Czech
Republic February 11-12, 2015.
http://www.p2d2.cz/

The Melbourne PostgreSQL meetup on February 18, 2015 will be hosting
Gabriele Bartolini on PostgreSQL 9.4 for devops.  Details below, and
R, SVP.
http://www.meetup.com/melpug/events/219082475/

pgDaySF 2015 will be held March 10, 2015 in Burlingame, California.
http://sfpostgres.org/pgday-sf-2015-call-for-speakers-and-sponsors/

The CfP is open for Nordic PostgreSQL Day 2015, which will be held
March 11, 2015 in  Copenhagen, Denmark.
http://2015.nordicpgday.org/cfp/

The CfP for PGConf US 2015 is open through December 17th, 2014
Notifications will go out on January 10, 2014.  The event takes place
March 25-27, 2015 in NYC.
http://nyc.pgconf.us/2015/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david@fetter.org, German language
to pwn@pgug.de, Italian language to pwn@itpug.org.  Spanish language
to pwn@arpug.com.ar.

== Applied Patches ==

Heikki Linnakangas pushed:

- Use a pairing heap for the priority queue in kNN-GiST searches.
  This performs slightly better, uses less memory, and needs slightly
  less code in GiST, than the Red-Black tree previously used.
  Reviewed by Peter Geoghegan
  http://git.postgresql.org/pg/commitdiff/e7032610f76e6c8345496ae7bbdf49d3c40df30f

- Turn much of the btree_gin macros into real functions.  This makes
  the functions much nicer to read and edit, and also makes debugging
  easier.
  http://git.postgresql.org/pg/commitdiff/7f0dccaed64a8ed6f5db8ad43e7612202fbeeeaf

- Move rbtree.c from src/backend/utils/misc to src/backend/lib.  We
  have other general-purpose data structures in src/backend/lib, so it
  seems like a better home for the red-black tree as well.
  http://git.postgresql.org/pg/commitdiff/955557ddccb4065831af80b0966cbd02937dfb72

Andrew Dunstan pushed:

- Further tidy up on json aggregate documentation
  http://git.postgresql.org/pg/commitdiff/2a3f2743f200dbcdd412a00ff69f30d3e34b662b

- Fix documentation of argument type of json_agg and jsonb_agg.
  json_agg was originally designed to aggregate records.  However, it
  soon became clear that it is useful for aggregating all kinds of
  values and that's what we have on 9.3 and 9.4, and in head for it
  and jsonb_agg.  The documentation suggested otherwise, so this fixes
  it.
  http://git.postgresql.org/pg/commitdiff/b2d145bff02e5f278649aca0e6fb6142ea2f298d

- Fix installcheck case for tap tests
  http://git.postgresql.org/pg/commitdiff/3f37b6c3166b2dc1956de8b0fb046ea6e7320d52

Peter Eisentraut pushed:

- Change local_preload_libraries to PGC_USERSET.  This allows it to be
  used with ALTER ROLE SET.  Although the old setting of PGC_BACKEND
  prevented changes after session start, after discussion it was more
  useful to allow ALTER ROLE SET instead and just document that
  changes during a session have no effect.  This is similar to how
  session_preload_libraries works already.  An alternative would be to
  change things to allow PGC_BACKEND and PGC_SU_BACKEND settings to be
  changed by ALTER ROLE SET.  But that might need further research
  (e.g., log_connections would probably not work).  based on patch by
  Kyotaro Horiguchi
  http://git.postgresql.org/pg/commitdiff/584e35d17c00b69d598522d3cf321fbd9b7dbd03

Álvaro Herrera pushed:

- get_object_address: separate domain constraints from table
  constraints.  Apart from enabling comments on domain constraints,
  this enables a future project to replicate object dropping to remote
  servers: with the current mechanism there's no way to distinguish
  between the two types of constraints, so there's no way to know what
  to drop.  Also added support for the domain constraint comments in
  psql's \dd and pg_dump.  Catalog version bumped due to the change in
  ObjectType enum.
  http://git.postgresql.org/pg/commitdiff/7eca575d1c28f6eee2bf4564f3d458d10c4a8f47

- Use a bitmask to represent role attributes.  The previous
  representation using a boolean column for each attribute would not
  scale as well as we want to add further attributes.  Extra
  auxilliary functions are added to go along with this change, to make
  up for the lost convenience of access of the old representation.
  Catalog version bumped due to change in catalogs and the new
  functions.  Author: Adam Brightwell, minor tweaks by Álvaro Reviewed
  by: Stephen Frost, Andres Freund, Álvaro Herrera
  http://git.postgresql.org/pg/commitdiff/1826987a46d079458007b7b6bbcbbd852353adbb

- Add SQL-callable pg_get_object_address.  This allows access to
  get_object_address from SQL, which is useful to obtain OID
  addressing information from data equivalent to that emitted by the
  parser.  This is necessary infrastructure of a project to let
  replication systems propagate object dropping events to remote
  servers, where the schema might be different than the server
  originating the DROP.  This patch also adds support for
  OBJECT_DEFAULT to get_object_address; that is, it is now possible to
  refer to a column's default value.  Catalog version bumped due to
  the new function.  Reviewed by Stephen Frost, Heikki Linnakangas,
  Robert Haas, Andres Freund, Abhijit Menon-Sen, Adam Brightwell.
  http://git.postgresql.org/pg/commitdiff/d7ee82e50f624221db76023c17137661fe69ec61

- Revert "Use a bitmask to represent role attributes".  This reverts
  commit 1826987a46d079458007b7b6bbcbbd852353adbb.  The overall design
  was deemed unacceptable, in discussion following the previous commit
  message; we might find some parts of it still salvageable, but I
  don't want to be on the hook for fixing it, so let's wait until we
  have a new patch.
  http://git.postgresql.org/pg/commitdiff/a609d96778c1714b9af916477b2c30891fbe578a

- Grab heavyweight tuple lock only before sleeping.  We were trying to
  acquire the lock even when we were subsequently not sleeping in some
  other transaction, which opens us up unnecessarily to deadlocks.  In
  particular, this is troublesome if an update tries to lock an
  updated version of a tuple and finds itself doing EvalPlanQual
  update chain walking; more than two sessions doing this concurrently
  will find themselves sleeping on each other because the HW tuple
  lock acquisition in heap_lock_tuple called from EvalPlanQualFetch
  races with the same tuple lock being acquired in heap_update -- one
  of these sessions sleeps on the other one to finish while holding
  the tuple lock, and the other one sleeps on the tuple lock.  Per
  trouble report from Andrew Sackville-West in
  http://www.postgresql.org/message-id/20140731233051.GN17765@andrew-ThinkPad-X230
  His scenario can be simplified down to a relatively simple
  isolationtester spec file which I don't include in this commit; the
  reason is that the current isolationtester is not able to deal with
  more than one blocked session concurrently and it blocks instead of
  raising the expected deadlock.  In the future, if we improve
  isolationtester, it would be good to include the spec file in the
  isolation schedule.  I posted it in
  http://www.postgresql.org/message-id/20141212205254.GC1768@alvh.no-ip.org
  Hat tip to Mark Kirkwood, who helped diagnose the trouble.
  http://git.postgresql.org/pg/commitdiff/0e5680f4737a9c6aa94aa9e77543e5de60411322

- Remove event trigger from object_address test.  It is causing
  trouble when run in parallel mode, because dropping the function
  other sessions are running concurrently causes them to fail due to
  inability to find the function.  Per buildfarm, as noted by Tom
  Lane.
  http://git.postgresql.org/pg/commitdiff/289121a452e3c255e4fd63dbb0252261605c2eaf

- Restrict name list len for domain constraints.  This avoids an
  ugly-looking "cache lookup failure" message.  Ugliness pointed out
  by Andres Freund.
  http://git.postgresql.org/pg/commitdiff/6630420fc9139c3a1ee051bc2f2ae8fa32e71c62

Tom Lane pushed:

- Remove failing collation case from object_address regression test.
  Per buildfarm, this test case does not yield consistent results.  I
  don't think it's useful enough to figure out a workaround, either.
  http://git.postgresql.org/pg/commitdiff/3e2275355954cabf89ba044fe8557ef347bf3fb1

- Add CST (China Standard Time) to our lists of timezone
  abbreviations.  For some reason this seems to have been missed when
  the lists in src/timezone/tznames/ were first constructed.  We can't
  put it in Default because of the conflict with US CST, but we should
  certainly list it among the alternative entries in Asia.txt.  (I
  checked for other oversights, but all the other abbreviations that
  are in current use according to the IANA files seem to be accounted
  for.)  Noted while responding to bug #12326.
  http://git.postgresql.org/pg/commitdiff/5b89473d870dc2a9fec0926c5afccf53042dbb0a

- Temporarily revert "Move pg_lzcompress.c to src/common." This
  reverts commit 60838df922345b26a616e49ac9fab808a35d1f85.  That
  change needs a bit more thought to be workable.  In view of the
  potentially machine-dependent stuff that went in today, we need all
  of the buildfarm to be testing those other changes.
  http://git.postgresql.org/pg/commitdiff/966115c305c297cf4a1ec5ebbd0a55be46ff33ad

- Remove duplicate assignment in new pg_get_object_address() function.
  Noted by Coverity.
  http://git.postgresql.org/pg/commitdiff/9a11df14498434bbb5e16dfa7aecbf71bf24d196

Bruce Momjian pushed:

- pgbench:  remove odd trailing period in init progress output
  http://git.postgresql.org/pg/commitdiff/83bcc70459c596ceb9a90e96f011d24b88d93a85

Fujii Masao pushed:

- Remove unused fields from ReindexStmt.  fe263d1 changed the REINDEX
  logic so that those fields are not used at all, but forgot to remove
  them.  Sawada Masahiko
  http://git.postgresql.org/pg/commitdiff/3b6ca123b5665044557154385a6be5d1df76c5e5

- Remove duplicate include of slot.h.  Back-patch to 9.4, where this
  problem was added.
  http://git.postgresql.org/pg/commitdiff/4a5593197b0ddec913fcd7758d61e782ab5c4d59

- Move pg_lzcompress.c to src/common.  Exposing compression and
  decompression APIs of pglz makes possible its use by extensions and
  contrib modules. pglz_decompress contained a call to elog to emit an
  error message in case of corrupted data. This function is changed to
  return a status code to let its callers return an error instead.
  This commit is required for upcoming WAL compression feature so that
  the WAL reader facility can decompress the WAL data by using
  pglz_decompress.  Michael Paquier
  http://git.postgresql.org/pg/commitdiff/60838df922345b26a616e49ac9fab808a35d1f85

Andres Freund pushed:

- Suppress MSVC warning in typeStringToTypeName function.  MSVC
  doesn't realize ereport(ERROR) doesn't return.  David Rowley
  http://git.postgresql.org/pg/commitdiff/cd5ebe1edd6d9c19caba406a90f626afec5d3f0a

- Convert the PGPROC->lwWaitLink list into a dlist instead of open
  coding it.  Besides being shorter and much easier to read it changes
  the logic in LWLockRelease() to release all shared lockers when
  waking up any.  This can yield some significant performance
  improvements - and the fairness isn't really much worse than before,
  as we always allowed new shared lockers to jump the queue.
  http://git.postgresql.org/pg/commitdiff/7882c3b0b95640e361f1533fe0f2d02e4e5d8610

- Add capability to suppress CONTEXT: messages to elog machinery.
  Hiding context messages usually is not a good idea - except for
  rather verbose debugging/development utensils like LOG_DEBUG. There
  the amount of repeated context messages just bloat the log without
  adding information.
  http://git.postgresql.org/pg/commitdiff/570bd2b3fd20d3f8896f5e6a8133d6aed385ac86

- Lockless StrategyGetBuffer clock sweep hot path.
  StrategyGetBuffer() has proven to be a bottleneck in a number of
  buffer acquisition heavy workloads. To some degree this has already
  been alleviated by 5d7962c6, but it still can be quite a heavy
  bottleneck.  The problem is that in unfortunate usage patterns a
  single StrategyGetBuffer() call will have to look at a large number
  of buffers - in turn making it likely that the process will be put
  to sleep while still holding the spinlock.  Replace most of the
  usage of the buffer_strategy_lock spinlock for the clock sweep by a
  atomic nextVictimBuffer variable. That variable, modulo NBuffers, is
  the current hand of the clock sweep. The buffer clock-sweep then
  only needs to acquire the spinlock after a wraparound. And even then
  only in the process that did the wrapping around. That alleviates
  nearly all the contention on the relevant spinlock, although
  significant contention on the cacheline can still exist.
  Reviewed-By: Robert Haas and Amit Kapila.  Discussion:
  20141010160020.GG6670@alap3.anarazel.de,
  20141027133218.GA2639@awork2.anarazel.de
  http://git.postgresql.org/pg/commitdiff/d72731a70450b5e7084991b9caa15cb58a2820df

- Blindly fix a dtrace probe in lwlock.c for a removed local variable.
  Per buildfarm member locust.
  http://git.postgresql.org/pg/commitdiff/740a4ec7f4adccd56eced452580ccec7157fc8d8

- Improve LWLock scalability.  The old LWLock implementation had the
  problem that concurrent lock acquisitions required exclusively
  acquiring a spinlock. Often that could lead to acquirers waiting
  behind the spinlock, even if the actual LWLock was free.  The new
  implementation doesn't acquire the spinlock when acquiring the lock
  itself. Instead the new atomic operations are used to atomically
  manipulate the state. Only the waitqueue, used solely in the slow
  path, is still protected by the spinlock. Check lwlock.c's header
  for an explanation about the used algorithm.  For some common
  workloads on larger machines this can yield significant performance
  improvements. Particularly in read mostly workloads.  Reviewed-By:
  Amit Kapila and Robert Haas.  Author: Andres Freund.  Discussion:
  20130926225545.GB26663@awork2.anarazel.de
  http://git.postgresql.org/pg/commitdiff/ab5194e6f617a9a9e7aadb3dd1cee948a42d0755

Noah Misch pushed:

- Have config_sspi_auth() permit IPv6 localhost connections.  Windows
  versions later than Windows Server 2003 map "localhost" to ::1.
  Account for that in the generated pg_hba.conf, fixing another
  oversight in commit f6dc6dd5ba54d52c0733aaafc50da2fbaeabb8b0.
  Back-patch to 9.0, like that commit.  David Rowley and Noah Misch
  http://git.postgresql.org/pg/commitdiff/8d9cb0bc483431780aef483c7c110ab39e465958

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Michael Paquier sent in another revision of a patch to clarify some
points about how numeric functions and operators work.

Etsuro Fujita sent in a patch to fix the fact that ExplainModifyTarget
does not always show the parent of the inheritance tree in
multi-target-table cases.

Heikki Linnakangas sent in another revision of a patch to advance
local xmin more aggressively using the new pairing heap code.

Robert Haas sent in another revision of a patch to implement parallel
mode and parallel contexts.

Petr (PJMODOS) Jelinek sent in another revision of a patch to
implement TABLESAMPLE.

Christoph Berg sent in a patch to correct the misapprehension that
pg_upgrade requires "postmaster," an entity not present in all live
versions of PostgreSQL, when "postgres" will do.

Michael Paquier sent in two more revisions of a patch to move contrib
binaries to bin.

Álvaro Herrera sent in a flock of five patches intended to complete
the feature which allows DROP to propagate via WAL.

Etsuro Fujita sent in two more revisions of a patch to allow foreign
tables to be part of table inheritance hierarchies.

Heikki Linnakangas and Teodor Sigaev traded patches to support ranges
with B-Tree GIN indexes.

Heikki Linnakangas sent in another revision of a patch to add a
compress method for SP-GiST.

Andres Freund sent in another revision of a patch to add the
capability to suppress CONTEXT: messages to elog machinery.

Andres Freund sent in another revision of a patch to convert the
PGPROC->lwWaitLink list into a dlist instead of open coding it, and
one to implement wait-free LW_SHARED LWLock acquisition.

Andres Freund sent in a patch to add a lockless StrategyGetBuffer
clock sweep hotpath.

Michael Paquier sent in a patch to install libpq.dll in bin directory
on Windows / Cygwin.

Andrew Dunstan sent in a patch to speed up the bin checks.

Teodor Sigaev sent in another revision of a patch to speed up
tidbitmap using a cache page.

Jim Nasby sent in a patch to add a hack to test out several hashing
strategies.

Andreas Karlsson sent in another revision of a patch to use 128-bit
integers for sum, avg and statistics aggregates.

Bruce Momjian sent in a patch to fix an issue where misaligned
BufferDescriptors is causing major performance problems on AMD.

Jeff Davis sent in another revision of a patch to implement
spill-to-disk for HashAggs.

Alexander Shulgin sent in another revision of a patch to turn
recovery.conf into GUCs.

Kaigai Kouhei sent in another revision of a patch to add a CTID scan
as an example of the custom scan capability.

David Rowley and Noah Misch traded patches to update Windows with the
changes that secured "make check."

Abhijit Menon-Sen sent in a patch to use CPUID at startup to determine
whether the SSE4.2 CRC instructions are available, and to use them
instead of the slice-by-8 implementation if they are.

Fujii Masao and Michael Paquier traded patches implementing
compression of full-page writes.

Abhijit Menon-Sen sent in a patch to implement a fast bloat
measurement tool.

Alexander Shulgin and Pavel Stehule traded patches to allow COPY to
handle wrong rows without rolling back entirely.

Alexey Vasiliev sent in a patch to  add recovery_timeout option to
control timeout of restore_command nonzero status code.

Peter Geoghegan sent in another revision of a patch to implement
INSERT ... ON CONFLICT {UPDATE | IGNORE}.

Ian Lawrence Barwick sent in a patch to fix psql's tab completion of
COMMENT ON not to repeat IS more than needed.

Amit Kapila and Dilip Kumar traded patches to enable vacuumdb to use
parallel cores.