== PostgreSQL Weekly News - March 23 2014 == - Mailing list pgsql-announce

== PostgreSQL Weekly News - March 23 2014 ==

PostgreSQL 9.3.4, 9.2.8, 9.1.13, 9.0.17, and 8.4.21 bug fix releases
are out.  Upgrade ASAP.
http://www.postgresql.org/about/news/1511/

The sixth PGDay Cubano be held on 13 and 14 October 2014 in Habana.
https://postgresql.uci.cu/?p=380

The CfPs for Char(13) and PGday UK, July 8 and 9, 2014,
respectively, are out and open until April 17, 2014.
speakers AT char14 DOT info, and speakers AT postgresqlusergroup DOT
org DOT uk, respectively.
http://www.char14.info

== PostgreSQL Product News ==

DataFiller version 2.0.0, a random data generator for PostgreSQL based
on the database schema + directives, released.
http://blog.coelho.net/database/2014/03/23/datafiller-2-0-0/

Npgsql2 2.1.0, a .NET connector for PostgreSQL, released.
https://www.nuget.org/packages/Npgsql/2.1.0
https://github.com/npgsql/Npgsql/releases/tag/v2.1.0

Repmgr 2.0, an HA and replication cluster management software for
PostgreSQL, released.
http://www.repmgr.org

== PostgreSQL Jobs for March ==

http://archives.postgresql.org/pgsql-jobs/2014-03/threads.php

== PostgreSQL Local ==

PGConf NYC 2014 will be held April 3-4, 2014 in New York, New York, USA.
http://nyc.pgconf.us/2014/

The Open Data Summit will be held Friday April 11, 2014 in Denver,
Colorado, USA.
http://www.opendatasummit.com

PGCon 2014, the world-wide developer conference for PostgreSQL, will
be in Ottawa, Ontario, Canada May 20-24, 2014.
http://www.pgcon.org/2014/

The sixth PGDay Cubano be held on 13 and 14 October 2014 in Habana.
https://postgresql.uci.cu/?p=380

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david@fetter.org, German language
to pwn@pgug.de, Italian language to pwn@itpug.org.  Spanish language
to pwn@arpug.com.ar.

== Applied Patches ==

Magnus Hagander pushed:

- Fix small typo in comment.  Michael Paquier
  http://git.postgresql.org/pg/commitdiff/02703ff2277791a7f1bccf61124bee830f2b5fcb

Fujii Masao pushed:

- Fix bug in clean shutdown of walsender that pg_receiving is
  connecting to.  On clean shutdown, walsender waits for all WAL to be
  replicated to a standby, and exits. It determined whether that
  replication had been completed by checking whether its sent location
  had been equal to a standby's flush location. Unfortunately this
  condition never becomes true when the standby such as pg_receivexlog
  which always returns an invalid flush location is connecting to
  walsender, and then walsender waits forever.  This commit changes
  walsender so that it just checks a standby's write location if a
  flush location is invalid.  Back-patch to 9.1 where enough
  infrastructure for this exists.
  http://git.postgresql.org/pg/commitdiff/5c6d9fc4b2b8b6688a482a4b4116d7642e36b9d9

- Fix typos in comments.  Thom Brown
  http://git.postgresql.org/pg/commitdiff/2bccced110025f48bf426a8a9d7f627ef3663fcd

- Fix help message and document in pg_receivexlog.  Add SLOTNAME
  placeholder to --slot option in help message and document.
  http://git.postgresql.org/pg/commitdiff/e726e59dc4845fff1e0e5849ef8d0bae958002fd

- Some minor improvements to logical decoding document.  Also improve
  help message in pg_recvlogical.
  http://git.postgresql.org/pg/commitdiff/fb1d92a9fa0226c435daf16d08e5d05d6986fc54

Heikki Linnakangas pushed:

- Fix thinko: have trueTriConsistentFn return GIN_TRUE.  While we're
  at it, also improve comments in ginlogic.c.
  http://git.postgresql.org/pg/commitdiff/d663d4399e767223e454302ea90d04f78b2f9d29

- Make the handling of interrupted B-tree page splits more robust.
  Splitting a page consists of two separate steps: splitting the child
  page, and inserting the downlink for the new right page to the
  parent.  Previously, we handled the case that you crash in between
  those steps with a cleanup routine after the WAL recovery had
  finished, which finished the incomplete split. However, that doesn't
  help if the page split is interrupted but the database doesn't
  crash, so that you don't perform WAL recovery. That could happen for
  example if you run out of disk space.  Remove the end-of-recovery
  cleanup step. Instead, when a page is split, the left page is marked
  with a new INCOMPLETE_SPLIT flag, and when the downlink is inserted
  to the parent, the flag is cleared again. If an insertion sees a
  page with the flag set, it knows that the split was interrupted for
  some reason, and inserts the missing downlink before proceeding.  I
  used the same approach to fix GIN and GiST split algorithms earlier.
  This was the last WAL cleanup routine, so we could get rid of that
  whole machinery now, but I'll leave that for a separate patch.
  Reviewed by Peter Geoghegan.
  http://git.postgresql.org/pg/commitdiff/40dae7ec537c5619fc93ad602c62f37be786d161

- Fix misc typos in comments.
  http://git.postgresql.org/pg/commitdiff/1d3b258cbe4aedfb49c92c28b9cbd7c18d277e04

- Fix compilation of pg_xlogdump, now that rm_safe_restartpoint is no
  more.  Oops. Pointed out by Andres Freund.
  http://git.postgresql.org/pg/commitdiff/033dc1c92cf018d396e983d425b821dda420cfff

- Remove rm_safe_restartpoint machinery.  It is no longer used, none
  of the resource managers have multi-record actions that would make
  it unsafe to perform a restartpoint.  Also don't allow rm_cleanup to
  write WAL records, it's also no longer required. Move the call to
  rm_cleanup routines to make it more symmetric with rm_startup.
  http://git.postgresql.org/pg/commitdiff/59a5ab3f426e74e3f901dc2cf533726bcea08ed2

- Replace the XLogInsert slots with regular LWLocks.  The special
  feature the XLogInsert slots had over regular LWLocks is the
  insertingAt value that was updated atomically with releasing
  backends waiting on it. Add new functions to the LWLock API to do
  that, and replace the slots with LWLocks. This reduces the amount of
  duplicated code.  (There's still some duplication, but at least it's
  all in lwlock.c now.) Reviewed by Andres Freund.
  http://git.postgresql.org/pg/commitdiff/68a2e52bbaf98f136a96b3a0d734ca52ca440a95

- Fix build with LWLOCK_STATS or dtrace.  Also fix the name of the
  dtrace probe for LWLockAcquireOrWait(). The function was renamed
  from LWLockWaitUntilFree to LWLockAqcuireOrWait, but the dtrace
  probe was neglected.  Pointed out by Andres Freund and the
  buildfarm.
  http://git.postgresql.org/pg/commitdiff/dea6ed2c980286e89caf4166ad329f506abbff29

- Fix thinkos in GinLogicValue enum.  It was incorrectly declared as
  global variable, not an enum type, and the comments for GIN_FALSE
  and GIN_TRUE were backwards.
  http://git.postgresql.org/pg/commitdiff/4c0e97c2d58f1cec9fc24237342962811de3cfee

Tom Lane pushed:

- During index build, check and elog (not just Assert) for broken HOT
  chain.  The recently-fixed bug in WAL replay could result in not
  finding a parent tuple for a heap-only tuple.  The existing code
  would either Assert or generate an invalid index entry, neither of
  which is desirable.  Throw a regular error instead.
  http://git.postgresql.org/pg/commitdiff/d70cf811f7dd26c07dbb78df4a51b667e7a3489b

- Release notes for 9.3.4, 9.2.8, 9.1.13, 9.0.17, 8.4.21.
  http://git.postgresql.org/pg/commitdiff/551fb5ac742eb7dbf92aa80743aa5a52b8a0189f

- Fix pg_dumpall option parsing: -i doesn't take an argument.  This
  used to work properly, but got fat-fingered in commit
  3dee636e0404885d07885d41c0d70e50c784f324.  Per bug #9620 from
  Nicolas Payart.
  http://git.postgresql.org/pg/commitdiff/19f2d6cdae2bfa97c2ce8a7f5ac453a91f40704a

- Fix relcache reference leak in refresh_by_match_merge().  One path
  through the loop over indexes forgot to do index_close().  Rather
  than adding a fourth call, restructure slightly so that there's only
  one.  In passing, get rid of an unnecessary syscache lookup: the
  pg_index struct for the index is already available from its relcache
  entry.  Per report from YAMAMOTO Takashi, though this is a bit
  different from his suggested patch.  This is new code in HEAD, so no
  need for back-patch.
  http://git.postgresql.org/pg/commitdiff/f7271c44278352516ec66b2de311952ce330b6d5

- Fix some remaining int64 vestiges in contrib/test_shm_mq.  Andres
  Freund and Tom Lane
  http://git.postgresql.org/pg/commitdiff/b6ec7c92ac7ab6223b3c45dc554efffd1953758f

- Fix memory leak during regular expression execution.  For a regex
  containing backrefs, pg_regexec() might fail to free all the
  sub-DFAs that were created during execution, resulting in a
  permanent (session lifespan) memory leak.  Problem was introduced by
  me in commit 587359479acbbdc95c8e37da40707e37097423f5.  Per report
  from Sandro Santilli; diagnosis by Greg Stark.
  http://git.postgresql.org/pg/commitdiff/ea8c7e9054abf23fa3de2f8e4414f60ac8a8b620

- Again fix initialization of auto-tuned effective_cache_size.  The
  previous method was overly complex and underly correct; in
  particular, by assigning the default value with PGC_S_OVERRIDE, it
  prevented later attempts to change the setting in postgresql.conf,
  as noted by Jeff Janes.  We should just assign the default value
  with source PGC_S_DYNAMIC_DEFAULT, which will have the desired
  priority relative to the boot_val as well as user-set values.  There
  is still a gap in this method: if there's an explicit assignment of
  effective_cache_size = -1 in the postgresql.conf file, and that
  assignment appears before shared_buffers is assigned, the code will
  substitute 4 times the bootstrap default for shared_buffers, and
  that value will then persist (since it will have source PGC_S_FILE).
  I don't see any very nice way to avoid that though, and it's not a
  case to be expected in practice.  The existing comments in
  guc-file.l look forward to a redesign of the DYNAMIC_DEFAULT
  mechanism; if that ever happens, we should consider this case as one
  of the things we'd like to improve.
  http://git.postgresql.org/pg/commitdiff/af930e606a3217db3909029c6c3f8d003ba70920

Robert Haas pushed:

- Make it easy to detach completely from shared memory.  The new
  function dsm_detach_all() can be used either by postmaster children
  that don't wish to take any risk of accidentally corrupting shared
  memory; or by forked children of regular backends with the same
  need.  This patch also updates the postmaster children that already
  do PGSharedMemoryDetach() to do dsm_detach_all() as well.  Per
  discussion with Tom Lane.
  http://git.postgresql.org/pg/commitdiff/79a4d24f31e09eb3c421deb34829eee0bf6acd67

- Improve shm_mq portability around MAXIMUM_ALIGNOF and sizeof(Size).
  Revise the original decision to expose a uint64-based interface and
  use Size everywhere possible.  Avoid assuming that MAXIMUM_ALIGNOF
  is 8, or making any assumption about the relationship between that
  value and sizeof(Size).  If MAXIMUM_ALIGNOF is bigger, we'll now
  insert padding after the length word; if it's smaller, we are now
  prepared to read and write the length word in chunks.  Per
  discussion with Tom Lane.
  http://git.postgresql.org/pg/commitdiff/3bd261ca18c67eafe18088e58fab511e3b965418

- Rewrite comment for shm_mq_receive_bytes.  The comment and the code
  diverged at some point before the initial commit of this feature,
  and I failed to notice.  Noted by Tom Lane.
  http://git.postgresql.org/pg/commitdiff/250f8a7bbe83a7dbc9618c0a506f7170ab610e4b

- test_shm_mq: Use Size rather than uint64.  Commit
  3bd261ca18c67eafe18088e58fab511e3b965418 updated the API but
  neglected to make the corresponding edits here.  Per Tom Lane and
  the buildfarm.
  http://git.postgresql.org/pg/commitdiff/c676ac0f3fc87911588ca8e5f92483d1671ba89f

- Add pg_recvlogical, a tool to receive data logical decoding data.
  This is fairly basic at the moment, but it's at least useful for
  testing and debugging, and possibly more.  Andres Freund
  http://git.postgresql.org/pg/commitdiff/8bdd12bbf093c846c9383c205e363d6ae35f11a6

- Fix uninitialized variable.  Report from Andres Freund, but not his
  fix.
  http://git.postgresql.org/pg/commitdiff/a3b30d4cfe5f3ae718fd2cc6aa05b7707cac88dd

- Logical decoding documentation corrections.  Thom Brown
  http://git.postgresql.org/pg/commitdiff/3ee4fcfc63452a1ab5586b2d4cb4b6908d1badff

- Documentation for logical decoding.  Craig Ringer, Andres Freund,
  Christian Kruse, with edits by me.
  http://git.postgresql.org/pg/commitdiff/49c0864d7ef5227faa24f903902db90e5c9d5d69

- test_shm_mq: Improve regression tests.  Per discussion with Tom
  Lane.
  http://git.postgresql.org/pg/commitdiff/d1bdab2fa39f9a29fc806e1f2b6b5428b88d7cc5

Alvaro Herrera pushed:

- Setup error context callback for transaction lock waits With this in
  place, a session blocking behind another one because of tuple locks
  will get a context line mentioning the relation name, tuple TID, and
  operation being done on tuple.  For example: LOG:  process 11367
  still waiting for ShareLock on transaction 717 after 1000.108 ms
  DETAIL:  Process holding the lock: 11366. Wait queue: 11367.
  CONTEXT:  while updating tuple (0,2) in relation "foo" STATEMENT:
  UPDATE foo SET value = 3; Most usefully, the new line is displayed
  by log entries due to log_lock_waits, although of course it will be
  printed by any other log message as well.  Author: Christian Kruse,
  some tweaks by Álvaro Herrera Reviewed-by: Amit Kapila, Andres
  Freund, Tom Lane, Robert Haas
  http://git.postgresql.org/pg/commitdiff/f88d4cfc9d417dac2ee41a8f5e593898e56fd2bd

Bruce Momjian pushed:

- libpq:  pass a memory allocation failure error up to
  PQconndefaults().  Previously user name memory allocation failures
  were ignored and the default user name set to NULL.
  http://git.postgresql.org/pg/commitdiff/a4c8f14364c27508233f8a31ac4b10a4c90235a9

- Properly check for readdir/closedir() failures.  Clear errno before
  calling readdir() and handle old MinGW errno bug while adding full
  test coverage for readdir/closedir failures.  Backpatch through 8.4.
  http://git.postgresql.org/pg/commitdiff/6f03927fce038096f53ca67eeab9adb24938f8a6

- Remove MinGW readdir/errno bug workaround fixed on 2003-10-10
  http://git.postgresql.org/pg/commitdiff/1494931d7375ccdc6afd34f135bc708f8954eecc

Noah Misch pushed:

- Address ccvalid/ccnoinherit in TupleDesc support functions.
  equalTupleDescs() neglected both of these ConstrCheck fields, and
  CreateTupleDescCopyConstr() neglected ccnoinherit.  At this time,
  the only known behavior defect resulting from these omissions is
  constraint exclusion disregarding a CHECK constraint validated by an
  ALTER TABLE VALIDATE CONSTRAINT statement issued earlier in the same
  transaction.  Back-patch to 9.2, where these fields were introduced.
  http://git.postgresql.org/pg/commitdiff/c31305de5f5a4880b0ba2f5983025ef0210a3b2a

- Offer triggers on foreign tables.  This covers all the SQL-standard
  trigger types supported for regular tables; it does not cover
  constraint triggers.  The approach for acquiring the old row mirrors
  that for view INSTEAD OF triggers.  For AFTER ROW triggers, we spool
  the foreign tuples to a tuplestore.  This changes the FDW API
  contract; when deciding which columns to populate in the slot
  returned from data modification callbacks, writable FDWs will need
  to check for AFTER ROW triggers in addition to checking for a
  RETURNING clause.  In support of the feature addition, refactor the
  TriggerFlags bits and the assembly of old tuples in ModifyTable.
  Ronan Dunklau, reviewed by KaiGai Kohei; some additional hacking by
  me.
  http://git.postgresql.org/pg/commitdiff/7cbe57c34dec4860243e6d0f81738cfbb6e5d069

- Improve comments about AfterTriggerBeginQuery() query level usage.
  http://git.postgresql.org/pg/commitdiff/6115480c543c0141011a99db78987ad13540be59

- Don't test xmin/xmax columns of a postgres_fdw foreign table.  Their
  values are unspecified and system-dependent.  Per buildfarm member
  kouprey.
  http://git.postgresql.org/pg/commitdiff/b2b2491b06074e68fc7c96148cb0fdf0c8eb0469

Andrew Dunstan pushed:

- Introduce jsonb, a structured format for storing json.  The new
  format accepts exactly the same data as the json type.  However, it
  is stored in a format that does not require reparsing the orgiginal
  text in order to process it, making it much more suitable for
  indexing and other operations.  Insignificant whitespace is
  discarded, and the order of object keys is not preserved. Neither
  are duplicate object keys kept - the later value for a given key is
  the only one stored.  The new type has all the functions and
  operators that the json type has, with the exception of the json
  generation functions (to_json, json_agg etc.) and with identical
  semantics. In addition, there are operator classes for hash and
  btree indexing, and two classes for GIN indexing, that have no
  equivalent in the json type.  This feature grew out of previous work
  by Oleg Bartunov and Teodor Sigaev, which was intended to provide
  similar facilities to a nested hstore type, but which in the end
  proved to have some significant compatibility issues.  Authors: Oleg
  Bartunov,  Teodor Sigaev, Peter Geoghegan and Andrew Dunstan.
  Review: Andres Freund
  http://git.postgresql.org/pg/commitdiff/d9134d0a355cfa447adc80db4505d5931084278a

- Fix mis-spelling in jsonb docs.  Per Thom Brown.
  http://git.postgresql.org/pg/commitdiff/ca07cd59b24e00e428ed26716754244cec7f56b7

- Do jsonb regression test input in the conventional way.  This should
  make the buildfarm happier.
  http://git.postgresql.org/pg/commitdiff/ab22b149c60a10b842e3ec7fe3eb3b0b66c6611a

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Mitsumasa KONDO sent in another revision of a patch to allow using a
Gaussian distribution in pgbench.

Jürgen Strobel sent in a patch to add a command line option
--no-table-lock to pg_dump.

Fabrízio de Royes Mello sent in another revision of a patch to fix
some wrong behavior in ALTER ... RESET.

Petr (PJMODOS) Jelinek sent in two more revisions of a patch to add
plpgsql.warn_shadow.

Vaishnavi Prabakaran sent in another revision of a patch to add a
catalog view to pg_hba.conf.

Heikki Linnakangas sent in a patch to change the way locks are
acquired on B-trees.

Bruce Momjian sent in two revisions of a patch to fix a bug in
pg_archivecleanup.

Jing Wang sent in another revision of a patch to issue a log message
to suggest VACUUM FULL if a table is nearly empty.

Gurjeet Singh sent in a patch to send transaction commit/rollback
stats to the stats collector unconditionally.

Kyotaro HORIGUCHI sent in another revision of a patch to fix an
infelicity in some situations in archive recovery.

Petr (PJMODOS) Jelinek sent in two more revisions of a patch to add
plpgsql.extra_warnings and plpgsql.extra_errors.

Fujii Masao sent in a patch to fix an issue where effective_cache_size
cannot be changed by a reload (HUP) of the backend.

Vik Fearing sent in a patch to fix some typos in the patch to reduce
lock strengths needed for ALTER TABLE.

Etsuro Fujita sent in another revision of a patch to allow foreign
tables in table inheritance hierarchies.

Dilip Kumar sent in another revision of a patch to add parallelism to
the vacuumdb program.

MauMau sent in another revision of a patch to fix an issue where
PostgreSQL fails to start on Windows if it crashes after tablespace
creation.

Andres Freund sent in a PoC patch not to require a NBuffer-sized
PrivateRefCount array of local buffer pins.

Emanuel Calvo sent in a patch to clarify the documentation of what
events rewrite RULEs can apply to.

Bruce Momjian sent in another revision of a patch to fix some psql
output for the Replica type displayed.

Noah Misch sent in two more revisions of a patch to warn about some
escalation attacks possible during "make check".