== PostgreSQL Weekly News - January 25 2015 == - Mailing list pgsql-announce

== PostgreSQL Weekly News - January 25 2015 ==

Early Bird registration is available for PGConf.US.
http://pgconfus2015.eventbrite.com/?aff=pgann1

== PostgreSQL Product News ==

POWA 1.2.1, a PostgreSQL workload analyzer, released.
http://dalibo.github.io/powa/

Pyrseas 0.7.2, a toolkit for PostgreSQL version control, released.
https://github.com/pyrseas/Pyrseas

Slony-I 2.2.4, a trigger-based replication system for PostgreSQL, released.
http://www.slony.info/

== PostgreSQL Jobs for January ==

http://archives.postgresql.org/pgsql-jobs/2015-01/threads.php

== PostgreSQL Local ==

FOSDEM PGDay (January 30, 2015) and the FOSDEM PostgreSQL
Dev Room (January 31-February 1, 2015)
http://fosdem2015.pgconf.eu/

Prague PostgreSQL Developer Day (P2D2) 2015 will be in Prague, Czech
Republic February 11-12, 2015.
http://www.p2d2.cz/

The Melbourne PostgreSQL meetup on February 18, 2015 will be hosting
Gabriele Bartolini on PostgreSQL 9.4 for devops.  Details below, and
R, SVP.
http://www.meetup.com/melpug/events/219082475/

pgDaySF 2015 will be held March 10, 2015 in Burlingame, California.
http://sfpostgres.org/pgday-sf-2015-call-for-speakers-and-sponsors/

Registration for Nordic PostgreSQL Day, March 11, 2015 in  Copenhagen,
Denmark, is open.
http://2015.nordicpgday.org/registration/

PGConf US 2015 takes place March 25-27, 2015 in NYC.
http://nyc.pgconf.us/2015/

PGCon 2015 is June 16-20 in Ottawa, Canada.
http://www.pgcon.org/2015/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david@fetter.org, German language
to pwn@pgug.de, Italian language to pwn@itpug.org.  Spanish language
to pwn@arpug.com.ar.

== Applied Patches ==

Robert Haas pushed:

- BRIN typo fix.  Amit Langote
  http://git.postgresql.org/pg/commitdiff/9d54b93239040dab49111b6d7e9d6bfe9c71f419

- Typo fix.  Etsuro Fujita
  http://git.postgresql.org/pg/commitdiff/1605291b6c14be92915948d17f5509191632c97f

- Use abbreviated keys for faster sorting of text datums.  This commit
  extends the SortSupport infrastructure to allow operator classes the
  option to provide abbreviated representations of Datums; in the case
  of text, we abbreviate by taking the first few characters of the
  strxfrm() blob.  If the abbreviated comparison is insufficent to
  resolve the comparison, we fall back on the normal comparator.  This
  can be much faster than the old way of doing sorting if the first
  few bytes of the string are usually sufficient to resolve the
  comparison.  There is the potential for a performance regression if
  all of the strings to be sorted are identical for the first 8+
  characters and differ only in later positions; therefore, the
  SortSupport machinery now provides an infrastructure to abort the
  use of abbreviation if it appears that abbreviation is producing
  comparatively few distinct keys.  HyperLogLog, a streaming
  cardinality estimator, is included in this commit and used to make
  that determination for text.  Peter Geoghegan, reviewed by me.
  http://git.postgresql.org/pg/commitdiff/4ea51cdfe85ceef8afabceb03c446574daa0ac23

- Add strxfrm_l to list of functions where Windows adds an underscore.
  Per buildfarm failure on bowerbird after last night's commit
  4ea51cdfe85ceef8afabceb03c446574daa0ac23.  Peter Geoghegan
  http://git.postgresql.org/pg/commitdiff/f32a1fa462c88676105445f26885d7a0656b6806

- Disable abbreviated keys on Windows.  Most of the Windows buildfarm
  members (bowerbird, hamerkop, currawong, jacana, brolga) are unhappy
  with yesterday's abbreviated keys patch, although there are some
  (narwhal, frogmouth) that seem OK with it.  Since there's no obvious
  pattern to explain why some are working and others are failing, just
  disable this across-the-board on Windows for now.  This is a bit
  unfortunate since the optimization will be a big win in some cases,
  but we can't leave the buildfarm broken.
  http://git.postgresql.org/pg/commitdiff/1be4eb1b2d436d1375899c74e4c74486890d8777

- Heavily refactor btsortsupport_worker.  Prior to commit
  4ea51cdfe85ceef8afabceb03c446574daa0ac23, this function only had one
  job, which was to decide whether we could avoid trampolining through
  the fmgr layer when performing sort comparisons.  As of that commit,
  it has a second job, which is to decide whether we can use
  abbreviated keys.  Unfortunately, those two tasks are somewhat
  intertwined in the existing coding, which is likely why neither
  Peter Geoghegan nor I noticed prior to commit that this calls
  pg_newlocale_from_collation() in cases where it didn't previously.
  The buildfarm noticed, though.  To fix, rewrite the logic so that
  the decision as to which comparator to use is more cleanly separated
  from the decision about abbreviation.
  http://git.postgresql.org/pg/commitdiff/b529b65d1bf8537ca7fa024760a9782d7c8b66e5

- More fixes for abbreviated keys infrastructure.  First, when
  LC_COLLATE = C, bttext_abbrev_convert should use memcpy() rather
  than strxfrm() to construct the abbreviated key, because the
  authoritative comparator uses memcpy().  If we do anything else
  here, we might get inconsistent answers, and the buildfarm says this
  risk is not theoretical.  It should be faster this way, too.
  Second, while I'm looking at bttext_abbrev_convert, convert a
  needless use of goto into the loop it's trying to implement into an
  actual loop.  Both of the above problems date to the original commit
  of abbreviated keys, commit
  4ea51cdfe85ceef8afabceb03c446574daa0ac23.  Third, fix a bogus
  assignment to tss->locale before tss is set up.  That's a new goof
  in commit b529b65d1bf8537ca7fa024760a9782d7c8b66e5.
  http://git.postgresql.org/pg/commitdiff/b181a91981203f6ec9403115a2917bd3f9473707

- Repair brain fade in commit b181a91981203f6ec9403115a2917bd3f9473707.
  The split between which things need to happen in the C-locale case
  and which needed to happen in the locale-aware case was a few bricks
  short of a load.  Try to fix that.
  http://git.postgresql.org/pg/commitdiff/d060e07fa919e0eb681e2fa2cfbe63d6c40eb2cf

- Add an explicit cast to Size to hyperloglog.c.  MSVC generates a
  warning here; we hope this will make it happy.  Report by Michael
  Paquier.  Patch by David Rowley.
  http://git.postgresql.org/pg/commitdiff/6a3c6ba0ba2bf86254d43b40569cdf3012cf12aa

- Don't use abbreviated keys for the final merge pass.  When we write
  tuples out to disk and read them back in, the abbreviated keys
  become non-abbreviated, because the readtup routines don't know
  anything about abbreviation.  But without this fix, the rest of the
  code still thinks the abbreviation-aware compartor should be used,
  so chaos ensues.  Report by Andrew Gierth; patch by Peter Geoghegan.
  http://git.postgresql.org/pg/commitdiff/5cefbf5a6c4466ac6b1cc2a4316b4eba9108c802

- Fix typos, update README.  Peter Geoghegan
  http://git.postgresql.org/pg/commitdiff/d1747571b63444615730ca17474e9e89974c48ad

Álvaro Herrera pushed:

- doc: Fix typos in make_timestamp{,tz} examples.  Pointed out by Alan
  Mogi (bug #12571)
  http://git.postgresql.org/pg/commitdiff/412f604adbf025618f1338ea6fed0330bc6a610d

- reinit.h: Fix typo in identification comment.  Author: Sawada
  Masahiko
  http://git.postgresql.org/pg/commitdiff/813ffc0ef9f3c19dc01de19b1a718f2dda495f4f

- Tweak BRIN minmax operator class. In the union support proc, we were
  not checking the hasnulls flag of value A early enough, so it could
  be skipped if the "allnulls" flag in value B is set.  Also, a check
  on the allnulls flag of value "B" was redundant, so remove it.  Also
  change inet_minmax_ops to not be the default opclass for type inet,
  as a future inclusion operator class would be more useful and it's
  pretty difficult to change default opclass for a datatype later on.
  (There is no catversion bump for this catalog change; this shouldn't
  be a problem.) Extracted from a larger patch to add an "inclusion"
  operator class.  Author: Emre Hasegeli
  http://git.postgresql.org/pg/commitdiff/972bf7d6f13005dfe89ae3f8a3b937a4a0580c85

- vacuumdb: enable parallel mode.  This mode allows vacuumdb to open
  several server connections to vacuum or analyze several tables
  simultaneously.  Author: Dilip Kumar.  Some reworking by Álvaro
  Herrera Reviewed by: Jeff Janes, Amit Kapila, Magnus Hagander,
  Andres Freund
  http://git.postgresql.org/pg/commitdiff/a17923204736d8842eade3517d6a8ee81290fca4

- Fix assignment operator thinko.  Pointed out by Michael Paquier
  http://git.postgresql.org/pg/commitdiff/f2789ab84ea0f731d97006c3ea91c656415c0fe5

Andres Freund pushed:

- Fix various shortcomings of the new PrivateRefCount infrastructure.
  As noted by Tom Lane the improvements in 4b4b680c3d6 had the problem
  that in some situations we searched, entered and modified entries in
  the private refcount hash while holding a spinlock. I had tried to
  keep the logic entirely local to PinBuffer_Locked(), but that's not
  really possible given it's called with a spinlock held...  Besides
  being disadvantageous from a performance point of view, this also
  has problems with error handling safety. If we failed inserting an
  entry into the hashtable due to an out of memory error, we'd error
  out with a held spinlock. Not good.  Change the way private
  refcounts are manipulated: Before a buffer can be tracked an entry
  has to be reserved using ReservePrivateRefCountEntry(); then, if a
  entry is not found using GetPrivateRefCountEntry(), it can be
  entered with NewPrivateRefCountEntry().  Also take advantage of the
  fact that PinBuffer_Locked() currently is never called for buffers
  that already have been pinned by the current backend and don't
  search the private refcount entries for preexisting local pins. That
  results in a small, but measurable, performance improvement.
  Additionally make ReleaseBuffer() always call UnpinBuffer() for
  shared buffers. That avoids duplicating work in an eventual
  UnpinBuffer() call that already has been done in ReleaseBuffer() and
  also saves some code.  Per discussion with Tom Lane.  Discussion:
  15028.1418772313@sss.pgh.pa.us
  http://git.postgresql.org/pg/commitdiff/2d115e47c861878669ba0814b3d97a4e4c347e8b

Tom Lane pushed:

- Adjust "pgstat wait timeout" message to be a translatable LOG
  message.  Per discussion, change the log level of this message to be
  LOG not WARNING.  The main point of this change is to avoid causing
  buildfarm run failures when the stats collector is exceptionally
  slow to respond, which it not infrequently is on some of the
  smaller/slower buildfarm members.  This change does lose notice to
  an interactive user when his stats query is looking at out-of-date
  stats, but the majority opinion (not necessarily that of yours
  truly) is that WARNING messages would probably not get noticed
  anyway on heavily loaded production systems.  A LOG message at least
  ensures that the problem is recorded somewhere where bulk auditing
  for the issue is possible.  Also, instead of an untranslated "pgstat
  wait timeout" message, provide a translatable and hopefully more
  understandable message "using stale statistics instead of current
  ones because stats collector is not responding".  The original text
  was written hastily under the assumption that it would never really
  happen in practice, which we now know to be unduly optimistic.
  Back-patch to all active branches, since we've seen the buildfarm
  issue in all branches.
  http://git.postgresql.org/pg/commitdiff/75b48e1fff8a4dedd3ddd7b76f6360b5cc9bb741

- In pg_regress, remove the temporary installation upon successful
  exit.  This results in a very substantial reduction in disk space
  usage during "make check-world", since that sequence involves
  creation of numerous temporary installations.  It should also help a
  bit in the buildfarm, even though the buildfarm script doesn't
  create as many temp installations, because the current script misses
  deleting some of them; and anyway it seems better to do this once in
  one place rather than expecting that script to get it right every
  time.  In 9.4 and HEAD, also undo the unwise choice in commit
  b1aebbb6a86e96d7 to report strerror(errno) after a rmtree() failure.
  rmtree has already reported that, possibly for multiple failures
  with distinct errnos; and what's more, by the time it returns there
  is no good reason to assume that errno still reflects the last
  reportable error.  So reporting errno here is at best redundant and
  at worst badly misleading.  Back-patch to all supported branches, so
  that future revisions of the buildfarm script can rely on this
  behavior.
  http://git.postgresql.org/pg/commitdiff/aa719391d5537d30068109e1f96991d0e2c7dfc5

- Prevent duplicate escape-string warnings when using
  pg_stat_statements.  contrib/pg_stat_statements will sometimes run
  the core lexer a second time on submitted statements.  Formerly, if
  you had standard_conforming_strings turned off, this led to
  sometimes getting two copies of any warnings enabled by
  escape_string_warning.  While this is probably no longer a big deal
  in the field, it's a pain for regression testing.  To fix, change
  the lexer so it doesn't consult the escape_string_warning GUC
  variable directly, but looks at a copy in the core_yy_extra_type
  state struct.  Then, pg_stat_statements can change that copy to
  disable warnings while it's redoing the lexing.  It seemed like a
  good idea to make this happen for all three of the GUCs consulted by
  the lexer, not just escape_string_warning.  There's not an immediate
  use-case for callers to adjust the other two AFAIK, but making it
  possible is easy enough and seems like good future-proofing.
  Arguably this is a bug fix, but there doesn't seem to be enough
  interest to justify a back-patch.  We'd not be able to back-patch
  exactly as-is anyway, for fear of breaking ABI compatibility of the
  struct.  (We could perhaps back-patch the addition of only
  escape_string_warning by adding it at the end of the struct, where
  there's currently alignment padding space.)
  http://git.postgresql.org/pg/commitdiff/eb213acfe2a51ab3dc6d9bbe02d5d8a903366779

- Remove no-longer-referenced src/port/gethostname.c.  This file
  hasn't been part of any build since 2005, and even before that
  wasn't used unless you configured --with-krb4 (and had a machine
  without gethostname(2), obviously).  What's more, we haven't
  actually called gethostname anywhere since then, either (except in
  thread_test.c, whose testing of this function is probably
  pointless).  So we don't need it.
  http://git.postgresql.org/pg/commitdiff/9222cd84b0f227287f65df395d52dc7973a62d29

- Replace a bunch more uses of strncpy() with safer coding.  strncpy()
  has a well-deserved reputation for being unsafe, so make an effort
  to get rid of nearly all occurrences in HEAD.  A large fraction of
  the remaining uses were passing length less than or equal to the
  known strlen() of the source, in which case no null-padding can
  occur and the behavior is equivalent to memcpy(), though doubtless
  slower and certainly harder to reason about.  So just use memcpy()
  in these cases.  In other cases, use either StrNCpy() or strlcpy()
  as appropriate (depending on whether padding to the full length of
  the destination buffer seems useful).  I left a few strncpy() calls
  alone in the src/timezone/ code, to keep it in sync with upstream
  (the IANA tzcode distribution).  There are also a few such calls in
  ecpg that could possibly do with more analysis.  AFAICT, none of
  these changes are more than cosmetic, except for the four
  occurrences in fe-secure-openssl.c, which are in fact buggy: an
  overlength source leads to a non-null-terminated destination buffer
  and ensuing misbehavior.  These don't seem like security issues,
  first because no stack clobber is possible and second because if
  your values of sslcert etc are coming from untrusted sources then
  you've got problems way worse than this.  Still, it's undesirable to
  have unpredictable behavior for overlength inputs, so back-patch
  those four changes to all active branches.
  http://git.postgresql.org/pg/commitdiff/586dd5d6a5d59e406bc8032bb52625ffb904311c

- Fix unsafe coding in ReorderBufferCommit().  "iterstate" must be
  marked volatile since it's changed inside the PG_TRY block and then
  used in the PG_CATCH stanza.  Noted by Mark Wilding of Salesforce.
  (We really need to see if we can't get the C compiler to warn about
  this.) Also, reset iterstate to NULL after the mainline
  ReorderBufferIterTXNFinish call, to ensure the PG_CATCH block
  doesn't try to do that a second time.
  http://git.postgresql.org/pg/commitdiff/f8a4dd2e141a12e349882edecc683504acb82ec8

- Clean up some mess in row-security patches.  Fix unsafe coding
  around PG_TRY in RelationBuildRowSecurity: can't change a variable
  inside PG_TRY and then use it in PG_CATCH without marking it
  "volatile".  In this case though it seems saner to avoid that by
  doing a single assignment before entering the TRY block.  I started
  out just intending to fix that, but the more I looked at the
  row-security code the more distressed I got.  This patch also fixes
  incorrect construction of the RowSecurityPolicy cache entries (there
  was not sufficient care taken to copy pass-by-ref data into the
  cache memory context) and a whole bunch of sloppiness around the
  definition and use of pg_policy.polcmd.  You can't use nulls in that
  column because initdb will mark it NOT NULL --- and I see no
  particular reason why a null entry would be a good idea anyway, so
  changing initdb's behavior is not the right answer.  The internal
  value of '\0' wouldn't be suitable in a "char" column either, so
  after a bit of thought I settled on using '*' to represent ALL.
  Chasing those changes down also revealed that somebody wasn't paying
  attention to what the underlying values of ACL_UPDATE_CHR etc really
  were, and there was a great deal of lackadaiscalness in the
  catalogs.sgml documentation for pg_policy and pg_policies too.  This
  doesn't pretend to be a complete code review for the row-security
  stuff, it just fixes the things that were in my face while dealing
  with the bugs in RelationBuildRowSecurity.
  http://git.postgresql.org/pg/commitdiff/fd496129d160950ed681c1150ea8f627b292c511

Bruce Momjian pushed:

- tools/ccsym:  update for modern versions of gcc.  This dumps the
  predefined preprocessor macros
  http://git.postgresql.org/pg/commitdiff/f259e71dbe98f653182f0c710143487a0188fbc5

- djust ACL owners for REASSIGN and ALTER OWNER TO.  When REASSIGN and
  ALTER OWNER TO are used, both the object owner and ACL list should
  be changed from the old owner to the new owner. This patch fixes
  types, foreign data wrappers, and foreign servers to change their
  ACL list properly;  they already changed owners properly.  BACKWARD
  INCOMPATIBILITY?  Report by Alexey Bashtanov
  http://git.postgresql.org/pg/commitdiff/59367fdf97cc1875b053ebf87cd1e071dc7f3640

- docs:  update libpq's PQputCopyData and PQputCopyEnd.  Clarify the
  meaning of libpq return values for PQputCopyData and PQputCopyEnd,
  particularly in non-blocking mode.  Report by Robert Haas
  http://git.postgresql.org/pg/commitdiff/b04d69161354f7cfc3d8153b67145dbebc4a4698

Peter Eisentraut pushed:

- Fix whitespace
  http://git.postgresql.org/pg/commitdiff/f5f2c2de163352849879fd40e9793a0195fd6594

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Etsuro Fujita sent in another revision of a patch to fix some odd
behavior in EvalQualPlan in the PostgreSQL FDW.

Pavel Stehule sent in another revision of a patch to disallow the
deprecated => operator.

Michael Paquier sent in a patch to install shared libraries in bin and
lib with MSVC.

Dean Rasheed sent in two revisions of a patch to make RLS checks run
before attempting to insert/update any data rather than afterwards.

Michael Paquier sent in two more revisions of a patch to add
recovery_timeout option to control timeout of restore_command nonzero
status code.

Michael Paquier sent in a patch to remove some useless checks for NULL
pointers.

Stephen Frost sent in four more revisions of a patch to fix a
column-privilege leak in error-message paths.

Gilles Darold sent in another revision of a patch to fix a bug in
pg_dump which gets table dependencies in extensions wrong.

Andrew (RhodiumToad) Gierth sent in another revision of a patch to
implement GROUPING SETS.

Kyotaro HORIGUCHI sent in two more revisions of a patch to implement
asynchronous execution of postgres_fdw.

Peter Geoghegan sent in a patch to fix the spill-to-disk behavior for
abbreviated keys.

Amit Kapila sent in two more revisions of a patch to implement
parallel sequential scan.

Pavel Stehule sent in two more revisions of a patch to implement
array_position().

Michael Paquier sent in a patch to clean up the tablefunc contrib
extension.

Kyotaro HORIGUCHI sent in a PoC patch to implement limiting FETCH in
CURSORs by the number of bytes.

Robert Haas sent in a patch to fix a collate failure reported by Kevin
Grittner.

Pavel Stehule sent in another revision of a patch to implement assert
in PL/pgsql.

Andreas Karlsson sent in two more revisions of a patch to reduce the
lock strength of trigger and foreign key DDL.

Petr (PJMODOS) Jelinek sent in another revision of a patch to
implemetn TABLESAMPLE.

Petr (PJMODOS) Jelinek sent in another revision of a patch to
implement a sequence access method.

Andreas Karlsson sent in another revision of a patch to use 128-bit
integers for sum, avg and statistics aggregates.

Peter Geoghegan sent in a patch to fix some minor issues with code
comments related to abbreviated keys.

Bruce Momjian sent in two revisions of a patch to recommend using
wal_log_hints to make hint bits on the standby match the master for
pg_upgrade in the rsync case.

Álvaro Herrera sent in another revision of a patch to track TRUNCATE
via pgstat.

Tomas Vondra sent in another revision of a patch to implement
multivariate statistics.

Pavel Stehule sent in another revision of a patch to implement
row_to_array.