== PostgreSQL Weekly News - November 15 2015 == - Mailing list pgsql-announce

== PostgreSQL Weekly News - November 15 2015 ==

PostgreSQL 9.5 Beta 2 released.  Test!
http://www.postgresql.org/about/news/1625/

== PostgreSQL Product News ==

pg_wintools, a PostgreSQL extension for interfacing with Windows OS,
released.
https://github.com/jmguazzo/pg_wintools

== PostgreSQL Jobs for November ==

http://archives.postgresql.org/pgsql-jobs/2015-11/

== PostgreSQL Local ==

PGConf Silicon Valley 2015 is November 17-18 at the South San
Francisco Convention Center.
http://www.pgconfsv.com

PGBR2015 (The Brazilian PostgreSQL Conference) will take place in Porto
Alegre, Rio Grande do Sul, on November 18, 19 and 20.  Registration is
open.
http://pgbr.postgresql.org.br/2015/en/

Postgres Conference China 2015 will be on November 20-22, 2015 in
Beijing.
http://postgres2015.eventdove.com/   (Chinese)
http://postgrescluster2015.eventdove.com/ (English)

PGConf.DE will be held on November 26-27, 2015 in Hamburg, Germany, at
the Lindner Hotel am Michel.
http://2015.pgconf.de/

PostgreSQL@SCaLE is a two day, two track event which takes place on Jan.
21-22, 2016, at Pasadena Convention Center, as part of SCaLE 14X.
https://www.socallinuxexpo.org/scale/14x/cfp

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david@fetter.org, German language
to pwn@pgug.de, Italian language to pwn@itpug.org.  Spanish language
to pwn@arpug.com.ar.

== Applied Patches ==

Andres Freund pushed:

- Add paragraph about ON CONFLICT interaction with partitioning.
  Author: Peter Geoghegan and Andres Freund Discussion:
  CAM3SWZScpWzQ-7EJC77vwqzZ1GO8GNmURQ1QqDQ3wRn7AbW1Cg@mail.gmail.com,
  CAHGQGwFUCWwSU7dtc2aRdRk73ztyr_jY5cPOyts+K8xKJ92X4Q@mail.gmail.com
  Backpatch: 9.5, where UPSERT was introduced
  http://git.postgresql.org/pg/commitdiff/c31f1dc55992bc8b2c35c0ff99795547d4df323a

Robert Haas pushed:

- Add a dummy return statement to TupleQueueRemap.  This is
  unreachable for multiple reasons, but per Amit Kapila the Windows
  compiler he is using still thinks we can get there.
  http://git.postgresql.org/pg/commitdiff/89ff5c7f7512a919e76407b4f8269e5b0b39dd2b

- Fix rebasing mistake in nodeGather.c The patches committed as
  6e71dd7ce9766582da453f493bc371d64977282f and
  3a1f8611f2582df0a16bcd35caed2e1526387643 were developed in parallel
  but dependent on each other in a way that I failed to notice.  This
  patch to fix the problem was prepared by Amit Kapila.
  http://git.postgresql.org/pg/commitdiff/bf3d0156311f0d8fa7705d8de0d07c4e0298051a

- Comment update.  Adjust to account for
  5fc4c26db5120bd90348b6ee3101fcddfdf54800.  Etsuro Fujita
  http://git.postgresql.org/pg/commitdiff/5c90a2ffddd2504a7441e80a22ba0fe8d484291a

- Add outfuncs.c support for GatherPath.  I dunno how commit
  3bd909b220930f21d6e15833a17947be749e7fde missed this, but it
  evidently did.
  http://git.postgresql.org/pg/commitdiff/f764ecd81b2a8a1e9000d43a73ca5eec8e8008bc

- Make sequential scans parallel-aware.  In addition, this path fills
  in a number of missing bits and pieces in the parallel
  infrastructure.  Paths and plans now have a parallel_aware flag
  indicating whether whatever parallel-aware logic they have should be
  engaged.  It is believed that we will need this flag for a number of
  path/plan types, not just sequential scans, which is why the flag is
  generic rather than part of the SeqScan structures specifically.
  Also, execParallel.c now gives parallel nodes a chance to initialize
  their PlanState nodes from the DSM during parallel worker startup.
  Amit Kapila, with a fair amount of adjustment by me.  Review of
  previous patch versions by Haribabu Kommi and others.
  http://git.postgresql.org/pg/commitdiff/f0661c4e8c44c0ec7acd4ea7c82e85b265447398

- Generate parallel sequential scan plans in simple cases.  Add a new
  flag, consider_parallel, to each RelOptInfo, indicating whether a
  plan for that relation could conceivably be run inside of a parallel
  worker.  Right now, we're pretty conservative: for example, it might
  be possible to defer applying a parallel-restricted qual in a
  worker, and later do it in the leader, but right now we just don't
  try to parallelize access to that relation.  That's probably the
  right decision in most cases, anyway.  Using the new flag, generate
  parallel sequential scan plans for plain baserels, meaning that we
  now have parallel sequential scan in PostgreSQL.  The logic here is
  pretty unsophisticated right now: the costing model probably isn't
  right in detail, and we can't push joins beneath Gather nodes, so
  the number of plans that can actually benefit from this is pretty
  limited right now.  Lots more work is needed.  Nevertheless, it
  seems time to enable this functionality so that all this code can
  actually be tested easily by users and developers.  Note that, if
  you wish to test this functionality, it will be necessary to set
  max_parallel_degree to a value greater than the default of 0.  Once
  a few more loose ends have been tidied up here, we might want to
  consider changing the default value of this GUC, but I'm leaving it
  alone for now.  Along the way, fix a bug in cost_gather: the
  previous coding thought that a Gather node's transfer overhead
  should be costed on the basis of the relation size rather than the
  number of tuples that actually need to be passed off to the leader.
  Patch by me, reviewed in earlier versions by Amit Kapila.
  http://git.postgresql.org/pg/commitdiff/80558c1f5aa109d08db0fbd76a6d370f900628a8

- Provide readfuncs support for custom scans.  Commit
  a0d9f6e434bb56f7e5441b7988f3982feead33b3 added this support for all
  other plan node types; this fills in the gap.  Since
  TextOutCustomScan complicates this and is pretty well useless,
  remove it.  KaiGai Kohei, with some modifications by me.
  http://git.postgresql.org/pg/commitdiff/a05dc4d7fd57d4ae084c1f0801973e5c1a1aa26e

- Make idle backends exit if the postmaster dies.  Letting backends
  continue to run if the postmaster has exited prevents PostgreSQL
  from being restarted, which in many environments is catastrophic.
  Worse, if some other backend crashes, we no longer have any
  protection against shared memory corruption.  So, arrange for them
  to exit instead.  We don't want to expend many cycles on this, but
  including postmaster death in the set of things that we wait for
  when a backend is idle seems cheap enough.  Rajeev Rastogi and
  Robert Haas
  http://git.postgresql.org/pg/commitdiff/ac1d7945f866b1928c2554c0f80fd52d7f977772

- Move each SLRU's lwlocks to a separate tranche.  This makes it
  significantly easier to identify these lwlocks in LWLOCK_STATS or
  Trace_lwlocks output.  It's also arguably better from a modularity
  standpoint, since lwlock.c no longer needs to know anything about
  the LWLock needs of the higher-level SLRU facility.  Ildus
  Kurbangaliev, reviewd by Álvaro Herrera and by me.
  http://git.postgresql.org/pg/commitdiff/fe702a7b3f9f2bc5bf6d173166d7d55226af82c8

- libpq: Notice errors a backend may have sent just before dying.  At
  least since the introduction of Hot Standby, the backend has
  sometimes sent fatal errors even when no client query was in
  progress, assuming that the client would receive it.  However,
  pqHandleSendFailure was not in sync with this assumption, and only
  tries to catch notices and notifies.  Add a parseInput call to the
  loop there to fix.  Andres Freund suggested the fix.  Comments are
  by me.  Reviewed by Michael Paquier.
  http://git.postgresql.org/pg/commitdiff/c3e7c24a1d60dc6ad56e2a0723399f1570c54224

- Remove accidentally-committed debugging code.  Amit Kapila
  http://git.postgresql.org/pg/commitdiff/179c97bf587df710f98aa3ce2d95f503b0757e15

Tom Lane pushed:

- Improve our workaround for 'TeX capacity exceeded' in building PDF
  files.  In commit a5ec86a7c787832d28d5e50400ec96a5190f2555 I wrote a
  quick hack that reduced the number of TeX string pool entries
  created while converting our documentation to PDF form.  That held
  the fort for awhile, but as of HEAD we're back up against the same
  limitation.  It turns out that the original coding of
  \FlowObjectSetup actually results in *three* string pool entries
  being generated for every "flow object" (that is, potential
  cross-reference target) in the documentation, and my previous hack
  only got rid of one of them.  With a little more care, we can reduce
  the string count to one per flow object plus one per
  actually-cross-referenced flow object (about 115000 + 5000 as of
  current HEAD); that should work until the documentation volume
  roughly doubles from where it is today.  As a not-incidental side
  benefit, this change also causes pdfjadetex to stop emitting
  unreferenced hyperlink anchors (bookmarks) into the PDF file.  It
  had been making one willy-nilly for every flow object; now it's just
  one per actually-cross-referenced object.  This results in close to
  a 2X savings in PDF file size.  We will still want to run the output
  through "jpdftweak" to get it to be compressed; but we no longer
  need removal of unreferenced bookmarks, so we might be able to find
  a quicker tool for that step.  Although the failure only affects
  HEAD and US-format output at the moment, 9.5 cannot be more than a
  few pages short of failing likewise, so it will inevitably fail
  after a few rounds of minor-version release notes.  I don't have a
  lot of faith that we'll never hit the limit in the older branches;
  and anyway it would be nice to get rid of jpdftweak across the
  board.  Therefore, back-patch to all supported branches.
  http://git.postgresql.org/pg/commitdiff/944b41fc00600b74f518005ac314cc222bf6abd5

- Add missing "static" qualifier.  Per buildfarm member pademelon.
  http://git.postgresql.org/pg/commitdiff/b05ae27e9a57a1c8f1f44720c23c4828106b7205

- Docs: fix misleading example.  Commit 8457d0beca731bf0 introduced an
  example which, while not incorrect, failed to exhibit the behavior
  it meant to describe, as a result of omitting an E'' prefix that
  needed to be there.  Noticed and fixed by Peter Geoghegan.  I (tgl)
  failed to resist the temptation to wordsmith nearby text a bit while
  at it.
  http://git.postgresql.org/pg/commitdiff/7b6fb76349fccc38f0ef96166f207c6acd21968c

- Be more noisy about "wrong number of nailed relations" initfile
  problems.  In commit 5d1ff6bd559ea8df1b7302e245e690b01b9a4fa4 I
  added some logic to relcache.c to try to ensure that the regression
  tests would fail if we made a mistake about which relations belong
  in the relcache init files.  I'm quite sure I tested that, but I
  must have done so only for the non-shared-catalog case, because a
  report from Adam Brightwell showed that the regression tests still
  pass just fine if we bollix the shared-catalog init file in the way
  this code was supposed to catch.  The reason is that that file gets
  loaded before we do client authentication, so the WARNING is not
  sent to the client, only to the postmaster log, where it's far too
  easily missed.  The least Rube Goldbergian answer to this is to put
  an Assert(false) after the elog(WARNING).  That will certainly get
  developers' attention, while not breaking production builds' ability
  to recover from corner cases with similar symptoms.  Since this is
  only of interest to developers, there seems no need for a
  back-patch, even though the previous commit went into all branches.
  http://git.postgresql.org/pg/commitdiff/da3751c8ea6f833d8e69fbdd168d29d328674398

- Improve documentation around autovacuum-related storage parameters.
  These were discussed in three different sections of the manual,
  which unsurprisingly had diverged over time; and the descriptions of
  individual variables lacked stylistic consistency even within each
  section (and frequently weren't in very good English anyway).  Clean
  up the mess, and remove some of the redundant information in hopes
  that future additions will be less likely to re-introduce
  inconsistency.  For instance I see no need for maintenance.sgml to
  include its very own list of all the autovacuum storage parameters,
  especially since that list was already incomplete.
  http://git.postgresql.org/pg/commitdiff/6404751ce91fb02999e46700a1ac62b3c6319abf

- Do a round of copy-editing on the 9.5 release notes.  Also fill in
  the previously empty "major enhancements" list.  YMMV as to which
  items should make the cut, but it's past time we had something more
  than a placeholder here.  (I meant to get this done before beta2 was
  wrapped, but got distracted by PDF build problems.  Better late than
  never.)
  http://git.postgresql.org/pg/commitdiff/39b9978d9cac34ad799a5fa3ff3846f3e0372b0a

- Fix unwanted flushing of libpq's input buffer when socket EOF is
  seen.  In commit 210eb9b743c0645d I centralized libpq's logic for
  closing down the backend communication socket, and made the new
  pqDropConnection routine always reset the I/O buffers to empty.
  Many of the call sites previously had not had such code, and while
  that amounted to an oversight in some cases, there was one place
  where it was intentional and necessary *not* to flush the input
  buffer: pqReadData should never cause that to happen, since we
  probably still want to process whatever data we read.  This is the
  true cause of the problem Robert was attempting to fix in
  c3e7c24a1d60dc6a, namely that libpq no longer reported the backend's
  final ERROR message before reporting "server closed the connection
  unexpectedly".  But that only accidentally fixed it, by invoking
  parseInput before the input buffer got flushed; and very likely
  there are timing scenarios where we'd still lose the message before
  processing it.  To fix, pass a flag to pqDropConnection to tell it
  whether to flush the input buffer or not.  On review I think
  flushing is actually correct for every other call site.  Back-patch
  to 9.3 where the problem was introduced.  In HEAD, also improve the
  comments added by c3e7c24a1d60dc6a.
  http://git.postgresql.org/pg/commitdiff/c405918858c09d4264ffc5a3c73d57dc3efb0213

- Improve type numeric's calculations for ln(), log(), exp(), pow().
  Set the "rscales" for intermediate-result calculations to ensure
  that suitable numbers of significant digits are maintained
  throughout.  The previous coding hadn't thought this through in any
  detail, and as a result could deliver results with many inaccurate
  digits, or in the worst cases even fail with divide-by-zero errors
  as a result of losing all nonzero digits of intermediate results.
  In exp_var(), get rid entirely of the logic that separated the
  calculation into integer and fractional parts: that was neither
  accurate nor particularly fast.  The existing range-reduction method
  of dividing by 2^n can be applied across the full input range
  instead of only 0..1, as long as we are careful to set an
  appropriate rscale for each step.  Also fix the logic in mul_var()
  for shortening the calculation when the caller asks for fewer output
  digits than an exact calculation would require.  This bug doesn't
  affect simple multiplications since that code path asks for an exact
  result, but it does contribute to accuracy issues in the
  transcendental math functions.  In passing, improve performance of
  mul_var() a bit by forcing the shorter input to be on the left, thus
  reducing the number of iterations of the outer loop and probably
  also reducing the number of carry-propagation steps needed.  This is
  arguably a bug fix, but in view of the lack of field complaints, it
  does not seem worth the risk of back-patching.  Dean Rasheed
  http://git.postgresql.org/pg/commitdiff/7d9a4737c268f61fb8800957631f12d3f13be218

- Fix ruleutils.c's dumping of whole-row Vars in ROW() and VALUES()
  contexts.  Normally ruleutils prints a whole-row Var as "foo.*".  We
  already knew that that doesn't work at top level of a SELECT list,
  because the parser would treat the "*" as a directive to expand the
  reference into separate columns, not a whole-row Var.  However,
  Joshua Yanovski points out in bug #13776 that the same thing happens
  at top level of a ROW() construct; and some nosing around in the
  parser shows that the same is true in VALUES().  Hence, apply the
  same workaround already devised for the SELECT-list case, namely to
  add a forced cast to the appropriate rowtype in these cases.  (The
  alternative of just printing "foo" was rejected because it is
  difficult to avoid ambiguity against plain columns named "foo".)
  Back-patch to all supported branches.
  http://git.postgresql.org/pg/commitdiff/7745bc352a82bd588be986479c7aabc3b076a375

Álvaro Herrera pushed:

- vacuumdb: don't prompt for passwords over and over.  Having the
  script prompt for passwords over and over was a preexisting problem
  when it processed multiple databases or when it processed multiple
  analyze stages, but the parallel mode introduced in commit
  a179232047 made it worse.  Fix the annoyance by keeping a copy of
  the password used by the first connection that requires one.  Since
  users can (currently) only have a single password, there's no need
  for more complex arrangements (such as remembering one password per
  database).  Per bug #13741 reported by Eric Brown.  Patch authored
  and cross-reviewed by Haribabu Kommi and Michael Paquier, slightly
  tweaked by Álvaro Herrera.  Discussion:
  http://www.postgresql.org/message-id/20151027193919.931.54948@wrigleys.postgresql.org
  Backpatch to 9.5, where parallel vacuumdb was introduced.
  http://git.postgresql.org/pg/commitdiff/83dec5a712af251af15effbf781ddaedc3bf6b3b

Stephen Frost pushed:

- Correct sepgsql docs with regard to RLS The sepgsql docs included a
  comment that PG doesn't support RLS.  That is only true for versions
  prior to 9.5.  Update the docs for 9.5 and master to say that PG
  supports RLS but that sepgsql does not yet.  Pointed out by Heikki.
  Back-patch to 9.5
  http://git.postgresql.org/pg/commitdiff/42aa1c032e3520d608a1e89abbf36409b022d4b7

Bruce Momjian pushed:

- pg_upgrade:  properly detect file copy failure on Windows.
  Previously, file copy failures were ignored on Windows due to an
  incorrect return value check.  Report by Manu Joye Backpatch through
  9.1
  http://git.postgresql.org/pg/commitdiff/025106e314627f5aacdf72fca52ba0ed985d696e

- Fix spelling error in postgresql.conf.  Report by Greg Clough
  http://git.postgresql.org/pg/commitdiff/e57646e96284d4bad79f87a5c1dfbcd8f5434020

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Adam Brightwell sent in two revisions of a patch to bootstrap
pg_shseclabel in relcache initialization.

Kaigai Kouhei sent in another revision of a patch to reconcile Foreign
join pushdown with EvalPlanQual.

Ashutosh Bapat sent in another revision of a patch to add a
transaction manager for foreign servers.

Marco Nenciarini sent in a patch to fix an incompatibility between
9.3's WAL stream and pg_receivelog.

Kaigai Kouhei sent in a patch for each of 9.5 and git master to allow
CustomScan on copyfuncs.

Pavel Stěhule sent in another revision of a patch to allow psql to
send multiple SQL statements via the command line.

Pavel Stěhule and Catalin Iacob traded patches to add an ereport
function to PL/PythonU.

Artur Zakirov sent in another revision of a patch to enhance the
performance of Hunspell dictionaries in full-text search.

Peter Geoghegan sent in a patch to fix some erroneous documentation of
char(n).

Michael Paquier sent in another revision of a patch to add
trigonometric functions which take degree values as input.

Andres Freund sent in another revision of a patch to control
continuous flushing of checkpoints.

Thomas Munro sent in two revisions of a patch to create a "causal
reads" capability.

Peter Eisentraut sent in another revision of a patch to handle missing
pieces of compilation infrastructure more gracefully.

Michael Paquier sent in another revision of a patch to show tuple data
in pageinspect.

Craig Ringer sent in a patch to implement pg_logical, a plugin based
on logical WAL streams.

Kyotaro HORIGUCHI sent in a patch to allow using regexes in frontend
code, in particular tab completion for psql.

Thomas Munro sent in another revision of a patch to simplify tab
completion in psql.

Alexander Korotkov sent in another revision of a patch to rework the
access method interface.

Peter Geoghegan sent in a patch to expand SortSupport to char(n),
bytea, and alternative opclasses.

Kaigai Kouhei sent in a patch to add serialization support to
CustomScan.

SAWADA Masahiko sent in two more revisions of a patch to add a
"frozen" bit to the visibility map.

Amit Kapila sent in a patch to fix an issue in the parallel seq scan
code.

Robert Haas sent in a patch to implement Append in parallel.

Jeff Janes and Tom Lane traded patches to speed up set_rtable_names.

Amit Kapila sent in a patch to fix some issues in parallel seq scan.

Robert Haas sent in a patch to tranche-ify buffer locks.

Haribabu Kommi sent in another revision of a patch to skip ALTER x SET
SCHEMA if the schema didn't change.