== PostgreSQL Weekly News - July 11 2010 == - Mailing list pgsql-announce

== PostgreSQL Weekly News - July 11 2010 ==

PostgreSQL 9.0 beta 3 will be out soon.  Get ready to test!
http://www.postgresql.org/developer/beta

PostgreSQL-PostGIS-Hoeck will be happening July 14th 2010, 6pm in
Zurich, Switzerland at the bQm kultur café & bar at the ETH Zurich.
Information and (informal) registration:
http://www.gis.hsr.ch/wiki/Agenda

Reuven Lerner will be teaching a 5-day course in PostgreSQL August
1-5, 2010 at the Hi-Tech College in Herzliya, Israel.
http://www.hi-tech.co.il/college/default.asp?PageID=12&CourseNum=4288

== PostgreSQL Product News ==

Benetl 3.5, an ETL tool for files using PostgreSQL, released.
http://www.benetl.net

DataArchitect 4.2, an ERD and reverse engineering tool which supports
PostgreSQL, released.
http://www.thekompany.com/products/dataarchitect/

PostgreSQL Maestro 10.7, a GUI admin tool for PostgreSQL, released.
http://www.sqlmaestro.com/products/postgresql/maestro/

Muldis-D 0.130.0, a specification for an object-relational language
intended to run atop, among other systems, PostgreSQL, released.
http://muldis.com/Muldis_D.html

RHQ 3.0.0, a server management and monitoring application, released.
http://rhq-project.org/

== PostgreSQL Jobs for July ==

http://archives.postgresql.org/pgsql-jobs/2010-07/threads.php

== PostgreSQL Local ==

PDXPUG Day will be July 18, 2010 at the Oregon Convention Center in
Portland, OR.  For more information, see:
http://wiki.postgresql.org/wiki/PDXPUGDay2010

OSCON will take place in Portland, Oregon July 19-23, 2010.
http://www.oscon.com/oscon2010

FrOSCon 2010 will take place in St. Augustin, Germany on August 21-22,
2010.  The German PostgreSQL User Group got his own devroom and is
looking for talks.  More info:
http://andreas.scherbaum.la/blog/archives/711-FrOSCon-2010-PostgreSQL-devroom-Call-for-papers.html

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david@fetter.org, German language
to pwn@pgug.de, Italian language to pwn@itpug.org.  Spanish language
to pwn@arpug.com.ar.

== Reviews ==

KaiGai Kohei reviewed Robert Haas's patch to add get_whatever_oid
functionality.
== Applied Patches ==

Tom Lane committed:

- Improve pg_dump's checkSeek() function to verify the functioning of
  ftello as well as fseeko, and to not assume that fseeko(fp, 0,
  SEEK_CUR) proves anything.  Also improve some related comments.  Per
  my observation that the SEEK_CUR test didn't actually work on some
  platforms, and subsequent discussion with Robert Haas.  Back-patch
  to 8.4.  In earlier releases it's not that important whether we get
  the hasSeek test right, but with parallel restore it matters.

- In pgsql/doc/src/sgml/release-9.0.sgml, add compatibility note
  warning that plpgsql is now stricter about the column datatypes of
  composite results, per gripe from Marcel Asio.  Some desultory
  copy-editing of plpgsql-related sections of the release notes.

- Get rid of some more stuff in pg_upgrade that duplicates c.h or
  port.h.

- In pgsql/doc/src/sgml/datatype.sgml, fix assorted misstatements and
  poor wording in the descriptions of the I/O formats for geometric
  types.  Per bug #5536 from Jon Strait, and my own testing.
  Back-patch to all supported branches, since this doco has been wrong
  right along -- we certainly haven't changed the I/O behavior of
  these types in many years.

- Replace max_standby_delay with two parameters,
  max_standby_archive_delay and max_standby_streaming_delay, and
  revise the implementation to avoid assuming that timestamps found in
  WAL records can meaningfully be compared to clock time on the
  standby server.  Instead, the delay limits are compared to the
  elapsed time since we last obtained a new WAL segment from archive
  or since we were last "caught up" to WAL data arriving via streaming
  replication.  This avoids problems with clock skew between primary
  and standby, as well as other corner cases that the original coding
  would misbehave in, such as the primary server having significant
  idle time between transactions.  Per my complaint some time ago and
  considerable ensuing discussion.  Do some desultory editing on the
  hot standby documentation, too.

- Make vacuum_defer_cleanup_age be PGC_SIGHUP level, since it's not
  sensible to have different values in different processes of the
  primary server.  Also put it into the "Streaming Replication" GUC
  category; it doesn't belong in "Standby Servers" because you use it
  on the master not the standby.  In passing also correct guc.c's idea
  of wal_keep_segments' category.

- In pgsql/src/backend/access/transam/xlog.c, don't set
  recoveryLastXTime when replaying a checkpoint --- that was a bogus
  idea from the start since the variable is only meant to track
  commit/abort events.  This patch reverts the logic around the
  variable to what it was in 8.4, except that the value is now kept in
  shared memory rather than a static variable, so that it can be
  reported correctly by CreateRestartPoint (which is executed in the
  bgwriter).

- Split the LDFLAGS make variable into two parts: LDFLAGS is now used
  for linking both executables and shared libraries, and we add on
  LDFLAGS_EX when linking executables or LDFLAGS_SL when linking
  shared libraries.  This provides a significantly cleaner way of
  dealing with link-time switches than the former behavior.  Also,
  make sure that the various platform-specific %.so: %.o rules
  incorporate LDFLAGS and LDFLAGS_SL; most of them missed that before.
  (I did not add these variables for the platforms that invoke $(LD)
  directly, however.  It's not clear if we can do that safely, since
  for the most part we assume these variables use CC command-line
  syntax.)  Per gripe from Aaron Swenson and subsequent investigation.

- Fix a few single-file (MODULES, not MODULE_big) contrib makefiles
  that were supposing that they should set SHLIB_LINK rather than
  LDFLAGS_SL.  Since these don't go through Makefile.shlib that was a
  no-op on most platforms.  Also regularize the few platform-specific
  Makefiles that did pay attention to SHLIB_LINK: it seems that the
  real value of that is to pull in BE_DLLLIBS, so do that instead.
  Per buildfarm failures on cygwin.

- In pgsql/src/makefiles/pgxs.mk, make sure LDFLAGS come before LIBS
  when linking contrib programs.  Solaris, at least, seems to be
  sensitive to the relative order of -L and -l switches, so this is
  needed.  Per buildfarm results.

- In pgsql/src/makefiles/pgxs.mk, dept. of third thoughts: PG_LIBS may
  contain a -L switch, so it had better stay in front of LDFLAGS.

- In pgsql/src/makefiles/Makefile.aix, allow for LDFLAGS_SL already
  having a value in Makefile.aix.  Per buildfarm results.

- In pgsql/src/Makefile.shlib, still more third thoughts: when linking
  shared libraries, LDFLAGS probably needs to appear before anything
  placed in SHLIB_LINK.  This is because SHLIB_LINK is typically a
  subset of LIBS, and LIBS has to appear after LDFLAGS on platforms
  that are sensitive to the relative order of -L and -l switches.

- In pgsql/src/backend/utils/mb/mbutils.c, undo pgindent breakage
  (again).  Per buildfarm.

- In pgsql/src/Makefile.global.in, accept slightly grotty coding in
  Makefile.global in order to keep the -L flag for src/port/ in front
  of any -L flags placed in LDFLAGS by configure.  This undoes an
  L-flag-ordering change that I had thought would be safe, but seems
  to be making at least one buildfarm member fail --- the only theory
  for orca's failure that I can think of is that it's got an old copy
  of libpgport.a in /usr/lib.  Also allow for LDFLAGS_SL to be set by
  contrib makefiles before they invoke Makefile.global.

- In pgsql/src/backend/utils/mb/mbutils.c, adjust mbutils.c so it
  won't get broken by future pgindent runs.  To do that, replace L'\0'
  by (WCHAR) 0.  Perhaps someday we should teach pgindent about
  wide-character literals, but so long as this is the only use-case in
  the entire Postgres sources, a workaround seems easier.

- Fix "cannot handle unplanned sub-select" error that can occur when a
  sub-select contains a join alias reference that expands into an
  expression containing another sub-select.  Per yesterday's report
  from Merlin Moncure and subsequent off-list investigation.
  Back-patch to 7.4.  Older versions didn't attempt to flatten
  sub-selects in ways that would trigger this problem.

- Fix variant float8 expected files to have exactly the expected
  spacing.  This wasn't important when we used diff's -w
  (--ignore-all-space) option to compare regression result files, but
  it is now.  Per buildfarm member canary, which evidently has been
  offline since we did that in November, but came to life again today.

- In pgsql/src/backend/access/transam/xlogutils.c, update obsolete
  comment.  Noted by Josh Tolley.

- In pgsql/doc/src/sgml/ref/create_operator.sgml, add a
  cross-reference to precedence information to CREATE OPERATOR's
  documentation.  Per suggestion from Marc Cousin.

- In pgsql/src/pl/plpython/plpython.c, minor kibitzing on previous
  patch from Peter Eisentraut which guards against using PL/PythonU2
  and PL/PythonU3 in the same session: no need to run check more than
  once.  (_PG_init should be called only once anyway, but as long as
  it's got an internal guard against repeat calls, that should be in
  front of the version check.)

- Stamp HEAD as 9.1devel.  (And there was much rejoicing.)

- In pgsql/src/backend/utils/adt/ruleutils.c, fix ruleutils'
  get_variable() to print something useful for Vars referencing
  resjunk outputs of subquery tlists, instead of throwing an error.
  Per bug #5548 from Daniel Grace.  We might at some point find we
  ought to back-patch this further than 9.0, but I think that such
  Vars can only occur as resjunk members of upper-level tlists, in
  which case the problem can't arise because prior versions didn't
  print resjunk tlist items in EXPLAIN VERBOSE.

- In pgsql/src/backend/utils/cache/lsyscache.c, avoid an Assert
  failure in deconstruct_array() by making get_attstatsslot() use the
  actual element type of the array it's disassembling, rather than
  trusting the type OID passed in by its caller.  This is needed
  because sometimes the planner passes in a type OID that's only
  binary-compatible with the target column's type, rather than being
  an exact match.  Per an example from Bernd Helmle.  Possibly we
  should refactor get_attstatsslot/free_attstatsslot to not expect the
  caller to supply type ID data at all, but for now I'll just do the
  minimum-change fix.  Back-patch to 7.4.  Bernd's test case only
  crashes back to 8.0, but since these subroutines are the same in
  7.4, I suspect there may be variant cases that would crash 7.4 as
  well.

- In pgsql/doc/src/sgml/sources.sgml, add a note about preferred
  window width to the section about code formatting conventions.

Bruce Momjian committed:

- In pgsql/doc/src/sgml/high-availability.sgml, remove tab from SGML.

- Add new Non-Durable Settings documentation section.  Document that
  synchronous_commit can lose transactions in a db crash, not just a
  OS crash.

- Fix doc double-wording in non-durable patch.  Report from Thom Brown.

- In pgsql/src/backend/access/transam/xact.c, add C comment about why
  synchronous_commit=off behavior can lose committed transactions in a
  postmaster crash.

- In pgsql/doc/src/sgml/config.sgml, document that shared_preload_libraries
  and local_preload_libraries lowercase the library names, unless
  double-quoted.

- In documentation, use "lower case"/"upper case" consistently (use
  space between words).

- In pgsql/doc/src/sgml/config.sgml, document that /bin/true on
  Windows is implemented by 'REM'.

- In pgsql/doc/src/sgml/release-9.0.sgml, update 9.0 release notes so
  streaming replication and hot standby is not assumed to require
  continuous archiving.  Per report from Fujii Masao.

- In pgsql/contrib/pg_upgrade/pg_upgrade.h, remove port.h prototypes
  from pg_upgrade.h, per report from Robert Haas.

- In pgsql/src/backend/commands/tablespace.c, issue 'mkdir' hint when
  replying CREATE TABLESPACE in recovery mode.  Per idea from Fujii
  Masao.

- Add CVS tags to pg_upgrade and pg_upgrade_support files, per request
  from Tom Lane.

- Add copyrights to pg_upgrade and pg_upgrade_tools files, per Tom
  Lane.

- Make pg_upgrade copyrights just 2010, not 2010-2010.

- In pgsql/doc/src/sgml/func.sgml, document more clearly on XML
  namespaces inside xpath function.  Nikolay Samokhvalov.

- In pgsql/doc/src/sgml/config.sgml, remove SGML tab.

- In pgsql/src/tools/fsync/test_fsync.c, report test_fynsc times in
  tests per second, instead of total seconds.

- In pgsql/src/tools/fsync/test_fsync.c, simplify test_fsync duration
  computation.

- Update pgindent testing instructions.

- pgindent run for 9.0, second run.

- In pgsql/src/tools/pgindent/README, mention why one C file fails
  pgindent.

- In pgsql/doc/src/sgml/wal.sgml, document the interaction of
  write-barrier-enabled file systems, and BBU caches, per June email
  thread.

- In pgsql/doc/src/sgml/wal.sgml, doc change:  effected -> affected,
  per correction from Matthew Wakeling.

- In pgsql/doc/src/sgml/release-9.0.sgml, update release notes for 9.0
  beta 3.

- Properly report errno/out-of-disk-space error from pg_upgrade when
  in copy mode, per report from depstein@alliedtesting.com.  Patch
  suggestion from Magnus.  Backpatch to 9.0.X.

Peter Eisentraut committed:

- In pgsql/doc/src/sgml/sources.sgml, add guidelines for formatting
  errcontext strings

- In pgsql/doc/src/sgml/datatype.sgml, add note clarifying that XML
  fragments don't accept DTDs, per complaint from Craig Ringer.

- Use different function names for plpython3 handlers, to avoid
  clashes in pg_pltemplate.  This should have a catversion bump, but
  it's still being debated whether it's worth it during beta.

- Message tuning.

- In pgsql/doc/src/sgml/plpython.sgml, add note that using PL/Python 2
  and 3 in the same session will probably crash.

- Install safeguard against running PL/Python 2 and 3 in the same
  session.

- Translation updates for 9.0beta3.

- In pgsql/doc/src/sgml/plperl.sgml, some small changes: wrapped long
  code-lines for pdf, fix typo.  Erik Rijkers

Robert Haas committed:

- In pgsql/src/backend/access/transam/xlog.c, emode_for_corrupt_record
  shouldn't reduce LOG messages to WARNING.  In non-interactive
  sessions, WARNING sorts below LOG.

- In pgsql/src/backend/commands/tablecmds.c, allow ALTER TABLE .. SET
  TABLESPACE to be interrupted.  Backpatch to 8.0, where tablespaces
  were introduced.  Guillaume Lelarge

- In pgsql/src/port/copydir.c, allow copydir() to be interrupted.
  This makes ALTER DATABASE .. SET TABLESPACE and CREATE DATABASE more
  sensitive to interrupts.  Backpatch to 8.4, where ALTER DATABASE ..
  SET TABLESPACE was introduced.  We could go back further, but in the
  absence of complaints about the CREATE DATABASE case it doesn't seem
  worth it.  Guillaume Lelarge, with a small correction by me.

- Move copydir.c from src/port to src/backend/storage/file.  The
  previous commit to make copydir() interruptible prevented
  postgres.exe from linking on MinGW and Cygwin, because on those
  platforms libpgport_srv.a can't freely reference symbols defined by
  the backend.  Since that code is already backend-specific anyway,
  just move the whole file into the backend rather than adding further
  kludges to deal with the symbols needed by CHECK_FOR_INTERRUPTS().
  This probably needs some further cleanup, but this commit just moves
  the file as-is, which should hopefully be enough to turn the
  buildfarm green again.

- Remove hstore % text[] operator; use slice() function instead.
  David Wheeler, with one small correction by me.

- Additional cross-references to window functions documentation.  Erik
  Rijkers.

- Allow REASSIGNED OWNED to handle opclasses and opfamilies.
  Backpatch to 8.3, which is as far back as we have opfamilies.  The
  opclass portion could probably be backpatched to 8.2, when REASSIGN
  OWNED was added, but for now I have not done that.  Asko Tiidumaa,
  with minor adjustments by me.

- Support setting the keepalive idle time on MacOS X.  MacOS X uses
  TCP_KEEPALIVE rather than TCP_KEEPIDLE for this purpose.  Thanks to
  Fujii Masao for the review.

- Make log_temp_files based on kB, and revert docs & comments to
  match.  Per extensive discussion on pgsql-hackers.  We are
  deliberately not back-patching this even though the behavior of 8.3
  and 8.4 is unquestionably broken, for fear of breaking existing
  users of this parameter.  This incompatibility should be
  release-noted.

- Add a hook in ExecCheckRTPerms().  This hook allows a loadable
  module to gain control when table permissions are checked.  It is
  expected to be used by an eventual SE-PostgreSQL implementation, but
  there are other possible applications as well.  A sample contrib
  module can be found in the archives at:
  http://archives.postgresql.org/pgsql-hackers/2010-05/msg01095.php
  Robert Haas and Stephen Frost.

- In pgsql/doc/src/sgml/ref/psql-ref.sgml, clarify that "psql -c"
  ignores psqlrc files.  Tim Landscheidt

Andrew Dunstan committed:

- In pgsql/doc/src/sgml/release-9.0.sgml, correct missing/misspelled
  surname.

- In pgsql/src/tools/msvc/Mkvcbuild.pm, unbreak MSVC builds by
  removing copydir.c from list of libpgport files

Heikki Linnakangas committed:

- stringToNode() and deparse_expression_pretty() crash on invalid
  input, but we have nevertheless exposed them to users via
  pg_get_expr().  It would be too much maintenance effort to
  rigorously check the input, so put a hack in place instead to
  restrict pg_get_expr() so that the argument must come from one of
  the system catalog columns known to contain valid expressions.  Per
  report from Rushabh Lathia.  Backpatch to 7.4 which is the oldest
  supported version at the moment.

- The previous fix in CVS HEAD and 8.4 for handling the case where a
  cursor being used in a PL/pgSQL FOR loop is closed was inadequate,
  as Tom Lane pointed out.  The bug affects FOR statement variants
  too, because you can close an implicitly created cursor too by
  guessing the "<unnamed portal X>" name created for it.  To fix that,
  "pin" the portal to prevent it from being dropped while it's being
  used in a PL/pgSQL FOR loop.  Backpatch all the way to 7.4 which is
  the oldest supported version.

Magnus Hagander committed:

- Add support for TCP keepalives on Windows, both for backend and the
  new libpq support.

- Make the Windows tcp keepalive support depend on the existance of
  the SIO_KEEPALIVE_VALS define instead of just WIN32, since MingW
  doesn't support this API (yet?).

Marc Fournier committed:

- tag beta3

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Robert Haas sent in another revision of the patch to suppress
automatic recovery after backend crash.  This is for a volatile
caching version of PostgreSQL.

Robert Haas sent in another revision of the patch to add
get_whatever_oid functionality.

Magnus Hagander sent in two revisions of a patch to fix keepalives on
Win32.

Robert Haas sent in a patch to implmenet a warning message in
pg_standby.

Mike Fowler sent in another revision of the xpath_exists patch.

Peter Eisentraut sent in another revision of the patch to allow server
authentication over Unix-domain sockets.

Mike Fowler sent in another revision of patch to add XMLEXISTS to the
grammar.

Robert Haas sent in two revisions of a patch to fix keepalives on OSX.

Bruce Momjian sent in a patch to show a hint only in recovery after a
crash of Hot Standby/Streaming Replication.

Fujii Masao sent in an updated patch to fix the docs re:
checkpoint_segments and keep_wal_segments for Hot Standby/Streaming
Replication.

Martin Pihlak sent in two revisions of a patch to add a GUC
log_file_mode which allows specifying the creation mode for log files.
This would enable, for example, permitting readers other than the
system PostgreSQL user to read the log files PostgreSQL generates.

Teodor Sigaev sent in a patch to implement more accurate cost
estimation function for GIN index scans.

Markus Shiltknecht sent in a patch to add a lock-based dynamic shared
memory allocator, which lets components delay memory allocation rather
than allocate all needed memory in advance.

Pavel Stehule sent in a patch to add parameters to DO statements.

Robert Haas sent in three more revisions of a patch to implement
get_whatever_oid, a refactoring which to a large extent centralizes
get_*_oid logic.

Robert Haas sent in a patch to fix log_temp_files docs and comments to
say kilobytes rather than bytes.

Greg Smith sent in another revision of the patch to make pgbench more
64-bit-clean.

ITAGAKI Takahiro sent in another revision of the patch to help make
CLUSTER behave better.

Mike Fowler sent in two more revisions of a patch to add an
xml_is_well_formed function.

Rajanikant Chirmade sent in a patch to make downcase_truncate_identifier()
multibyte-character-aware.

Pavel Stehule sent in two more revisions of the string functions for
9.1.

Robert Haas sent in another revision of the patch to include the
backend ID in relpath of temp rels.

Pavel Baros sent in another revision of the patch to add materialized
views.

Kevin Grittner sent in another revision of the patch to implement
predicate locking, in part a prerequisite for better serialization.

Robert Haas sent in a patch to slim down the representation of
numerics by about two bytes.

Zhai Boxuan sent in a patch to implement a MERGE command.

Erik Rijkers sent in a patch to fix the contrib makefile to include
the stringfunc patch.