== PostgreSQL Weekly News - September 24 2007 == - Mailing list pgsql-announce

== PostgreSQL Weekly News - September 24 2007 ==

Postgres releases 8.2.5, 8.1.10, 8.0.14, 7.4.18, and 7.3.20 are out.
Upgrade ASAP!

Following the release of PostgreSQL 8.3 later this year, PostgreSQL
8.1.x and 8.0.x will cease to be officially supported on Windows.

== PostgreSQL Product News ==

MS Access to PostgreSQL Converter 2.0 released.
http://pgfoundry.org/projects/access2postgres/

PgPool 3.4.1 released.
http://pgfoundry.org/projects/pgpool/

New PostgreSQL RPM sets are available for Fedora / RHEL
http://www.postgresql.org/download/

== PostgreSQL Jobs for September ==

http://archives.postgresql.org/pgsql-jobs/2007-09/threads.php

== PostgreSQL Local ==

PostgreSQL Fall Conference will be October 20, 2007 in Portland.
http://www.postgresqlconference.org/

There will be a PostgreSQL booth at Ohio Linuxfest on September 29th,
and Robert Treat will be speaking.
http://www.ohiolinux.org/speakers.html#talk1_06

== PostgreSQL in the News ==

Planet PostgreSQL: http://www.planetpostgresql.org/

General Bits, Archives and occasional new articles:
http://www.varlena.com/GeneralBits/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david@fetter.org, German to
pwn@pgug.de.


== Applied Patches ==

Michael Meskes committed:

- Removed superfluous ECPGfree() call from
  pgsql/src/interfaces/ecpg/ecpglib/execute.c.

Neil Conway committed:

- In pgsql/src/backend/utils/adt/float.c, prevent corr() from
  returning the wrong results for negative correlation values. The
  previous coding essentially assumed that x = sqrt(x*x), which does
  not hold for x < 0.  Thanks to Jie Zhang at Greenplum and Gavin
  Sherry for reporting this issue.

Andrew Dunstan committed:

- Add perl replacements for build.bat and vcregress.bat. In due course
  the .bat files will be altered to become tiny wrappers for these
  scripts, and one or two other .bat files will disappear.

- In pgsql/src/tools/msvc/Install.pm, restrict overly broad searches,
  and fix typo, in recent fix. Per Hannes Eder.

- In pgsql/src/tools/msvc/Install.pm, replace calls to external dir
  program with perlish globs and File::Find calls. Fixes complaint
  fron Hannes Eder, whose environment found a different dir program.

- Go back to using a separate method for doing ILIKE for single byte
  character encodings that doesn't involve calling lower().  This
  should cure the performance regression in this case complained of by
  Guillaume Smet.  It still leaves the horrid performance for
  multi-byte encodings introduced in 8.2, but there's no obvious
  solution for that in sight.

- Cleanup items from csvlog changes, per ITAGAKI Takahiro.

- Close previously open holes for invalidly encoded data to enter the
  database via builtin functions, as recently discussed on -hackers.
  chr() now returns a character in the database encoding. For UTF8
  encoded databases the argument is treated as a Unicode code point.
  For other multi-byte encodings the argument must designate a strict
  ascii character, or an error is raised, as is also the case if the
  argument is 0.  ascii() is adjusted so that it remains the inverse
  of chr().

Teodor Sigaev committed:

- In pgsql/src/backend/utils/adt/tsrank.c, fix msvc warnings, patch by
  Hannes Eder.

- In pgsql/src/backend/tsearch/dict_thesaurus.c, avoid
  possibly-unportable initializer, per Gregory Stark and buildfarm
  warning.

Bruce Momjian committed:

- In FAQ_DEV, fix typo per Brendan Jurd.

- Mark TODOs as done: "Consider shrinking expired tuples to just their
  headers," "Allow heap reuse of UPDATEd rows if no indexed columns
  are changed, and old and new versions are on the same heap page,"
  and remove no-longer-needed TODO: "Reuse index tuples that point to
  heap tuples that are not visible to anyone?"

- In pgsql/doc/src/sgml/func.sgml, remove tabs from SGML file.

- In pgsql/src/tools/RELEASE_CHANGES, add reminder that integer pg
  version also needs updating.

- Remove extra tab in postgresql.conf

Tom Lane committed:

- Replace linear searches with binary searches in pg_dump's code to
  lookup objects by OID.  Per gripe from nikitathespider.

- In pgsql/src/backend/utils/adt/xml.c, fix bugs in XML binary I/O
  functions.  Heikki and Tom

- Make autovacuum report the start time of its current activity in
  pg_stat_activity.  Per gripe from Jim Nasby.

- In pgsql/src/backend/storage/ipc/procarray.c,
  transactionIdIsInProgress can skip scanning the ProcArray if the
  target XID is later than latestCompletedXid, per Florian Pflug.
  Also some minor improvements in the XIDCACHE_DEBUG code --- make
  sure each call of TransactionIdIsInProgress is counted one way or
  another.

- Temporarily modify tsearch regression tests to suppress notice that
  comes out at erratic times, because it is creating a totally
  unacceptable level of noise in our buildfarm results.  This patch
  can be reverted when and if the code is fixed to not issue notices
  during cache reload events.

- Fix cost estimates for EXISTS subqueries that are evaluated as
  initPlans (because they are uncorrelated with the immediate parent
  query).  We were charging the full run cost to the parent node,
  disregarding the fact that only one row need be fetched for EXISTS.
  While this would only be a cosmetic issue in most cases, it might
  possibly affect planning outcomes if the parent query were itself a
  subquery to some upper query.  Per recent discussion with Steve
  Crawford.

- In pgsql/doc/src/sgml/config.sgml, document the translations from
  Postgres message severity levels to syslog and eventlog severity
  levels, per suggestion from Josh Drake.  Also, some wordsmithing for
  the csvlog documentation.

- In pgsql/src/backend/postmaster/syslogger.c, fix erroneous Assert()
  in syslogger process start in EXEC_BACKEND case, per ITAGAKI
  Takahiro.  Also, rewrite syslogger_forkexec() in hopes of
  eliminating the confusion in the first place.

- In pgsql/src/backend/utils/adt/oracle_compat.c, fix bogus
  calculation of potential output string length in translate().

- In pgsql/src/include/fmgr.h, parenthesize macro arguments safely.  I
  see no bug among the current uses of PG_DETOAST_DATUM_SLICE, but
  it's clearly trouble waiting to happen.

- Although I'd misdiagnosed the reason for the recent failures on
  buildfarm member grebe, I see no reason to revert the
  1-byte-header-friendly changes I made in varlena.c.  Instead, tweak
  the code a little bit to get more advantage out of that.

- In pgsql/src/backend/utils/adt/regexp.c, defend against the case
  where malloc returns NULL for malloc(0).

- Fix varlena.c routines to allow 1-byte-header text values.  This is
  now demonstrably necessary for text_substring() since regexp_split
  functions may pass it such a value; and we might as well convert the
  whole file at once.  Per buildfarm results (though I wonder why most
  machines aren't showing a failure).

- Fix regex, LIKE, and some other second-rank text-manipulation
  functions to not cause needless copying of text datums that have
  1-byte headers.  Greg Stark, in response to performance gripe from
  Guillaume Smet and ITAGAKI Takahiro.

- Improve handling of prune/no-prune decisions by storing a page's
  oldest unpruned XMAX in its header.  At the cost of 4 bytes per
  page, this keeps us from performing heap_page_prune when there's no
  chance of pruning anything.  Seems to be necessary per Heikki's
  preliminary performance testing.

- In pgsql/src/backend/utils/time/tqual.c, change tqual.c tests to use
  !TransactionIdIsCurrentTransactionId, rather than
  TransactionIdDidAbort, when handling the case that xmin is one of
  the current transaction's XIDs and the tuple has been deleted.  xmax
  must also be one of the current transaction's XIDs, since no one
  else can see it yet, and it's cheaper to look at local state than
  shared state to find out if xmax aborted.  Per an idea of Heikki's.

- In  pgsql/src/backend/storage/ipc/procarray.c, make some simple
  performance improvements in TransactionIdIsInProgress().  For XIDs
  of our own transaction and subtransactions, it's cheaper to ask
  TransactionIdIsCurrentTransactionId() than to look in shared memory.
  Also, the xids[] work array is always the same size within any given
  process, so malloc it just once instead of doing a palloc/pfree on
  every call; aside from being faster this lets us get rid of some
  goto's, since we no longer have any end-of-function pfree to do.
  Both ideas by Heikki Linnakangas.

- In pgsql/src/backend/access/transam/twophase.c, fix comments that
  misspelled TransactionIdIsInProgress, per Heikki Linnakangas.

- If we're gonna provide an --enable-profiling configure option,
  surely it ought to know that you need -DLINUX_PROFILE on Linux.

- Insert a hack in pl/tcl to disable Tcl's built-in Notifier
  subsystem, which has a bad habit of launching multiple threads
  within the backend and thereby causing all kinds of havoc.
  Fortunately, we don't need it, and recent Tcl versions provide an
  easy way to disable it.  Diagnosis and fix by Steve Marshall, Paul
  Bayer, and Doug Knight of WSI Corporation.

- Solaris portability fix that was previously made in contrib/tsearch2
  but got lost from the version committed to main tree.  Per Greg
  Stark.

- Revert ill-fated patch to release exclusive lock early after vacuum
  truncates a table.  Introduces race condition, as shown by buildfarm
  failures.

- HOT updates.  When we update a tuple without changing any of its
  indexed columns, and the new version can be stored on the same heap
  page, we no longer generate extra index entries for the new version.
  Instead, index searches follow the HOT-chain links to ensure they
  find the correct tuple version.  In addition, this patch introduces
  the ability to "prune" dead tuples on a per-page basis, without
  having to do a complete VACUUM pass to recover space.  VACUUM is
  still needed to clean up dead index entries, however.  Pavan
  Deolasee, with help from a bunch of other people.

- Remove Assert(BgWriterShmem != NULL), which is rather pointless
  since we'd dump core anyway immediately afterward if it were null;
  and it seems to confuse some versions of icc into generating bad
  code.  Per report from Sergey Koposov.  Patched in HEAD only, for
  the moment, since this is only likely to affect developers.

- Fix overflow in extract(epoch from interval) for intervals exceeding
  68 years.  Seems to have been introduced in 8.1 by careless
  SECS_PER_DAY search-and-replace.

- Update release notes for last-minute fix.

- Fix aboriginal mistake in lazy VACUUM's code for truncating away
  no-longer-needed pages at the end of a table.  We thought we could
  throw away pages containing HEAPTUPLE_DEAD tuples; but this is not
  so, because such tuples very likely have index entries pointing at
  them, and we wouldn't have removed the index entries.  The problem
  only emerges in a somewhat unlikely race condition: the dead tuples
  have to have been inserted by a transaction that later aborted, and
  this has to have happened between VACUUM's initial scan of the page
  and then rechecking it for empty in count_nondeletable_pages.  But
  that timespan will include an index-cleaning pass, so it's not all
  that hard to hit.  This seems to explain a couple of previously
  unsolved bug reports.

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Bernd Helmle sent in a patch which adds a "Login" field to psql's \du
command, displaying wether a role has the LOGIN privilege granted or
not.

Heikki Linnakangas sent in a patch which optimizes pg_next_dst_boundary.