Thread: 9.5 release notes

9.5 release notes

From
Bruce Momjian
Date:
I have committed the first draft of the 9.5 release notes.  You can view
the output here:
http://momjian.us/pgsql_docs/release-9-5.html
and it will eventually appear here:
http://www.postgresql.org/docs/devel/static/release.html

I am ready to make suggested adjustments, though I am traveling for
conferences for the next ten days so there might a delay in my replies.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Amit Kapila
Date:
On Thu, Jun 11, 2015 at 9:45 AM, Bruce Momjian <bruce@momjian.us> wrote:
>
> I have committed the first draft of the 9.5 release notes.  You can view
> the output here:
>
>         http://momjian.us/pgsql_docs/release-9-5.html
>

Thanks for writing the Release notes.

Some comments:

Have pg_basebackup use a tablespace mapping file, to allow for file paths of 100+ characters in length

I think this is not completely correct. This is mainly done to allow
usage of tar format in Windows when tablespaces are present
in database, although we have eventually done it for both
Windows and Linux in the same way.  So how about: 

Have pg_basebackup use a tablespace mapping file, to allow usage of tar format
consistently across all platforms

Also shall we mention about below in Migrations to 9.5 section

"pg_basebackup will not not work in tar mode against 9.4 and older servers,
 as we have introduced a new protocol option in that mode."


With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Re: 9.5 release notes

From
Michael Paquier
Date:
On Thu, Jun 11, 2015 at 1:15 PM, Bruce Momjian <bruce@momjian.us> wrote:
> I have committed the first draft of the 9.5 release notes.  You can view
> the output here:
>
>         http://momjian.us/pgsql_docs/release-9-5.html
>
> and it will eventually appear here:
>
>         http://www.postgresql.org/docs/devel/static/release.html
>
> I am ready to make suggested adjustments, though I am traveling for
> conferences for the next ten days so there might a delay in my replies.

Here are some review comments:
+       <para>
+        RETURN WHERE
+       </para>
What is that?

+       <para>
+        WHAT IS A STATISTICS SNAPSHOT?
+       </para>
+      </listitem>
It defines the last time when the global statistics file of pgstat has
been written. Perhaps documentation should be made clearer.

+       <para>
+        The remote snapshot must have been exported by
+        <function>pg_export_snapshot()</> or been defined as a
+        logical replication slot.  This can be used by parallel
+        <application>pg_dump</> to use a consistent snapshot across
+        <application>pg_dump</> processes.
+       </para>
Perhaps "or been defined when creating a logical replication slot
through a replication connection".

+      <listitem>
+       <para>
+        Simplify <acronym>WAL</> record format (Heikki Linnakangas)
+       </para>
+
+       <para>
+        This allows external tools to more easily process <acronym>WAL</>
+        files.
+       </para>
+      </listitem>
More precision could be brought here. What the new format allows is
actually to track more easily what are the blocks modified for
relations, something possible without having the knowledge of the
record type directly.

+       <para>
+        This is particularly helpful for warm standbys.
+       </para>
"for warm standbys to control the timing at which WAL segment files
are retrieved from a WAL archive."

I think that the following items should be added as well:
- Improvement of file version information for Windows builds (Noah
Misch, Michael Paquier), commit ee9569e. The file version information
was missing for a set of contrib modules as well as a handful of
libraries and binaries (like conversion_procs, zic, pg_regress, etc.).
This item should mention that all the binaries and libraries produced
by a Windows build now contain file version information. This could be
merged as well with this item as both are related:
+       <para>
+        Add icons to all <productname>MSVC</>-built binaries (Noah Misch)
+       </para>
- Move pg_lzcompress and pg_lzdecompress to libpqcommon, commit
40bede54. This was some legwork for wal_compression but external
binary tools can take advantage of using it now more freely. Those
APIs have been reworked as well to be more generic, somewhat similarly
to the interface lz4 exposes to the user.
- Addition of palloc_extended (8c8a886) to give module developers a
fallback plan instead of OOM ERROR that palloc throws mandatorily.
MemoryContextAllocExtended() can be used on another memory context
than the current one similarly (bd4e2fd9). Feel free to discard this
one if this is not appropriate in the release notes.
Regards,
-- 
Michael



Re: 9.5 release notes

From
Amit Langote
Date:
On 2015-06-11 PM 01:15, Bruce Momjian wrote:
> I have committed the first draft of the 9.5 release notes.  You can view
> the output here:
> 
>     http://momjian.us/pgsql_docs/release-9-5.html
>     
> and it will eventually appear here:
> 
>     http://www.postgresql.org/docs/devel/static/release.html
> 
> I am ready to make suggested adjustments, though I am traveling for
> conferences for the next ten days so there might a delay in my replies.
> 

In the last section E.1.3.11.1. pgbench, there is:

+      <listitem>
+       <para>
+        Add information about buffer pins to <application>pg_buffercache</>
+        display (Andres Freund)
+       </para>
+      </listitem>

Should be moved its own section?

Thanks for working on this!

Regards,
Amit




Re: 9.5 release notes

From
David Rowley
Date:

On 11 June 2015 at 16:15, Bruce Momjian <bruce@momjian.us> wrote:
I have committed the first draft of the 9.5 release notes.  You can view
the output here:

        http://momjian.us/pgsql_docs/release-9-5.html


Thanks Bruce.

Would you also be able to mention something about f15821e and d222585 ?

Regards

David Rowley

--
 David Rowley                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services
 

Re: 9.5 release notes

From
Tomas Vondra
Date:
Hi,

On 06/11/15 06:15, Bruce Momjian wrote:
> I have committed the first draft of the 9.5 release notes.  You can view
> the output here:
>
>     http://momjian.us/pgsql_docs/release-9-5.html
>     
> and it will eventually appear here:
>
>     http://www.postgresql.org/docs/devel/static/release.html
>
> I am ready to make suggested adjustments, though I am traveling for
> conferences for the next ten days so there might a delay in my replies.

I wonder whether this point:

* Improve hash creation performance (Tomas Vondra, Teodor Sigaev, Tom  Lane)

is really about and 45f6240a, 8cce08f1 and 30d7ae3c. I can't remember or 
find other hash-related patches I've been working on.

If that's the case, it really is not about hash creation performance 
(except maybe partially the first commit), but about the lookup 
performance on the hash table. So it should rather be 'improve hash-join 
performance'.

Also, I think the list of authors is wrong. Robert Haas did a lot of 
work on those patches, so he should be mentioned there, and I don't 
remember Teodor working on this.

But maybe I was working on another patch and managed to forget about it.

--
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Jun 11, 2015 at 10:20:13AM +0530, Amit Kapila wrote:
> On Thu, Jun 11, 2015 at 9:45 AM, Bruce Momjian <bruce@momjian.us> wrote:
> >
> > I have committed the first draft of the 9.5 release notes.  You can view
> > the output here:
> >
> >         http://momjian.us/pgsql_docs/release-9-5.html
> >
>
> Thanks for writing the Release notes.
>
> Some comments:
>
> Have pg_basebackup use a tablespace mapping file, to allow for file paths of
> 100+ characters in length
>
> I think this is not completely correct. This is mainly done to allow
> usage of tar format in Windows when tablespaces are present
> in database, although we have eventually done it for both
> Windows and Linux in the same way.  So how about: 
>
> Have pg_basebackup use a tablespace mapping file, to allow usage of tar format
> consistently across all platforms

Actually, the file fixes 100+ char on all platforms _and_ Windows symbolic links:

        Map basebackup tablespaces using a tablespace_map file

        Windows can't reliably restore symbolic links from a tar format, so
        instead during backup start we create a tablespace_map file, which is
        used by the restoring postgres to create the correct links in pg_tblspc.
        The backup protocol also now has an option to request this file to be
        included in the backup stream, and this is used by pg_basebackup when
        operating in tar mode.

        This is done on all platforms, not just Windows.

        This means that pg_basebackup will not not work in tar mode against 9.4
        and older servers, as this protocol option isn't implemented there.

        Amit Kapila, reviewed by Dilip Kumar, with a little editing from me.

    (Andrew Dunstan)
    [72d422a52] 2015-05-12 09:29:10 -0400

        pg_basebackup -F t now succeeds with a long symlink target

        Error when creating names too long for tar format

        The tar format (at least the version we are using), does not support
        file names or symlink targets longer than 99 bytes.  Until now, the tar
        creation code would silently truncate any names that are too long.  (Its
        original application was pg_dump, where this never happens.)  This
        creates problems when running base backups over the replication
        protocol.

        The most important problem is when a tablespace path is longer than 99
        bytes, which will result in a truncated tablespace path being backed up.
        Less importantly, the basebackup protocol also promises to back up any
        other files it happens to find in the data directory, which would also
        lead to file name truncation if someone put a file with a long name in
        there.

        Now both of these cases result in an error during the backup.

        Add tests that fail when a too-long file name or symlink is attempted to
        be backed up.

        Reviewed-by: Robert Hass <robertmhaas@gmail.com>

    (Peter Eisentraut)
    [23a78352c] 2015-02-24 13:41:07 -0500

>
> Also shall we mention about below in Migrations to 9.5 section
>
> "pg_basebackup will not not work in tar mode against 9.4 and older servers,
>  as we have introduced a new protocol option in that mode."

Yes, added.  The attached, applied patch has both of these mentions, and
mentions 'tar' mode.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + Everyone has their own god. +

Attachment

Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Jun 11, 2015 at 01:54:23PM +0900, Michael Paquier wrote:
> On Thu, Jun 11, 2015 at 1:15 PM, Bruce Momjian <bruce@momjian.us> wrote:
> > I have committed the first draft of the 9.5 release notes.  You can view
> > the output here:
> >
> >         http://momjian.us/pgsql_docs/release-9-5.html
> >
> > and it will eventually appear here:
> >
> >         http://www.postgresql.org/docs/devel/static/release.html
> >
> > I am ready to make suggested adjustments, though I am traveling for
> > conferences for the next ten days so there might a delay in my replies.
> 
> Here are some review comments:
> +       <para>
> +        RETURN WHERE
> +       </para>
> What is that?

It says the OID is returned, but where?  I don't see it in psql.

> +       <para>
> +        WHAT IS A STATISTICS SNAPSHOT?
> +       </para>
> +      </listitem>
> It defines the last time when the global statistics file of pgstat has
> been written. Perhaps documentation should be made clearer.

Update with this text:
       This represents the last time the snapshot files was written to       the file system.

> +       <para>
> +        The remote snapshot must have been exported by
> +        <function>pg_export_snapshot()</> or been defined as a
> +        logical replication slot.  This can be used by parallel
> +        <application>pg_dump</> to use a consistent snapshot across
> +        <application>pg_dump</> processes.
> +       </para>
> Perhaps "or been defined when creating a logical replication slot
> through a replication connection".

Sure, updated paragraph:
       The remote snapshot must have been exported by       <function>pg_export_snapshot()</> or been defined when
creating      a logical replication slot.  This can be used by parallel       <application>pg_dump</> to use a
consistentsnapshot across       <application>pg_dump</> processes.
 

> +      <listitem>
> +       <para>
> +        Simplify <acronym>WAL</> record format (Heikki Linnakangas)
> +       </para>
> +
> +       <para>
> +        This allows external tools to more easily process <acronym>WAL</>
> +        files.
> +       </para>
> +      </listitem>
> More precision could be brought here. What the new format allows is
> actually to track more easily what are the blocks modified for
> relations, something possible without having the knowledge of the
> record type directly.

OK, new text:
       This allows external tools to more easily track what blocks       are modified.

> +       <para>
> +        This is particularly helpful for warm standbys.
> +       </para>
> "for warm standbys to control the timing at which WAL segment files
> are retrieved from a WAL archive."

That feels redundant to the major description of the item.

> I think that the following items should be added as well:
> - Improvement of file version information for Windows builds (Noah
> Misch, Michael Paquier), commit ee9569e. The file version information
> was missing for a set of contrib modules as well as a handful of
> libraries and binaries (like conversion_procs, zic, pg_regress, etc.).
> This item should mention that all the binaries and libraries produced
> by a Windows build now contain file version information. This could be
> merged as well with this item as both are related:
> +       <para>
> +        Add icons to all <productname>MSVC</>-built binaries (Noah Misch)
> +       </para>

OK, merged into the existing item:
       Add icons to all <productname>MSVC</>-built binaries and version       information to all <systemitem
class="osname">MSWindows</>       binaries (Noah Misch)
 

> - Move pg_lzcompress and pg_lzdecompress to libpqcommon, commit
> 40bede54. This was some legwork for wal_compression but external
> binary tools can take advantage of using it now more freely. Those
> APIs have been reworked as well to be more generic, somewhat similarly
> to the interface lz4 exposes to the user.

Uh, do we actually want to document that API for users?  I didn't think so.


> - Addition of palloc_extended (8c8a886) to give module developers a
> fallback plan instead of OOM ERROR that palloc throws mandatorily.
> MemoryContextAllocExtended() can be used on another memory context
> than the current one similarly (bd4e2fd9). Feel free to discard this
> one if this is not appropriate in the release notes.

Same question.  I am happy to mention it, but if we mention it, we are
encouraging people to use it.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Jun 11, 2015 at 02:00:08PM +0900, Amit Langote wrote:
> On 2015-06-11 PM 01:15, Bruce Momjian wrote:
> > I have committed the first draft of the 9.5 release notes.  You can view
> > the output here:
> > 
> >     http://momjian.us/pgsql_docs/release-9-5.html
> >     
> > and it will eventually appear here:
> > 
> >     http://www.postgresql.org/docs/devel/static/release.html
> > 
> > I am ready to make suggested adjustments, though I am traveling for
> > conferences for the next ten days so there might a delay in my replies.
> > 
> 
> In the last section E.1.3.11.1. pgbench, there is:
> 
> +      <listitem>
> +       <para>
> +        Add information about buffer pins to <application>pg_buffercache</>
> +        display (Andres Freund)
> +       </para>
> +      </listitem>
> 
> Should be moved its own section?

Oops, wrong section.   Moved.  Thanks.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Jun 11, 2015 at 05:16:07PM +1200, David Rowley wrote:
> 
> On 11 June 2015 at 16:15, Bruce Momjian <bruce@momjian.us> wrote:
> 
>     I have committed the first draft of the 9.5 release notes.  You can view
>     the output here:
> 
>             http://momjian.us/pgsql_docs/release-9-5.html
> 
> 
> 
> Thanks Bruce.
> 
> Would you also be able to mention something about f15821e and d222585 ?

I am going to defer to Tom on that.  I have added optimizer changes in
the past but he didn't feel it made sense unless there was some
user-visible change.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Jun 11, 2015 at 02:16:59PM +0200, Tomas Vondra wrote:
> Hi,
> 
> On 06/11/15 06:15, Bruce Momjian wrote:
> >I have committed the first draft of the 9.5 release notes.  You can view
> >the output here:
> >
> >    http://momjian.us/pgsql_docs/release-9-5.html
> >    
> >and it will eventually appear here:
> >
> >    http://www.postgresql.org/docs/devel/static/release.html
> >
> >I am ready to make suggested adjustments, though I am traveling for
> >conferences for the next ten days so there might a delay in my replies.
> 
> I wonder whether this point:
> 
> * Improve hash creation performance (Tomas Vondra, Teodor Sigaev, Tom
>   Lane)
> 
> is really about and 45f6240a, 8cce08f1 and 30d7ae3c. I can't
> remember or find other hash-related patches I've been working on.
> 
> If that's the case, it really is not about hash creation performance
> (except maybe partially the first commit), but about the lookup
> performance on the hash table. So it should rather be 'improve
> hash-join performance'.

That is a merged item;  here is the other one;
    Improve hash_create's API for selecting simple-binary-key hash functions.    Previously, if you wanted anything
besidesC-string hash keys, you had to    specify a custom hashing function to hash_create().  Nearly all such
callerswere specifying tag_hash or oid_hash; which is tedious, and rather    error-prone, since a caller could easily
missthe opportunity to optimize    by using hash_uint32 when appropriate.  Replace this with a design whereby
callersusing simple binary-data keys just specify HASH_BLOBS and don't    need to mess with specific support functions.
hash_create() itself will    take care of optimizing when the key size is four bytes.    This nets out saving a few
hundredbytes of code space, and offers    a measurable performance improvement in tidbitmap.c (which was not
exploitingthe opportunity to use hash_uint32 for its 4-byte keys).    There might be some wins elsewhere too, I didn't
analyzeclosely.    In future we could look into offering a similar optimized hashing function    for 8-byte keys.
Underthis design that could be done in a centralized    and machine-independent fashion, whereas getting it right for
keysof    platform-dependent sizes would've been notationally painful before.    For the moment, the old way still
worksfine, so as not to break source    code compatibility for loadable modules.  Eventually we might want to    remove
tag_hashand friends from the exported API altogether, since there's    no real need for them to be explicitly
referencedfrom outside dynahash.c.    Teodor Sigaev and Tom Lane(Tom Lane)[4a14f13a0] 2014-12-18 13:36:36 -0500
 


> Also, I think the list of authors is wrong. Robert Haas did a lot of
> work on those patches, so he should be mentioned there, and I don't
> remember Teodor working on this.
> 
> But maybe I was working on another patch and managed to forget about it.

New text:
       Improve hash creation and lookup performance (Tomas Vondra,       Teodor Sigaev, Tom Lane, Robert Haas)

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Robert Haas
Date:
On Thu, Jun 11, 2015 at 11:32 AM, Bruce Momjian <bruce@momjian.us> wrote:
>         Improve hash creation and lookup performance (Tomas Vondra,
>         Teodor Sigaev, Tom Lane, Robert Haas)

I suggest haveing two separate items.  One of those is about the Hash
executor node and the other is about our dynahash stuff.  So they're
completely different code bases.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Jun 11, 2015 at 12:13:26PM -0400, Robert Haas wrote:
> On Thu, Jun 11, 2015 at 11:32 AM, Bruce Momjian <bruce@momjian.us> wrote:
> >         Improve hash creation and lookup performance (Tomas Vondra,
> >         Teodor Sigaev, Tom Lane, Robert Haas)
> 
> I suggest haveing two separate items.  One of those is about the Hash
> executor node and the other is about our dynahash stuff.  So they're
> completely different code bases.

OK, can you give me some text?  Do users really care which part it is
in?

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Tom Lane
Date:
Bruce Momjian <bruce@momjian.us> writes:
> On Thu, Jun 11, 2015 at 05:16:07PM +1200, David Rowley wrote:
>> Would you also be able to mention something about�f15821e and�d222585 ?

> I am going to defer to Tom on that.  I have added optimizer changes in
> the past but he didn't feel it made sense unless there was some
> user-visible change.

I'd be inclined to document both of those.  We mentioned outer join
removal when it was first added, in 9.0, so making a significant
improvement in it seems worthy of mention also.  Both of these things
are "user visible" to the extent that they affect EXPLAIN output.

I'm not sure whether we need to document the semantic hazard that the
second commit message worries about.
        regards, tom lane



Re: 9.5 release notes

From
Tom Lane
Date:
Robert Haas <robertmhaas@gmail.com> writes:
> On Thu, Jun 11, 2015 at 11:32 AM, Bruce Momjian <bruce@momjian.us> wrote:
>> Improve hash creation and lookup performance (Tomas Vondra,
>> Teodor Sigaev, Tom Lane, Robert Haas)

> I suggest haveing two separate items.  One of those is about the Hash
> executor node and the other is about our dynahash stuff.  So they're
> completely different code bases.

As far as 4a14f13a0 goes, I would think that ought to be mentioned under
"Source Code" since it's a change in a rather widely used API.  I doubt
that the performance aspect of it is really all that exciting to end
users, but third-party modules calling the dynahash code would care.
The hash join changes are a completely different thing.
        regards, tom lane



Re: 9.5 release notes

From
Robert Haas
Date:
On Thu, Jun 11, 2015 at 12:14 PM, Bruce Momjian <bruce@momjian.us> wrote:
> On Thu, Jun 11, 2015 at 12:13:26PM -0400, Robert Haas wrote:
>> On Thu, Jun 11, 2015 at 11:32 AM, Bruce Momjian <bruce@momjian.us> wrote:
>> >         Improve hash creation and lookup performance (Tomas Vondra,
>> >         Teodor Sigaev, Tom Lane, Robert Haas)
>>
>> I suggest haveing two separate items.  One of those is about the Hash
>> executor node and the other is about our dynahash stuff.  So they're
>> completely different code bases.
>
> OK, can you give me some text?  Do users really care which part it is
> in?

One item should say something like:

Improve hash join performance.

The Teodor/Tom thing should say something like:

Improve the performance of various in-memory hash tables.  In
particular, this can improve the performance of bitmap index scans.

I assume users would view those as separate things.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: 9.5 release notes

From
Josh Berkus
Date:
On 06/10/2015 09:50 PM, Amit Kapila wrote:
> Also shall we mention about below in Migrations to 9.5 section
> 
> "pg_basebackup will not not work in tar mode against 9.4 and older servers,
>  as we have introduced a new protocol option in that mode."

AFAIK, pg_basebackup has never worked across versions.  So there's no
reason for this note.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com



Re: 9.5 release notes

From
Magnus Hagander
Date:
On Thu, Jun 11, 2015 at 8:56 PM, Josh Berkus <josh@agliodbs.com> wrote:
On 06/10/2015 09:50 PM, Amit Kapila wrote:
> Also shall we mention about below in Migrations to 9.5 section
>
> "pg_basebackup will not not work in tar mode against 9.4 and older servers,
>  as we have introduced a new protocol option in that mode."

AFAIK, pg_basebackup has never worked across versions.  So there's no
reason for this note.

It has. The resulting backup has not been usable cross version, but pg_basebackup itself has. Though not always, and I'm not sure we've ever claimed it was supported, but it has worked. 


--

Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Wed, Jun 10, 2015 at 9:15 PM, Bruce Momjian <bruce@momjian.us> wrote:
> I have committed the first draft of the 9.5 release notes.  You can view
> the output here:

+      <listitem>
+       <para>
+        Improve the speed of sorting character and numeric fields (Robert
+        Haas, Peter Geoghegan, Andrew Gierth)
+       </para>
+      </listitem>

A few comments on this.

First of all, I think it should be separately noted that the
sortsupport infrastructure is now used in virtually all places where
it's useful (see commit 5ea86e6e6). So for example, CREATE INDEX on
integer columns ought to be notably faster (and CLUSTER, too). The 9.2
era sortsupport stuff was simply never adopted to do that until now.
That has nothing to do with abbreviated keys, except that the idea of
abbreviated keys gave me a strong reason to care about sortsupport a
lot more. But commit 5ea86e6e6 predates abbreviated keys, and is
certainly independently useful (this really should have made it into
9.2).

Secondly, Robert didn't credit himself as an author in his commit
message for the abbreviated keys infrastructure + text opclass support
*at all*. However, I think that Robert should be listed as a secondary
author of the abbreviated keys infrastructure, and that he would agree
that I am clearly the primary author. Andrew Gierth did work on the
datum case for sortsupport + abbreviation, so I agree he should be
listed as a secondary author of the infrastructure too, after Robert.

I think there should be a total of 4 items related to sorting. The
wording I come up with may not be appropriate, but will give you an
idea:

* Allow sorting to be performed by inlined, non-SQL-callable
comparison functions for CREATE INDEX, REINDEX and CLUSTER operations
based on a B-Tree operator class. (5ea86e6e6 -- Geoghegan)

* Add abbreviated key sorting infrastructure. This allows B-Tree
operator classes to provide compact abbreviated representations of
pass-by-reference types which are sorted with inexpensive comparisons.
This makes sort operations with support for the infrastructure very
significantly faster in the common case where most comparisons can be
resolved with the abbreviated representation alone. (4ea51cdfe85 --
Geoghegan, Haas, Gierth, with Gierth's contribution coming from
78efd5c1 alone)

* Add sortsupport (support for non-SQL callable interface for
comparators) with abbreviation capability to text/varlena operator
class. This significantly accelerates sorting on text columns.
(4ea51cdfe85 too, but also b34e37bf. Worth noting separately IMV.
Geoghegan, Haas).

* Add sortsupport (support for non-SQL callable interface for
comparators) with abbreviation capability to numeric operator class.
This significantly accelerates sorting on numeric columns. (abd94bcac,
Gierth)

I'm not sure if it's worth mentioning the "cheap equality for text"
commit (e246b3d6eac09). I guess that it probably is, because it will
help with things like index scans, too. Arguably that isn't a sorting
thing (it's certainly not *just* a sorting thing).

I've blogged on the abbreviated key stuff quite a bit, which may be
useful should you require additional background information:

http://pgeoghegan.blogspot.com/2015/01/abbreviated-keys-exploiting-locality-to.html

http://pgeoghegan.blogspot.com/2015/04/abbreviated-keys-for-numeric-to.html

Thanks
-- 
Peter Geoghegan



Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Thu, Jun 11, 2015 at 1:23 PM, Peter Geoghegan <pg@heroku.com> wrote:
> * Add sortsupport (support for non-SQL callable interface for
> comparators) with abbreviation capability to text/varlena operator
> class. This significantly accelerates sorting on text columns.
> (4ea51cdfe85 too, but also b34e37bf. Worth noting separately IMV.
> Geoghegan, Haas).

Also, note that there is no character(n) support for abbreviation as
yet (nor is there any contrib/citext support), so it's not quite true
that the sorting stuff accelerates sorts on "character fields".

-- 
Peter Geoghegan



Re: 9.5 release notes

From
Robert Haas
Date:
On Thu, Jun 11, 2015 at 4:23 PM, Peter Geoghegan <pg@heroku.com> wrote:
> Secondly, Robert didn't credit himself as an author in his commit
> message for the abbreviated keys infrastructure + text opclass support
> *at all*. However, I think that Robert should be listed as a secondary
> author of the abbreviated keys infrastructure, and that he would agree
> that I am clearly the primary author. Andrew Gierth did work on the
> datum case for sortsupport + abbreviation, so I agree he should be
> listed as a secondary author of the infrastructure too, after Robert.

I'd probably say Peter, Andrew, me.

> I think there should be a total of 4 items related to sorting. The
> wording I come up with may not be appropriate, but will give you an
> idea:
>
> * Allow sorting to be performed by inlined, non-SQL-callable
> comparison functions for CREATE INDEX, REINDEX and CLUSTER operations
> based on a B-Tree operator class. (5ea86e6e6 -- Geoghegan)
>
> * Add abbreviated key sorting infrastructure. This allows B-Tree
> operator classes to provide compact abbreviated representations of
> pass-by-reference types which are sorted with inexpensive comparisons.
> This makes sort operations with support for the infrastructure very
> significantly faster in the common case where most comparisons can be
> resolved with the abbreviated representation alone. (4ea51cdfe85 --
> Geoghegan, Haas, Gierth, with Gierth's contribution coming from
> 78efd5c1 alone)
>
> * Add sortsupport (support for non-SQL callable interface for
> comparators) with abbreviation capability to text/varlena operator
> class. This significantly accelerates sorting on text columns.
> (4ea51cdfe85 too, but also b34e37bf. Worth noting separately IMV.
> Geoghegan, Haas).
>
> * Add sortsupport (support for non-SQL callable interface for
> comparators) with abbreviation capability to numeric operator class.
> This significantly accelerates sorting on numeric columns. (abd94bcac,
> Gierth)

I think this is overkill.  Bruce's single entry seems fine to me,
although we could adjust the wording a bit.  We need this to be simple
enough that someone not immersed in the code can understand it, and as
soon as you talk about sortsupport as if it were a thing, you lose
that.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Thu, Jun 11, 2015 at 2:17 PM, Robert Haas <robertmhaas@gmail.com> wrote:
> On Thu, Jun 11, 2015 at 4:23 PM, Peter Geoghegan <pg@heroku.com> wrote:
>> Secondly, Robert didn't credit himself as an author in his commit
>> message for the abbreviated keys infrastructure + text opclass support
>> *at all*. However, I think that Robert should be listed as a secondary
>> author of the abbreviated keys infrastructure, and that he would agree
>> that I am clearly the primary author. Andrew Gierth did work on the
>> datum case for sortsupport + abbreviation, so I agree he should be
>> listed as a secondary author of the infrastructure too, after Robert.
>
> I'd probably say Peter, Andrew, me.

If you bunch everything together, then yes, I'd agree.

>> I think there should be a total of 4 items related to sorting. The
>> wording I come up with may not be appropriate, but will give you an
>> idea:
>>
>> * Allow sorting to be performed by inlined, non-SQL-callable
>> comparison functions for CREATE INDEX, REINDEX and CLUSTER operations
>> based on a B-Tree operator class. (5ea86e6e6 -- Geoghegan)
>>
>> * Add abbreviated key sorting infrastructure. This allows B-Tree
>> operator classes to provide compact abbreviated representations of
>> pass-by-reference types which are sorted with inexpensive comparisons.
>> This makes sort operations with support for the infrastructure very
>> significantly faster in the common case where most comparisons can be
>> resolved with the abbreviated representation alone. (4ea51cdfe85 --
>> Geoghegan, Haas, Gierth, with Gierth's contribution coming from
>> 78efd5c1 alone)
>>
>> * Add sortsupport (support for non-SQL callable interface for
>> comparators) with abbreviation capability to text/varlena operator
>> class. This significantly accelerates sorting on text columns.
>> (4ea51cdfe85 too, but also b34e37bf. Worth noting separately IMV.
>> Geoghegan, Haas).
>>
>> * Add sortsupport (support for non-SQL callable interface for
>> comparators) with abbreviation capability to numeric operator class.
>> This significantly accelerates sorting on numeric columns. (abd94bcac,
>> Gierth)
>
> I think this is overkill.  Bruce's single entry seems fine to me,
> although we could adjust the wording a bit.  We need this to be simple
> enough that someone not immersed in the code can understand it, and as
> soon as you talk about sortsupport as if it were a thing, you lose
> that.

I think that that is a valid point of view, but that we should get
across that sortsupport works for all types with CREATE INDEX +
CLUSTER now, including integers, for example. The reviewer of that
particular piece of work, Andreas Karlsson, put it at a 10% - 15%
improvement for CREATE INDEX on an integer column IIRC. That's not a
bad little boost.

-- 
Peter Geoghegan



Re: 9.5 release notes

From
Amit Kapila
Date:
On Thu, Jun 11, 2015 at 8:24 PM, Bruce Momjian <bruce@momjian.us> wrote:
>
> On Thu, Jun 11, 2015 at 10:20:13AM +0530, Amit Kapila wrote:
> > On Thu, Jun 11, 2015 at 9:45 AM, Bruce Momjian <bruce@momjian.us> wrote:
> > >
> > > I have committed the first draft of the 9.5 release notes.  You can view
> > > the output here:
> > >
> > >         http://momjian.us/pgsql_docs/release-9-5.html
> > >
> >
> > Thanks for writing the Release notes.
> >
> > Some comments:
> >
> > Have pg_basebackup use a tablespace mapping file, to allow for file paths of
> > 100+ characters in length
> >
> >
> > Also shall we mention about below in Migrations to 9.5 section
> >
> > "pg_basebackup will not not work in tar mode against 9.4 and older servers,
> >  as we have introduced a new protocol option in that mode."
>
> Yes, added.  The attached, applied patch has both of these mentions, and
> mentions 'tar' mode.
>

Attached patch makes sense.  Thanks.


With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Re: 9.5 release notes

From
Fujii Masao
Date:
On Thu, Jun 11, 2015 at 1:15 PM, Bruce Momjian <bruce@momjian.us> wrote:
> I have committed the first draft of the 9.5 release notes.  You can view
> the output here:
>
>         http://momjian.us/pgsql_docs/release-9-5.html
>
> and it will eventually appear here:
>
>         http://www.postgresql.org/docs/devel/static/release.html

I found some minor issues.
       e.g. <literal>IDENTIFY_COMMAND</>, are not logged, even when       <varname>log_statements</> is set to
<literal>all</>.

Typos.
s/IDENTIFY_COMMAND/IDENTIFY_SYSTEM
s/log_statements/log_statement

      <para>       RETURN WHERE      </para>

Looks like garbage.

  Add <literal>VERBOSE</> option to <command>REINDEX</> (Fujii Masao)

Could you change the author name to Sawada Masahiko?
He is the author of the feature.

Regards,

-- 
Fujii Masao



Re: 9.5 release notes

From
Petr Jelinek
Date:
Hi,

+      <listitem>
+       <para>
+        Add <type>JSONB</> functions <function>jsonb_set()</> and
+        <function>jsonb_pretty</> (Dmitry Dolgov, Andrew Dunstan)
+       </para>
+      </listitem>

I believe I should be 3rd author for this one as I rewrote large parts 
of this functionality as part of the review.


-- Petr Jelinek                  http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services




Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Jun 11, 2015 at 01:27:23PM -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > On Thu, Jun 11, 2015 at 05:16:07PM +1200, David Rowley wrote:
> >> Would you also be able to mention something about�f15821e and�d222585 ?
>
> > I am going to defer to Tom on that.  I have added optimizer changes in
> > the past but he didn't feel it made sense unless there was some
> > user-visible change.
>
> I'd be inclined to document both of those.  We mentioned outer join
> removal when it was first added, in 9.0, so making a significant
> improvement in it seems worthy of mention also.  Both of these things
> are "user visible" to the extent that they affect EXPLAIN output.
>
> I'm not sure whether we need to document the semantic hazard that the
> second commit message worries about.

OK, I have added two items;  applied patch attached.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + Everyone has their own god. +

Attachment

Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Jun 11, 2015 at 01:31:01PM -0400, Tom Lane wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
> > On Thu, Jun 11, 2015 at 11:32 AM, Bruce Momjian <bruce@momjian.us> wrote:
> >> Improve hash creation and lookup performance (Tomas Vondra,
> >> Teodor Sigaev, Tom Lane, Robert Haas)
>
> > I suggest haveing two separate items.  One of those is about the Hash
> > executor node and the other is about our dynahash stuff.  So they're
> > completely different code bases.
>
> As far as 4a14f13a0 goes, I would think that ought to be mentioned under
> "Source Code" since it's a change in a rather widely used API.  I doubt
> that the performance aspect of it is really all that exciting to end
> users, but third-party modules calling the dynahash code would care.
> The hash join changes are a completely different thing.

Applied patch attached.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + Everyone has their own god. +

Attachment

Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Jun 11, 2015 at 09:02:35PM +0200, Magnus Hagander wrote:
> On Thu, Jun 11, 2015 at 8:56 PM, Josh Berkus <josh@agliodbs.com> wrote:
> 
>     On 06/10/2015 09:50 PM, Amit Kapila wrote:
>     > Also shall we mention about below in Migrations to 9.5 section
>     >
>     > "pg_basebackup will not not work in tar mode against 9.4 and older
>     servers,
>     >  as we have introduced a new protocol option in that mode."
> 
>     AFAIK, pg_basebackup has never worked across versions.  So there's no
>     reason for this note.
> 
> 
> It has. The resulting backup has not been usable cross version, but
> pg_basebackup itself has. Though not always, and I'm not sure we've ever
> claimed it was supported, but it has worked. 

So we should still mention it in the release notes?

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Jun 11, 2015 at 05:17:54PM -0400, Robert Haas wrote:
> On Thu, Jun 11, 2015 at 4:23 PM, Peter Geoghegan <pg@heroku.com> wrote:
> > Secondly, Robert didn't credit himself as an author in his commit
> > message for the abbreviated keys infrastructure + text opclass support
> > *at all*. However, I think that Robert should be listed as a secondary
> > author of the abbreviated keys infrastructure, and that he would agree
> > that I am clearly the primary author. Andrew Gierth did work on the
> > datum case for sortsupport + abbreviation, so I agree he should be
> > listed as a secondary author of the infrastructure too, after Robert.
> 
> I'd probably say Peter, Andrew, me.

Order changed, thanks.  This entry was a consolidation of several
commits so the proper order wasn't clear to me.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Fri, Jun 12, 2015 at 12:49:11PM +0900, Fujii Masao wrote:
> On Thu, Jun 11, 2015 at 1:15 PM, Bruce Momjian <bruce@momjian.us> wrote:
> > I have committed the first draft of the 9.5 release notes.  You can view
> > the output here:
> >
> >         http://momjian.us/pgsql_docs/release-9-5.html
> >
> > and it will eventually appear here:
> >
> >         http://www.postgresql.org/docs/devel/static/release.html
> 
> I found some minor issues.
> 
>         e.g. <literal>IDENTIFY_COMMAND</>, are not logged, even when
>         <varname>log_statements</> is set to <literal>all</>.
> 
> Typos.
> s/IDENTIFY_COMMAND/IDENTIFY_SYSTEM
> s/log_statements/log_statement

Updated.

>         RETURN WHERE
>        </para>
> 
> Looks like garbage.

It is actually a question;  I have reworded it.

>    Add <literal>VERBOSE</> option to <command>REINDEX</> (Fujii Masao)
> 
> Could you change the author name to Sawada Masahiko?
> He is the author of the feature.

Thanks, done.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Fri, Jun 12, 2015 at 02:47:22PM +0200, Petr Jelinek wrote:
> Hi,
> 
> +      <listitem>
> +       <para>
> +        Add <type>JSONB</> functions <function>jsonb_set()</> and
> +        <function>jsonb_pretty</> (Dmitry Dolgov, Andrew Dunstan)
> +       </para>
> +      </listitem>
> 
> I believe I should be 3rd author for this one as I rewrote large
> parts of this functionality as part of the review.

Added.  Thanks.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Amit Kapila
Date:
On Sat, Jun 13, 2015 at 7:47 AM, Bruce Momjian <bruce@momjian.us> wrote:
>
> On Thu, Jun 11, 2015 at 09:02:35PM +0200, Magnus Hagander wrote:
> > On Thu, Jun 11, 2015 at 8:56 PM, Josh Berkus <josh@agliodbs.com> wrote:
> >
> >     On 06/10/2015 09:50 PM, Amit Kapila wrote:
> >     > Also shall we mention about below in Migrations to 9.5 section
> >     >
> >     > "pg_basebackup will not not work in tar mode against 9.4 and older
> >     servers,
> >     >  as we have introduced a new protocol option in that mode."
> >
> >     AFAIK, pg_basebackup has never worked across versions.  So there's no
> >     reason for this note.
> >
> >
> > It has. The resulting backup has not been usable cross version, but
> > pg_basebackup itself has. Though not always, and I'm not sure we've ever
> > claimed it was supported, but it has worked.
>
> So we should still mention it in the release notes?
>

If it has never lead to usable backup's for cross version backup, then I think
there is no pressing need to mention it.


With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Re: 9.5 release notes

From
Alvaro Herrera
Date:
Bruce Momjian wrote:
> I have committed the first draft of the 9.5 release notes.  You can view
> the output here:
> 
>     http://momjian.us/pgsql_docs/release-9-5.html
>     
> and it will eventually appear here:
> 
>     http://www.postgresql.org/docs/devel/static/release.html
> 
> I am ready to make suggested adjustments, though I am traveling for
> conferences for the next ten days so there might a delay in my replies.

Many people have commented on the "RETURN WHERE" text.  The item in
question is this:Have CREATE TABLE AS and REFRESH return an OID

This code was later changed to return ObjectAddress rather than OID;
this is used to feed event triggers.  To answer the "where" question:
the OID is returned to ProcessUtilitySlow, where it can be used by the
DDL-command-collection infrastructure.

I don't think this particular change needs a mention in the release
notes, since the ddl command collection is mentioned separately.  I
would just remove it.


In the pg_upgrade section, I think this is a bit strangely worded:Remove pg_upgrade support for 8.3 old clusters (Bruce
Momjian)
How about this?Remove support for upgrading from 8.3 clusters (Bruce Momjian)

Why do we need a separate mention that pg_upgrade_support was moved to
backend in the "Additional Modules" section?  Seems to me that having
it mentioned in the pg_upgrade section would be better.


This entry is missing a word at the end:Allow CURRENT/SESSION_USER to specify the current user incertain

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Re: 9.5 release notes

From
Alvaro Herrera
Date:
Petr Jelinek wrote:
> Hi,
> 
> +      <listitem>
> +       <para>
> +        Add <type>JSONB</> functions <function>jsonb_set()</> and
> +        <function>jsonb_pretty</> (Dmitry Dolgov, Andrew Dunstan)
> +       </para>
> +      </listitem>
> 
> I believe I should be 3rd author for this one as I rewrote large parts of
> this functionality as part of the review.

Also, it looks like we could spell your last name with an accent,
assuming the i-acute character in Latin1 is fine.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Thu, Jun 11, 2015 at 1:23 PM, Peter Geoghegan <pg@heroku.com> wrote:
> I'm not sure if it's worth mentioning the "cheap equality for text"
> commit (e246b3d6eac09). I guess that it probably is, because it will
> help with things like index scans, too. Arguably that isn't a sorting
> thing (it's certainly not *just* a sorting thing).

I think we should really address this. Attached patch adds a new
release note item for it. It also adds to the documentation that
explains why users should prefer varchar(n)/text to character(n); the
lack of abbreviated key support now becomes a huge disadvantage for
character(n), whereas in previous versions the disadvantages were
fairly minor.

In passing, I updated the existing sort item to reflect that only
varchar(n), text, and numeric benefit from the abbreviation
optimization (not character types more generally + numeric), and added
a note on the effectiveness of the abbreviation optimization alone.

--
Peter Geoghegan

Attachment

Re: 9.5 release notes

From
Alvaro Herrera
Date:
Bruce Momjian wrote:
> I have committed the first draft of the 9.5 release notes.  You can view
> the output here:
> 
>     http://momjian.us/pgsql_docs/release-9-5.html

I noticed something while reading this and would like the input of our
Japanese contributors.

Normally, western names are written using the given name first, then the
surname ("last name").  I am not well-versed on Japanese culture but my
understanding is that they use their family name first, then their given
name -- for instance, in "Yamada Tarō", Yamada is the family name and
Tarō is the given name.

Apparently, when interacting with Western people, some Japanese seem to
invert this and put the given name first.  Also, some write the family
name in all caps -- I guess this is done so that it is clear which part
is which.

Now, I think we should consider using a single style for all the
Japanese names used in the release notes.  Can we have our Japanese
contributors all agree on which style to use, and then let us know
what's their name in that style?

The names we currently have in the 9.5 release notes are:

Fujii Masao
Kyotaro Horiguchi
Sawada Masahiko
KaiGai Kohei
Shigeru Hanada
Etsuro Fujita
Furuya Osamu
MauMau
Mitsumasa Kondo


BTW, it is pretty cool to have contributor names that are natively in
scripts other than latin.  It is a pity that because of toolchain
limitations we cannot display names in kanji, cyrillic or other
character sets (in addition to their transliteration to latin script).


-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Re: 9.5 release notes

From
Alvaro Herrera
Date:
Bruce Momjian wrote:
> I have committed the first draft of the 9.5 release notes.  You can view
> the output here:
> 
>     http://momjian.us/pgsql_docs/release-9-5.html

One more thing on names.  You have Alexander Shulgin as Alex Shulgin in
the second place his name appears.  Also, since he moved to Zalando (I
guess to Berlin) he writes his name as Oleksandr.  I guess he'd have to
tell us which forms he would like to be used in the release notes.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Sat, Jun 13, 2015 at 08:25:55AM +0530, Amit Kapila wrote:
> On Sat, Jun 13, 2015 at 7:47 AM, Bruce Momjian <bruce@momjian.us> wrote:
> >
> > On Thu, Jun 11, 2015 at 09:02:35PM +0200, Magnus Hagander wrote:
> > > On Thu, Jun 11, 2015 at 8:56 PM, Josh Berkus <josh@agliodbs.com> wrote:
> > >
> > >     On 06/10/2015 09:50 PM, Amit Kapila wrote:
> > >     > Also shall we mention about below in Migrations to 9.5 section
> > >     >
> > >     > "pg_basebackup will not not work in tar mode against 9.4 and older
> > >     servers,
> > >     >  as we have introduced a new protocol option in that mode."
> > >
> > >     AFAIK, pg_basebackup has never worked across versions.  So there's no
> > >     reason for this note.
> > >
> > >
> > > It has. The resulting backup has not been usable cross version, but
> > > pg_basebackup itself has. Though not always, and I'm not sure we've ever
> > > claimed it was supported, but it has worked.
> >
> > So we should still mention it in the release notes?
> >
> 
> If it has never lead to usable backup's for cross version backup, then I think
> there is no pressing need to mention it.

Removed.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Sat, Jun 13, 2015 at 05:47:59AM -0300, Alvaro Herrera wrote:
> Petr Jelinek wrote:
> > Hi,
> > 
> > +      <listitem>
> > +       <para>
> > +        Add <type>JSONB</> functions <function>jsonb_set()</> and
> > +        <function>jsonb_pretty</> (Dmitry Dolgov, Andrew Dunstan)
> > +       </para>
> > +      </listitem>
> > 
> > I believe I should be 3rd author for this one as I rewrote large parts of
> > this functionality as part of the review.
> 
> Also, it looks like we could spell your last name with an accent,
> assuming the i-acute character in Latin1 is fine.

Ah, you are correct.  I found a few commits that did have that accent. 
All _seven_ of Petr's 9.5 commit mentions updated to add the accent. 
:-)  Thanks.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Sat, Jun 13, 2015 at 07:52:22AM -0300, Alvaro Herrera wrote:
> Bruce Momjian wrote:
> > I have committed the first draft of the 9.5 release notes.  You can view
> > the output here:
> >
> >     http://momjian.us/pgsql_docs/release-9-5.html
> >
> > and it will eventually appear here:
> >
> >     http://www.postgresql.org/docs/devel/static/release.html
> >
> > I am ready to make suggested adjustments, though I am traveling for
> > conferences for the next ten days so there might a delay in my replies.
>
> Many people have commented on the "RETURN WHERE" text.  The item in
> question is this:
>     Have CREATE TABLE AS and REFRESH return an OID
>
> This code was later changed to return ObjectAddress rather than OID;
> this is used to feed event triggers.  To answer the "where" question:
> the OID is returned to ProcessUtilitySlow, where it can be used by the
> DDL-command-collection infrastructure.
>
> I don't think this particular change needs a mention in the release
> notes, since the ddl command collection is mentioned separately.  I
> would just remove it.

OK, removed.

> In the pg_upgrade section, I think this is a bit strangely worded:
>     Remove pg_upgrade support for 8.3 old clusters (Bruce Momjian)
> How about this?
>     Remove support for upgrading from 8.3 clusters (Bruce Momjian)
>
> Why do we need a separate mention that pg_upgrade_support was moved to
> backend in the "Additional Modules" section?  Seems to me that having
> it mentioned in the pg_upgrade section would be better.

OK, moved.

> This entry is missing a word at the end:
>     Allow CURRENT/SESSION_USER to specify the current user in
>     certain

OK, wording updated, applied patch attached.

Thanks for the review.
--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + Everyone has their own god. +

Attachment

Re: 9.5 release notes

From
Bruce Momjian
Date:
On Sat, Jun 13, 2015 at 08:20:48PM -0300, Alvaro Herrera wrote:
> Bruce Momjian wrote:
> > I have committed the first draft of the 9.5 release notes.  You can view
> > the output here:
> > 
> >     http://momjian.us/pgsql_docs/release-9-5.html
> 
> One more thing on names.  You have Alexander Shulgin as Alex Shulgin in
> the second place his name appears.  Also, since he moved to Zalando (I

OK, good catch.  Fixed as "Alexander".

> guess to Berlin) he writes his name as Oleksandr.  I guess he'd have to
> tell us which forms he would like to be used in the release notes.

Yes.  Does anyone have a current email address for him?  I assume his
Command Prompt email address doesn't work, but I will try it here.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Sat, Jun 13, 2015 at 03:53:43PM -0700, Peter Geoghegan wrote:
> On Thu, Jun 11, 2015 at 1:23 PM, Peter Geoghegan <pg@heroku.com> wrote:
> > I'm not sure if it's worth mentioning the "cheap equality for text"
> > commit (e246b3d6eac09). I guess that it probably is, because it will
> > help with things like index scans, too. Arguably that isn't a sorting
> > thing (it's certainly not *just* a sorting thing).
> 
> I think we should really address this. Attached patch adds a new
> release note item for it. It also adds to the documentation that
> explains why users should prefer varchar(n)/text to character(n); the
> lack of abbreviated key support now becomes a huge disadvantage for
> character(n), whereas in previous versions the disadvantages were
> fairly minor.
> 
> In passing, I updated the existing sort item to reflect that only
> varchar(n), text, and numeric benefit from the abbreviation
> optimization (not character types more generally + numeric), and added
> a note on the effectiveness of the abbreviation optimization alone.

This all also seems like overkill to me.  Users just don't care about
this level of detail, and are easily confused by it.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Sat, Jun 13, 2015 at 08:18:26PM -0300, Alvaro Herrera wrote:
> Bruce Momjian wrote:
> > I have committed the first draft of the 9.5 release notes.  You can view
> > the output here:
> > 
> >     http://momjian.us/pgsql_docs/release-9-5.html
> 
> I noticed something while reading this and would like the input of our
> Japanese contributors.
> 
> Normally, western names are written using the given name first, then the
> surname ("last name").  I am not well-versed on Japanese culture but my
> understanding is that they use their family name first, then their given
> name -- for instance, in "Yamada Tarō", Yamada is the family name and
> Tarō is the given name.
> 
> Apparently, when interacting with Western people, some Japanese seem to
> invert this and put the given name first.  Also, some write the family
> name in all caps -- I guess this is done so that it is clear which part
> is which.

Yes, I have Western-ordered all of them as best I could.

> Now, I think we should consider using a single style for all the
> Japanese names used in the release notes.  Can we have our Japanese
> contributors all agree on which style to use, and then let us know
> what's their name in that style?
> 
> The names we currently have in the 9.5 release notes are:
> 
> Fujii Masao
> Kyotaro Horiguchi
> Sawada Masahiko
> KaiGai Kohei
> Shigeru Hanada
> Etsuro Fujita
> Furuya Osamu
> MauMau
> Mitsumasa Kondo

Yes, it would be good to know this.  The only problem is that I would
have to _guess_ who is Japanese to do this consistently as the supplied
names have varied orderings.  Do we really want to go there?  Are there
other countries where this would be appropriate?

> BTW, it is pretty cool to have contributor names that are natively in
> scripts other than latin.  It is a pity that because of toolchain
> limitations we cannot display names in kanji, cyrillic or other
> character sets (in addition to their transliteration to latin script).

Yes, it would be cool to have the non-Latin name lettering for all
submitters, then a Latin-ized version.  (I guess having my name in
Armenian letters would be overkill ... hmmm.)  Anyway, Alvaro is right
that our tooling doesn't have the capabilities to do this.  We could
link to a webpage that did list all contributors with their native
character-set names though.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Alvaro Herrera
Date:
Bruce Momjian wrote:
> On Sat, Jun 13, 2015 at 08:20:48PM -0300, Alvaro Herrera wrote:
> > Bruce Momjian wrote:
> > > I have committed the first draft of the 9.5 release notes.  You can view
> > > the output here:
> > > 
> > >     http://momjian.us/pgsql_docs/release-9-5.html
> > 
> > One more thing on names.  You have Alexander Shulgin as Alex Shulgin in
> > the second place his name appears.  Also, since he moved to Zalando (I
> 
> OK, good catch.  Fixed as "Alexander".
> 
> > guess to Berlin) he writes his name as Oleksandr.  I guess he'd have to
> > tell us which forms he would like to be used in the release notes.
> 
> Yes.  Does anyone have a current email address for him?  I assume his
> Command Prompt email address doesn't work, but I will try it here.

Oleksandr Shulgin <oleksandr.shulgin@zalando.de>

See "PS" note near the bottom of
https://www.postgresql.org/message-id/CACACo5TjBAHYRJJc-U788DnfXbbPasX_nGUHFd%2B%3DcehA6SRs-w%40mail.gmail.com

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Re: 9.5 release notes

From
Kouhei Kaigai
Date:
> On Sat, Jun 13, 2015 at 08:18:26PM -0300, Alvaro Herrera wrote:
> > Bruce Momjian wrote:
> > > I have committed the first draft of the 9.5 release notes.  You can view
> > > the output here:
> > >
> > >     http://momjian.us/pgsql_docs/release-9-5.html
> >
> > I noticed something while reading this and would like the input of our
> > Japanese contributors.
> >
> > Normally, western names are written using the given name first, then the
> > surname ("last name").  I am not well-versed on Japanese culture but my
> > understanding is that they use their family name first, then their given
> > name -- for instance, in "Yamada Tarō", Yamada is the family name and
> > Tarō is the given name.
> >
> > Apparently, when interacting with Western people, some Japanese seem to
> > invert this and put the given name first.  Also, some write the family
> > name in all caps -- I guess this is done so that it is clear which part
> > is which.
> 
> Yes, I have Western-ordered all of them as best I could.
> 
> > Now, I think we should consider using a single style for all the
> > Japanese names used in the release notes.  Can we have our Japanese
> > contributors all agree on which style to use, and then let us know
> > what's their name in that style?
> >
> > The names we currently have in the 9.5 release notes are:
> >
> > Fujii Masao
> > Kyotaro Horiguchi
> > Sawada Masahiko
> > KaiGai Kohei
> > Shigeru Hanada
> > Etsuro Fujita
> > Furuya Osamu
> > MauMau
> > Mitsumasa Kondo
> 
> Yes, it would be good to know this.  The only problem is that I would
> have to _guess_ who is Japanese to do this consistently as the supplied
> names have varied orderings.  Do we really want to go there?  Are there
> other countries where this would be appropriate?
>
> > BTW, it is pretty cool to have contributor names that are natively in
> > scripts other than latin.  It is a pity that because of toolchain
> > limitations we cannot display names in kanji, cyrillic or other
> > character sets (in addition to their transliteration to latin script).
> 
> Yes, it would be cool to have the non-Latin name lettering for all
> submitters, then a Latin-ized version.  (I guess having my name in
> Armenian letters would be overkill ... hmmm.)  Anyway, Alvaro is right
> that our tooling doesn't have the capabilities to do this.  We could
> link to a webpage that did list all contributors with their native
> character-set names though.
>
I usually use the Eastern-style. KaiGai Kohei = <family name> <given name>.
However, other people use their own manner.
Probably, it is not obvious for foreign people, and me also.
So, I don't think community need to take care.
How about a simply policy to show the name when contributors submitted
and discussed? If people prefer the Latin-style, they put their first
name first. Elsewhere, they put their family name first.

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: 9.5 release notes

From
Jan de Visser
Date:
On June 13, 2015 09:30:12 PM Bruce Momjian wrote:
> Are there other countries where this would be appropriate?

I'm pretty sure Hungarian uses the family name-given name ordering, and it's 
also sometimes used in French, and therefore you often see French family names 
spelled in all caps. 




Re: 9.5 release notes

From
Tatsuo Ishii
Date:
> Bruce Momjian wrote:
>> I have committed the first draft of the 9.5 release notes.  You can view
>> the output here:
>>
>>     http://momjian.us/pgsql_docs/release-9-5.html
>
> I noticed something while reading this and would like the input of our
> Japanese contributors.
>
> Normally, western names are written using the given name first, then the
> surname ("last name").  I am not well-versed on Japanese culture but my
> understanding is that they use their family name first, then their given
> name -- for instance, in "Yamada Tarō", Yamada is the family name and
> Tarō is the given name.
>
> Apparently, when interacting with Western people, some Japanese seem to
> invert this and put the given name first.  Also, some write the family
> name in all caps -- I guess this is done so that it is clear which part
> is which.
>
> Now, I think we should consider using a single style for all the
> Japanese names used in the release notes.  Can we have our Japanese
> contributors all agree on which style to use, and then let us know
> what's their name in that style?

The real question here is, if contributors are required to show their
"official" names (along with their alias or nickname if any).  If yes,
it is apparent that all contributor names should be shown in a unified
formal style.

If not, we could leave as it is.

BTW, for Japanese almost 99% of Japanese names are obvious in that
which is the first name and which is the last name. So we don't need
to ask each contributor.

> The names we currently have in the 9.5 release notes are:
>
> Fujii Masao
> Kyotaro Horiguchi
> Sawada Masahiko
> KaiGai Kohei
> Shigeru Hanada
> Etsuro Fujita
> Furuya Osamu
> MauMau
> Mitsumasa Kondo

F.Y.I. here is a list rewritten in first-name/last-name style:

Masao Fujii
Kyotaro Horiguchi
Masahiko Sawada
Kohei KaiGai
Shigeru Hanada
Etsuro Fujita
Osamu Furuya
Mitsumasa Kondo
MauMau

Note that I guess "MauMau" is a nickname.

> BTW, it is pretty cool to have contributor names that are natively in
> scripts other than latin.  It is a pity that because of toolchain
> limitations we cannot display names in kanji, cyrillic or other
> character sets (in addition to their transliteration to latin script).

Japan User's Group (JPUG) recently starts to translate PostgreSQL doc
using UTF-8 encoding. So technically this is possible. I'm not sure if
developers want to have official document in other than ASCII though.

Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese:http://www.sraoss.co.jp



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Sun, Jun 14, 2015 at 11:44:25AM +0900, Tatsuo Ishii wrote:
> The real question here is, if contributors are required to show their
> "official" names (along with their alias or nickname if any).  If yes,
> it is apparent that all contributor names should be shown in a unified
> formal style.
> 
> If not, we could leave as it is.
> 
> BTW, for Japanese almost 99% of Japanese names are obvious in that
> which is the first name and which is the last name. So we don't need
> to ask each contributor.

OK, good.

> > The names we currently have in the 9.5 release notes are:
> > 
> > Fujii Masao
> > Kyotaro Horiguchi
> > Sawada Masahiko
> > KaiGai Kohei
> > Shigeru Hanada
> > Etsuro Fujita
> > Furuya Osamu
> > MauMau
> > Mitsumasa Kondo
> 
> F.Y.I. here is a list rewritten in first-name/last-name style:
> 
> Masao Fujii

I have to admit I thought Fujii was his last name, then I thought it was
his first name, now you are saying it is his last name.  Obviously, I
am getting confused, and it does affect how I address him when I meet
him.

> Kyotaro Horiguchi
> Masahiko Sawada 
> Kohei KaiGai 
> Shigeru Hanada
> Etsuro Fujita
> Osamu Furuya 
> Mitsumasa Kondo
> MauMau
> 
> Note that I guess "MauMau" is a nickname.

I think we are fine consistenly putting Japanese last names first _if_
we always capitalize the last name, e.g. "FUJITA Etsuro" --- is that a
good idea? Does everyone like that?  Does any other country want that?

> > BTW, it is pretty cool to have contributor names that are natively in
> > scripts other than latin.  It is a pity that because of toolchain
> > limitations we cannot display names in kanji, cyrillic or other
> > character sets (in addition to their transliteration to latin script).
> 
> Japan User's Group (JPUG) recently starts to translate PostgreSQL doc
> using UTF-8 encoding. So technically this is possible. I'm not sure if
> developers want to have official document in other than ASCII though.

Ah, yes, you could do them special for that translation.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Michael Paquier
Date:
On Sun, Jun 14, 2015 at 12:30 PM, Bruce Momjian wrote:
> On Sun, Jun 14, 2015 at 11:44:25AM +0900, Tatsuo Ishii wrote:
>> The real question here is, if contributors are required to show their
>> "official" names (along with their alias or nickname if any).  If yes,
>> it is apparent that all contributor names should be shown in a unified
>> formal style.
>>
>> If not, we could leave as it is.
>>
>> BTW, for Japanese almost 99% of Japanese names are obvious in that
>> which is the first name and which is the last name. So we don't need
>> to ask each contributor.
>
> OK, good.
>
>> > The names we currently have in the 9.5 release notes are:
>> >
>> > Fujii Masao
>> > Kyotaro Horiguchi
>> > Sawada Masahiko
>> > KaiGai Kohei
>> > Shigeru Hanada
>> > Etsuro Fujita
>> > Furuya Osamu
>> > MauMau
>> > Mitsumasa Kondo
>>
>> F.Y.I. here is a list rewritten in first-name/last-name style:
>>
>> Masao Fujii

Last name: Fujii, first name: Masao.

>> Kyotaro Horiguchi

Last name: Horiguchi, first name: Kyotaro

>> Masahiko Sawada

Last name; Sawada, first name: Masahiko.

>> Kohei KaiGai

Last name: Kaigai, first name Kohei

>> Shigeru Hanada

Last name: Hanada, first name: Shigeru

>> Etsuro Fujita

Last name: Fujita, first name: Esturo.

>> Osamu Furuya

Last name: Furuya, first name: Osamu.

>> Mitsumasa Kondo

Last name: Kondo, first name: Mitsumasa

>> MauMau

Last name: Mau, First name: Mau (??) or
Last name: None, first name: MauMau or
Last name: Maumau, first name: None.
In any case that's an interesting nickname.
-- 
Michael



Re: 9.5 release notes

From
Petr Jelinek
Date:
On 2015-06-14 03:02, Bruce Momjian wrote:
> On Sat, Jun 13, 2015 at 05:47:59AM -0300, Alvaro Herrera wrote:
>> Petr Jelinek wrote:
>>> Hi,
>>>
>>> +      <listitem>
>>> +       <para>
>>> +        Add <type>JSONB</> functions <function>jsonb_set()</> and
>>> +        <function>jsonb_pretty</> (Dmitry Dolgov, Andrew Dunstan)
>>> +       </para>
>>> +      </listitem>
>>>
>>> I believe I should be 3rd author for this one as I rewrote large parts of
>>> this functionality as part of the review.
>>
>> Also, it looks like we could spell your last name with an accent,
>> assuming the i-acute character in Latin1 is fine.
>
> Ah, you are correct.  I found a few commits that did have that accent.
> All _seven_ of Petr's 9.5 commit mentions updated to add the accent.
> :-)  Thanks.
>

Yes i-accute is correct, thanks. I don't really mind my name being 
without the accent either though :)

--  Petr Jelinek                  http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training &
Services



Re: 9.5 release notes

From
Dean Rasheed
Date:
On 11 June 2015 at 05:15, Bruce Momjian <bruce@momjian.us> wrote:
> I have committed the first draft of the 9.5 release notes.  You can view
> the output here:
>
>         http://momjian.us/pgsql_docs/release-9-5.html
>

I think it's worth mentioning
dcbf5948e12aa60b4d6ab65b6445897dfc971e01, probably under "General
Performance". It's an optimisation, and also a user-visible change to
the way LEAKPROOF works. Perhaps something like

Allow pushdown of non-leakproof functions into security barrier views
if the function is not passed any arguments from the view.

Previously only functions marked as LEAKPROOF could be pushed down
into security barrier views.

Regards,
Dean



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Sat, Jun 13, 2015 at 11:30:30PM -0400, Bruce Momjian wrote:
> > Note that I guess "MauMau" is a nickname.
> 
> I think we are fine consistenly putting Japanese last names first _if_
> we always capitalize the last name, e.g. "FUJITA Etsuro" --- is that a
> good idea? Does everyone like that?  Does any other country want that?

OK, new idea.  What about, instead of having the last name be all-caps,
we have the last name start with an uppercase letter, then use smallcaps
for the rest of the last name:
https://en.wikipedia.org/wiki/Small_caps

That way, the last name will not appear too large, but will be clear as
something different from other names.  Peter, I assume small-caps is
possible.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Stephen Frost
Date:
* Bruce Momjian (bruce@momjian.us) wrote:
> On Sat, Jun 13, 2015 at 11:30:30PM -0400, Bruce Momjian wrote:
> > > Note that I guess "MauMau" is a nickname.
> >
> > I think we are fine consistenly putting Japanese last names first _if_
> > we always capitalize the last name, e.g. "FUJITA Etsuro" --- is that a
> > good idea? Does everyone like that?  Does any other country want that?
>
> OK, new idea.  What about, instead of having the last name be all-caps,
> we have the last name start with an uppercase letter, then use smallcaps
> for the rest of the last name:
>
>     https://en.wikipedia.org/wiki/Small_caps
>
> That way, the last name will not appear too large, but will be clear as
> something different from other names.  Peter, I assume small-caps is
> possible.

I realize this might come across as overkill, but, well, we actually
have this database system for anyone who has an account on
postgresql.org, the wiki, been to one of the conferences that uses the
PG auth system, etc.

Perhaps we just need to add a "display as" field or similar to that
system?  Or have a check-box for each individual to indicate if they'd
prefer to use 'First Last' or 'LAST First'.  We already track first and
last names as different fields.  We also have a unique username field
and we might be able to come up with a system along the lines of
'user:username' which is then replaced by whatever is in the
"display as" field in the database.  My thought would be for that to be
a one-time thing which is then reviewed and then the release notes are
published as they are today.  I don't think we'd want to dynamically
change the release notes based on this on the actual webpage, for
various reasons.

My 2c on the whole thing is that it should be up to the individual and
not based on country, as I can imagine individuals who woluld be annoyed
if their name started showing up in the release notes as
'FROST Stephen' simply because they moved to Japan (no, I don't have any
intentions of doing so, just saying).
Thanks,
    Stephen

Re: 9.5 release notes

From
Tom Lane
Date:
Bruce Momjian <bruce@momjian.us> writes:
> OK, new idea.  What about, instead of having the last name be all-caps,
> we have the last name start with an uppercase letter, then use smallcaps
> for the rest of the last name:
>     https://en.wikipedia.org/wiki/Small_caps
> That way, the last name will not appear too large, but will be clear as
> something different from other names.  Peter, I assume small-caps is
> possible.

FWIW, I vote strongly against having any contributor names in caps in the
release notes.  It would be visually distracting, and it would make the
name look like the most important thing in the entry, while in point of
fact it's the *least* important.  (Maybe not to the contributor, but
certainly to anybody else.)

For pretty much the same reason, I'm not in favor of small caps either.
Even assuming we can do that consistently (which I bet we can't; we
do not have all that much control over how web browsers render HTML),
it would be calling attention to itself, which is exactly not the result
I think we should be after.
        regards, tom lane



Re: 9.5 release notes

From
Alvaro Herrera
Date:
Bruce Momjian wrote:
> On Sat, Jun 13, 2015 at 05:47:59AM -0300, Alvaro Herrera wrote:

> > Also, it looks like we could spell your last name with an accent,
> > assuming the i-acute character in Latin1 is fine.
> 
> Ah, you are correct.  I found a few commits that did have that accent. 
> All _seven_ of Petr's 9.5 commit mentions updated to add the accent. 
> :-)  Thanks.

... though two of them now say Petr Petr.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Sun, Jun 14, 2015 at 01:57:08PM -0300, Alvaro Herrera wrote:
> Bruce Momjian wrote:
> > On Sat, Jun 13, 2015 at 05:47:59AM -0300, Alvaro Herrera wrote:
> 
> > > Also, it looks like we could spell your last name with an accent,
> > > assuming the i-acute character in Latin1 is fine.
> > 
> > Ah, you are correct.  I found a few commits that did have that accent. 
> > All _seven_ of Petr's 9.5 commit mentions updated to add the accent. 
> > :-)  Thanks.
> 
> ... though two of them now say Petr Petr.

Thanks, fixed.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Sat, Jun 13, 2015 at 6:24 PM, Bruce Momjian <bruce@momjian.us> wrote:
>> I think we should really address this. Attached patch adds a new
>> release note item for it. It also adds to the documentation that
>> explains why users should prefer varchar(n)/text to character(n); the
>> lack of abbreviated key support now becomes a huge disadvantage for
>> character(n), whereas in previous versions the disadvantages were
>> fairly minor.
>>
>> In passing, I updated the existing sort item to reflect that only
>> varchar(n), text, and numeric benefit from the abbreviation
>> optimization (not character types more generally + numeric), and added
>> a note on the effectiveness of the abbreviation optimization alone.
>
> This all also seems like overkill to me.  Users just don't care about
> this level of detail, and are easily confused by it.

Really? The pre-check thing wasn't too complex for Magnus to have a
couple of bullet points on it *specifically* in his high level NYC
talk on Postgres 9.5 features [1]. I don't think it's hard to
understand at all.

Also, it's simply incorrect to describe abbreviation as: "Improve the
speed of sorting character and numeric fields". Character fields
presumably include character(n), and as I pointed out character(n)
lacks abbreviation support. That needs to be documented as an
additional disadvantage of character(n) vs text/varchar(n) in the
documentation where the trade-off is already discussed (i.e. not in
the release notes), because it makes a *huge* difference. And ISTM
that it should also be clear from the release notes themselves. We
seemingly always have <type> </type> tags when types are named in
release notes.

[1] http://www.hagander.net/talks/postgresql95.pdf
-- 
Peter Geoghegan



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Sun, Jun 14, 2015 at 11:21:35AM -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > OK, new idea.  What about, instead of having the last name be all-caps,
> > we have the last name start with an uppercase letter, then use smallcaps
> > for the rest of the last name:
> >     https://en.wikipedia.org/wiki/Small_caps
> > That way, the last name will not appear too large, but will be clear as
> > something different from other names.  Peter, I assume small-caps is
> > possible.
> 
> FWIW, I vote strongly against having any contributor names in caps in the
> release notes.  It would be visually distracting, and it would make the
> name look like the most important thing in the entry, while in point of
> fact it's the *least* important.  (Maybe not to the contributor, but
> certainly to anybody else.)

Yes, we are already trying to deemphasize contributor names, so
capitalizing them is certainly moving in the wrong direction.

> For pretty much the same reason, I'm not in favor of small caps either.
> Even assuming we can do that consistently (which I bet we can't; we
> do not have all that much control over how web browsers render HTML),
> it would be calling attention to itself, which is exactly not the result
> I think we should be after.

I am sure almost every browser can render smallcaps, even if it doesn't
have a smallcaps-specific font installed --- same for PDF.  We couldn't
do that for a text file, but I don't think we ship a text HISTORY file
anymore.

I didn't think smallcaps would be any more visible than standard
lower-case text.  In fact, smallcaps is designed to fit the font size of
lowercase letters.  I would run a text but it doesn't seem we have any
references to smallcaps in our SGML files.  This could get tricky
because, as I remember, the rendering control is in the web style
sheets, and you would need to have all rendering do the same thing.

Any new ideas on how we can signify family names first?  It seems
culturally-insensitive to always put the family name last if people
don't want that, but it also seems odd to have a mix of ordering in the
same document.  Certainly others must have had the same issue.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Alvaro Herrera
Date:
Bruce Momjian wrote:
> On Sun, Jun 14, 2015 at 11:21:35AM -0400, Tom Lane wrote:

> > For pretty much the same reason, I'm not in favor of small caps either.
> > Even assuming we can do that consistently (which I bet we can't; we
> > do not have all that much control over how web browsers render HTML),
> > it would be calling attention to itself, which is exactly not the result
> > I think we should be after.
> 
> I am sure almost every browser can render smallcaps, even if it doesn't
> have a smallcaps-specific font installed --- same for PDF.

For HTML, smallcaps is a CSS property.  The way it works is that you
write in lowercase, and then the browser displays smallcaps.  So for the
browsers that don't do it correctly, it would just work fine by
displaying as lower case.

One trouble is how to write the SGML so that the smallcaps bits reach
the generated HTML.

I don't know about PDF.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Re: 9.5 release notes

From
Andres Freund
Date:
Hi,

On 2015-06-11 00:15:21 -0400, Bruce Momjian wrote:
> I have committed the first draft of the 9.5 release notes.  You can view
> the output here:

So, I did a pass through master's state:

>      <listitem>
>       <para>
>        Add <link linkend="BRIN">Block Range Indexes</> (<acronym>BRIN</>)
>        (Álvaro Herrera, Heikki Linnakangas, Emre Hasegeli)
>       </para>
>
>       <para>
>        <acronym>BRIN</> indexes are very compact and store the min/max
>        values for a range of heap blocks.
>       </para>
>      </listitem>

Maybe we should mention that they're cheap to maintain?

>      <listitem>
>       <para>
>        Improve in-memory hash performance (Tomas Vondra, Robert Haas)
>       </para>
>      </listitem>

"hash performance" is pretty general, there's lot of places where we use
hashing that aren't affected.

>      <listitem>
>       <para>
>        Improve concurrency of <link linkend="guc-shared-buffers">shared
>        buffer</> replacement (Robert Haas, Amit Kapila)
>       </para>
>      </listitem>

I think in the end that patch was enhanced to a significant degree by
making it lockless in d72731a70450b. I think the three (?) involved
patches should just combined under one entry.

>      <listitem>
>       <para>
>        Improve concurrent locking and buffer scan performance (Andres
>        Freund, Kevin Grittner)
>       </para>
>      </listitem>

If this is ab5194e6f, I don't think it makes sense to mention "buffer
scan" - it's just any lwlock, and buffer locks aren't the primary
benefit (ProcArrayLock, buffer mapping lock probably are that). I also
don't think Kevin was involved?

I think ed127002d8 and 4b4b680c should be mentioned in this section as
well. 4b4b680c will considerably reduce the per backend memory usage for
servers with large shared buffers.

>    <sect4>
>     <title>Server Settings</title>
>
>     <itemizedlist>
>
>      <listitem>
>       <para>
>        Replace <varname>checkpoint_segments</> with <link
>        linkend="guc-min-wal-size"><varname>min_wal_size</></> and
>        <link linkend="guc-max-wal-size"><varname>max_wal_size</></>
>        (Heikki Linnakangas)
>       </para>
>
>       <para>
>        This allows the allocation of a large number of <acronym>WAL</>
>        files without keeping them if they are not needed.
>       </para>
>      </listitem>

Hm. This affects performance significantly, should we also list it there?

>      <listitem>
>       <para>
>        Add <acronym>GUC</> <link
>        linkend="guc-wal-compression"><varname>wal_compression</></> to
>        enable compression of full page images stored in <acronym>WAL</>
>        (Rahila Syed, Michael Paquier)
>       </para>
>      </listitem>

Also rather performance relevant?

>      <listitem>
>       <para>
>        Archive <acronym>WAL</> files with suffix <literal>.partial</>
>        during standby promotion (Heikki Linnakangas)
>       </para>
>      </listitem>

This should be expanded, will mention to Heikki. Possibly also need to
be mentioned in the backward incompat section.

>      <listitem>
>       <para>
>        Allow the <link linkend="pg-replication-origin-create">labeling</>
>        of the origin of logical replication changes (Andres Freund)
>       </para>
>
>       <para>
>        This helps with change tracking.
>       </para>
>      </listitem>

I think it should be 'origin and progress'. The explanation should
probably rather be 'This is helpful when implementing replication
solutions" or something like it.


>      <listitem>
>       <para>
>        Allow control of table <acronym>WAL</> logging after table creation
>        with <link linkend="SQL-ALTERTABLE"><command>ALTER TABLE .. SET
>        LOGGED / UNLOGGED</></> (Fabrízio de Royes Mello)
>       </para>
>      </listitem>

This sounds a bit confusing. Maybe "Allow to convert a WAL logged table
to an UNLOGGED one, and the other way round"?


>    <sect4>
>     <title>System Information Functions and Views</title>

I wonder if

>      <listitem>
>       <para>
>        Report the backends holding replication slots in <link
>        linkend="catalog-pg-replication-slots"><structname>pg_replication_slots</></>
>        (Craig Ringer)
>       </para>
>
>       <para>
>        The new output column is <structname>active_pid</>.
>       </para>
>      </listitem>

shouldn't be moved her?

>      <listitem>
>       <para>
>        Allow <application>pg_dump</> to share a snapshot taken by another
>        session using <option>--snapshot</> (Simon Riggs, Michael Paquier)
>       </para>
>
>       <para>
>        The remote snapshot must have been exported by
>        <function>pg_export_snapshot()</> or been defined when creating
>        a logical replication slot.

'or exported by logical replication slot creation'?

>        This can be used by parallel
>        <application>pg_dump</> to use a consistent snapshot across
>        <application>pg_dump</> processes.
>       </para>

What do you mean by this comment? Parallel pg_dump internally does all
the snapshot stuff already, and it's independent of this option.

>      <listitem>
>       <para>
>        Change <application>pg_ctl</> default shutdown mode from
>        <literal>smart</> to <literal>fast</> (Bruce Momjian)
>       </para>
>      </listitem>

Wonder if this should be listed as an incompatibility. This does have
have impact on existing setups/scripts.

>      <listitem>
>       <para>
>        Add basic atomics <acronym>API</> support (Andres Freund, Oskari
>        Saarenmaa)
>       </para>
>      </listitem>

Out of fairness I think either Oskari shouldn't be listed, or Amit
should as well. Amit surely has spent more time on the patch than
Oskari.

>      <listitem>
>       <para>
>        Add native compiler and memory barriers for <productname>Solaris
>        Studio</> (Oskari Saarenmaa)
>       </para>
>
>       <para>
>        IS THIS PART OF ATOMICS?
>       </para>
>      </listitem>

Not really, barriers are independent. But I guess we could just combine
it nonetheless.

>   <sect3>
>    <title>Additional Modules</title>
>
>    <itemizedlist>
>
>     <listitem>
>      <para>
>       Add <link linkend="app-pgrewind"><application>pg_rewind</></>,
>       which allows re-synchronizing a master server after failback
>       (Heikki Linnakangas)
>      </para>
>     </listitem>

Hm. pg_rewind isn't in contrib, so is this the right place for this?


I think 647248e3708, 4fe384bd85, 4f85fde8, 59f71a0d0 should also be
mentioned. They're amongst the most invasive changes in 9.5, and they do
have implications for developers and users.

f5ae3ba4/779fdcd should be listed as improving bitmap index scan
performance.

Greetings,

Andres Freund



Re: 9.5 release notes

From
Andres Freund
Date:
Hi,

On 2015-06-11 00:15:21 -0400, Bruce Momjian wrote:
> I have committed the first draft of the 9.5 release notes.  You can view
> the output here:

I'm looking through all the commits, checking which I think should
possibly be mentioned additionally:
- 9f03ca91 - Speed up CREATE INDEX by avoiding to copy index tuples
- 9da86753 - Reject duplicate column names in foreign key referenced-columns lists. behavioural change, should be
documentedas intentional
 
- 680513ab - Break out OpenSSL-specific code to separate files. should perhaps be mentioned in the developer section.
- 0709b7ee7 - Change the spinlock primitives to function as compiler barriers. Significant behavioural change for
developers.
- 94028691 - Advance backend's advertised xmin more aggressively. Pretty good feature imo.
- 5028f22f6 - Switch to CRC-32C in WAL and other places. This might have compability impact in some external tools.
- 4f1b890b1 - Unsure if this should be mentioned, at least it's a fairly large change.
- 27846f02 - Optimize locking a tuple already locked by another subxact Several users ran into this slowness, so I
thinkwe should mention the optimization.
 

Regards,

Andres



Re: 9.5 release notes

From
Rajeev rastogi
Date:
On 11 June 2015 09:45, Bruce Momjian Wrote:

>
> I have committed the first draft of the 9.5 release notes.  You can
> view the output here:
>
>     http://momjian.us/pgsql_docs/release-9-5.html
>
> and it will eventually appear here:
>
>     http://www.postgresql.org/docs/devel/static/release.html
>
> I am ready to make suggested adjustments, though I am traveling for
> conferences for the next ten days so there might a delay in my replies.

Below performance improvement in the "General Performance" category is missing:
   Reduce btree scan overhead for < and > strategies      For <, <=, > and >= strategies, mark the first scan key   as
alreadymatched if scanning in an appropriate direction.   If index tuple contains no nulls we can skip the first
re-checkfor each tuple.      Author: Kumar Rajeev Rastogi   Committer: Simon Riggs 

Thanks and Regards,
Kumar Rajeev Rastogi



Re: 9.5 release notes

From
Kevin Grittner
Date:
Andres Freund <andres@anarazel.de> wrote:
>>      <listitem>
>>       <para>
>>        Improve concurrent locking and buffer scan performance (Andres
>>        Freund, Kevin Grittner)
>>       </para>
>>      </listitem>
>
> If this is ab5194e6f, I don't think it makes sense to mention "buffer
> scan" - it's just any lwlock, and buffer locks aren't the primary
> benefit (ProcArrayLock, buffer mapping lock probably are that). I also

> don't think Kevin was involved?

It seems likely that 2ed5b87f9 was combined with something else in
this reference.  By reducing buffer pins and buffer content locking
during btree index scans it shows a slight performance gain in
btree scans and avoids some blocking of btree index vacuuming.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: 9.5 release notes

From
Robert Haas
Date:
On Tue, Jun 23, 2015 at 5:48 PM, Kevin Grittner <kgrittn@ymail.com> wrote:
> Andres Freund <andres@anarazel.de> wrote:
>>>      <listitem>
>>>       <para>
>>>        Improve concurrent locking and buffer scan performance (Andres
>>>        Freund, Kevin Grittner)
>>>       </para>
>>>      </listitem>
>>
>> If this is ab5194e6f, I don't think it makes sense to mention "buffer
>> scan" - it's just any lwlock, and buffer locks aren't the primary
>> benefit (ProcArrayLock, buffer mapping lock probably are that). I also
>
>> don't think Kevin was involved?
>
> It seems likely that 2ed5b87f9 was combined with something else in
> this reference.  By reducing buffer pins and buffer content locking
> during btree index scans it shows a slight performance gain in
> btree scans and avoids some blocking of btree index vacuuming.

I think maybe we should separate that back out.  The list needs to be
user-accessible, but if it's hard to understand what it's referring
to, that's not good either.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: 9.5 release notes

From
Andres Freund
Date:
On 2015-06-23 21:08:36 -0400, Robert Haas wrote:
> On Tue, Jun 23, 2015 at 5:48 PM, Kevin Grittner <kgrittn@ymail.com> wrote:
> > Andres Freund <andres@anarazel.de> wrote:
> >>>      <listitem>
> >>>       <para>
> >>>        Improve concurrent locking and buffer scan performance (Andres
> >>>        Freund, Kevin Grittner)
> >>>       </para>
> >>>      </listitem>
> >>
> >> If this is ab5194e6f, I don't think it makes sense to mention "buffer
> >> scan" - it's just any lwlock, and buffer locks aren't the primary
> >> benefit (ProcArrayLock, buffer mapping lock probably are that). I also
> >
> >> don't think Kevin was involved?
> >
> > It seems likely that 2ed5b87f9 was combined with something else in
> > this reference.  By reducing buffer pins and buffer content locking
> > during btree index scans it shows a slight performance gain in
> > btree scans and avoids some blocking of btree index vacuuming.

Oh. That's what it was combined with. I don't think it makes sense to
throw these three items together into one note. Their benefit/risk
potential is pretty different.

> I think maybe we should separate that back out.  The list needs to be
> user-accessible, but if it's hard to understand what it's referring
> to, that's not good either.

Yea. And if then Bruce goes and compares feature counts... :)

Greetings,

Andres Freund



Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Sun, Jun 14, 2015 at 2:08 PM, Peter Geoghegan <pg@heroku.com> wrote:
> Really? The pre-check thing wasn't too complex for Magnus to have a
> couple of bullet points on it *specifically* in his high level NYC
> talk on Postgres 9.5 features [1]. I don't think it's hard to
> understand at all.
>
> Also, it's simply incorrect to describe abbreviation as: "Improve the
> speed of sorting character and numeric fields". Character fields
> presumably include character(n), and as I pointed out character(n)
> lacks abbreviation support.

Where are we on this? Bruce mentioned that he'd revisit this during pgCon.

Aside from the issue of whether or not the pre-check thing is
mentioned, and aside from the issue of correctly identifying which
types the abbreviation optimization applies to, I have another
concern: I cannot imagine why we'd fail to mention a totally
independent speed up of about 10% [1] for CREATE INDEX on integer
columns. This speed-up has nothing to do with abbreviation or anything
else mentioned in the 9.5 release notes currently; it's down to commit
5ea86e6e, which extended sortsupport to work with cases like CREATE
INDEX and CLUSTER.

[1] http://www.postgresql.org/message-id/545AC1D9.1040009@proxel.se
-- 
Peter Geoghegan



Re: 9.5 release notes

From
Amit Langote
Date:
Hi,

On 2015-06-11 PM 01:15, Bruce Momjian wrote:
> I have committed the first draft of the 9.5 release notes.  You can view
> the output here:
> 
>     http://momjian.us/pgsql_docs/release-9-5.html
>     
> and it will eventually appear here:
> 
>     http://www.postgresql.org/docs/devel/static/release.html
> 
> I am ready to make suggested adjustments, though I am traveling for
> conferences for the next ten days so there might a delay in my replies.
> 

Is it intentional that the following items are separate?
     <listitem>      <para>       Move <application>pg_upgrade</> from <filename>contrib</> to
<filename>src/bin</>(Peter Eisentraut)      </para>     </listitem>
 

    <listitem>     <para>      Move <application>pg_upgrade_support</> code into backend and      remove the module
(PeterEisentraut)     </para>    </listitem>
 

Or could they made into one item?

Thanks,
Amit




Re: 9.5 release notes

From
Robert Haas
Date:
On Wed, Jun 24, 2015 at 6:47 PM, Peter Geoghegan <pg@heroku.com> wrote:
> On Sun, Jun 14, 2015 at 2:08 PM, Peter Geoghegan <pg@heroku.com> wrote:
>> Really? The pre-check thing wasn't too complex for Magnus to have a
>> couple of bullet points on it *specifically* in his high level NYC
>> talk on Postgres 9.5 features [1]. I don't think it's hard to
>> understand at all.
>>
>> Also, it's simply incorrect to describe abbreviation as: "Improve the
>> speed of sorting character and numeric fields". Character fields
>> presumably include character(n), and as I pointed out character(n)
>> lacks abbreviation support.
>
> Where are we on this? Bruce mentioned that he'd revisit this during pgCon.
>
> Aside from the issue of whether or not the pre-check thing is
> mentioned, and aside from the issue of correctly identifying which
> types the abbreviation optimization applies to, I have another
> concern: I cannot imagine why we'd fail to mention a totally
> independent speed up of about 10% [1] for CREATE INDEX on integer
> columns. This speed-up has nothing to do with abbreviation or anything
> else mentioned in the 9.5 release notes currently; it's down to commit
> 5ea86e6e, which extended sortsupport to work with cases like CREATE
> INDEX and CLUSTER.

Can you put your suggestions here in the form of a patch to the release notes?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: 9.5 release notes

From
Robert Haas
Date:
On Thu, Jun 25, 2015 at 12:09 AM, Amit Langote
<Langote_Amit_f8@lab.ntt.co.jp> wrote:
> On 2015-06-11 PM 01:15, Bruce Momjian wrote:
>> I have committed the first draft of the 9.5 release notes.  You can view
>> the output here:
>>
>>       http://momjian.us/pgsql_docs/release-9-5.html
>>
>> and it will eventually appear here:
>>
>>       http://www.postgresql.org/docs/devel/static/release.html
>>
>> I am ready to make suggested adjustments, though I am traveling for
>> conferences for the next ten days so there might a delay in my replies.
>>
>
> Is it intentional that the following items are separate?
>
>       <listitem>
>        <para>
>         Move <application>pg_upgrade</> from <filename>contrib</> to
>         <filename>src/bin</> (Peter Eisentraut)
>        </para>
>       </listitem>
>
>
>      <listitem>
>       <para>
>        Move <application>pg_upgrade_support</> code into backend and
>        remove the module (Peter Eisentraut)
>       </para>
>      </listitem>
>
> Or could they made into one item?

I think one item would be fine.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: 9.5 release notes

From
Alvaro Herrera
Date:
Robert Haas wrote:
> On Thu, Jun 25, 2015 at 12:09 AM, Amit Langote
> <Langote_Amit_f8@lab.ntt.co.jp> wrote:

> > Is it intentional that the following items are separate?
> > [...]
> > Or could they made into one item?
> 
> I think one item would be fine.

I suggested the same a couple of weeks ago.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Re: 9.5 release notes

From
Robert Haas
Date:
On Fri, Jun 26, 2015 at 2:01 PM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
> Robert Haas wrote:
>> On Thu, Jun 25, 2015 at 12:09 AM, Amit Langote
>> <Langote_Amit_f8@lab.ntt.co.jp> wrote:
>
>> > Is it intentional that the following items are separate?
>> > [...]
>> > Or could they made into one item?
>>
>> I think one item would be fine.
>
> I suggested the same a couple of weeks ago.

Great, done.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Fri, Jun 26, 2015 at 9:29 AM, Robert Haas <robertmhaas@gmail.com> wrote:
> Can you put your suggestions here in the form of a patch to the release notes?

The attached patch generalizes from the 9.2 release note wording. I
use the word "inlined" here too, even though commit 5ea86e6e6 did not
do too much with inlining of C code (unlike the 9.2 stuff that went in
a little after SortSupport itself).

Seems better to be consistent with the earlier item, and it is still
probably in some sense true, because the new SortSupport-wise inlined
comparator probably benefits from inlining more than the historic
scanKey-wise inlined comparator, due to the removal of indirection.

As I'm fond of pointing out, inlining is mostly useful as an "enabling
transformation" these days.
--
Peter Geoghegan

Attachment

Re: 9.5 release notes

From
Robert Haas
Date:
On Fri, Jun 26, 2015 at 2:25 PM, Peter Geoghegan <pg@heroku.com> wrote:
> On Fri, Jun 26, 2015 at 9:29 AM, Robert Haas <robertmhaas@gmail.com> wrote:
>> Can you put your suggestions here in the form of a patch to the release notes?
>
> The attached patch generalizes from the 9.2 release note wording. I
> use the word "inlined" here too, even though commit 5ea86e6e6 did not
> do too much with inlining of C code (unlike the 9.2 stuff that went in
> a little after SortSupport itself).
>
> Seems better to be consistent with the earlier item, and it is still
> probably in some sense true, because the new SortSupport-wise inlined
> comparator probably benefits from inlining more than the historic
> scanKey-wise inlined comparator, due to the removal of indirection.

OK, but how about this wording instead?

diff --git a/doc/src/sgml/release-9.5.sgml b/doc/src/sgml/release-9.5.sgml
index 4086c6b..5e8cc15 100644
--- a/doc/src/sgml/release-9.5.sgml
+++ b/doc/src/sgml/release-9.5.sgml
@@ -221,6 +221,15 @@
      <listitem>       <para>
+        Extend the infrastructure allow sorting to be performed by inlined,
+        non-<acronym>SQL</acronym>-callable comparison functions to cover
+        <command>CREATE INDEX</>, <command>REINDEX</>, and
+        <command>CLUSTER</> (Peter Geoghegan)
+       </para>
+      </listitem>
+
+      <listitem>
+       <para>        Improve in-memory hash performance (Tomas Vondra, Robert Haas)       </para>      </listitem>

BTW, shouldn't Andrew also be credited here, since he did the work on
datum sorts?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Fri, Jun 26, 2015 at 11:29 AM, Robert Haas <robertmhaas@gmail.com> wrote:
>
> OK, but how about this wording instead?

That seems fine.

> BTW, shouldn't Andrew also be credited here, since he did the work on
> datum sorts?

Andrew's work was entirely confined to making datum sorts work with
abbreviation, which seems entirely orthogonal (but was enough to make
me want to credit him as an author of abbreviated keys, even after
breaking out his work on numeric support into a separate item). This
particular piece of work has nothing to do with the datum sort case,
though.

Datum sorts always supported SortSupport. This commit, 5ea86e6e6,
really should have been in 9.2 (especially since it had a net-negative
code footprint and clearly simplified tuplesort), and had nothing to
do with abbreviation -- it went in before abbreviation, and before it
was 100% clear that abbreviation would ever land.

-- 
Peter Geoghegan



Re: 9.5 release notes

From
Robert Haas
Date:
On Fri, Jun 26, 2015 at 2:37 PM, Peter Geoghegan <pg@heroku.com> wrote:
> On Fri, Jun 26, 2015 at 11:29 AM, Robert Haas <robertmhaas@gmail.com> wrote:
>>
>> OK, but how about this wording instead?
>
> That seems fine.
>
>> BTW, shouldn't Andrew also be credited here, since he did the work on
>> datum sorts?
>
> Andrew's work was entirely confined to making datum sorts work with
> abbreviation, which seems entirely orthogonal (but was enough to make
> me want to credit him as an author of abbreviated keys, even after
> breaking out his work on numeric support into a separate item). This
> particular piece of work has nothing to do with the datum sort case,
> though.
>
> Datum sorts always supported SortSupport. This commit, 5ea86e6e6,
> really should have been in 9.2 (especially since it had a net-negative
> code footprint and clearly simplified tuplesort), and had nothing to
> do with abbreviation -- it went in before abbreviation, and before it
> was 100% clear that abbreviation would ever land.

OK, understood, and thanks for the clarification.  I've committed the
version I proposed.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Wed, Jun 10, 2015 at 9:15 PM, Bruce Momjian <bruce@momjian.us> wrote:
> I am ready to make suggested adjustments

I attach a compatibility note that is clearly needed; adding this is
an open item of mine for 9.5. This concerns foreign data wrappers and
UPSERT.

--
Peter Geoghegan

Attachment

Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Fri, Jun 26, 2015 at 3:39 PM, Peter Geoghegan <pg@heroku.com> wrote:
> On Wed, Jun 10, 2015 at 9:15 PM, Bruce Momjian <bruce@momjian.us> wrote:
>> I am ready to make suggested adjustments

Also, I attach a new description of the UPSERT feature. For me, UPSERT
was always about guarantees that make statements work robustly under
standard operating conditions -- a novice user should be able to write
a simple UPSERT statement and forget about it. It's really hard to
explain how that can be guaranteed, but the basic guarantee itself is
simple, and matters a lot.

I think that the proposed wording conveys that. I also think that it's
important that this message is prominently conveyed, because these
issues cause no end of confusion.

--
Peter Geoghegan

Attachment

Re: 9.5 release notes

From
Andres Freund
Date:
Hi,

On 2015-06-11 00:15:21 -0400, Bruce Momjian wrote:
> I have committed the first draft of the 9.5 release notes.

I'm working on integrating the suggestions I made last week to the
release notes. Would anybody mind if I start to add commit ids in
comments, similar to what Tom has done for minor releases, to news
items?

Greetings,

Andres Freund



Re: 9.5 release notes

From
Tom Lane
Date:
Andres Freund <andres@anarazel.de> writes:
> I'm working on integrating the suggestions I made last week to the
> release notes. Would anybody mind if I start to add commit ids in
> comments, similar to what Tom has done for minor releases, to news
> items?

+1.  Helps confirm which items are meant to correspond to which commits.

In case you didn't realize it already, the stuff I put into the minor
release notes is from src/tools/git_changelog.  Dunno whether we need
to use that exact format for major releases though; there's no need to
identify branches in major release notes.
        regards, tom lane



Re: 9.5 release notes

From
Andres Freund
Date:
On 2015-06-29 17:05:59 -0400, Tom Lane wrote:
> +1.  Helps confirm which items are meant to correspond to which commits.

That's what made me think of it.

> In case you didn't realize it already, the stuff I put into the minor
> release notes is from src/tools/git_changelog.  Dunno whether we need
> to use that exact format for major releases though; there's no need to
> identify branches in major release notes.

I'd recognized the format, but I didn't want to exactly go that way. As
you say, the branch information is redundant.

Haven't yet thought much about the format, maybe in the style of
git log --pretty='format:[%h] %aN [%ci]: %s' upstream/master
I'd personally like to see the hash and the time, the rest isn't
particularly important to me.

Greetings,

Andres Freund



Re: 9.5 release notes

From
Tom Lane
Date:
Andres Freund <andres@anarazel.de> writes:
> Haven't yet thought much about the format, maybe in the style of
> git log --pretty='format:[%h] %aN [%ci]: %s' upstream/master
> I'd personally like to see the hash and the time, the rest isn't
> particularly important to me.

[ ... plays with that ... ]  Hm.  To keep down the bulk of the release
notes, I'd kind of like to fit this info into single SGML comment
lines, ie

<!-- info here -->

and if that's to fit into 80 columns without wrapping, the "info" is
pretty constrained.  I get something like this when running git log
in a 70-column window:

[f78329d] Tom Lane [2015-06-29 15:42:18 -0400]: Stamp 9.5alpha1.
[85c25fd] Tom Lane [2015-06-29 15:38:46 -0400]: Desultory review of 9.
[cbc8d65] Tom Lane [2015-06-29 12:42:52 -0400]: Code + docs review for
[07cb8b0] Andres Freund [2015-06-29 14:53:32 +0200]: Replace ia64 S_UN
[c5e5d44] Peter Eisentraut [2015-06-28 23:56:55 -0400]: Translation up
[2bdc51a] Tom Lane [2015-06-28 20:49:35 -0400]: Run the C portions of 
[62d16c7] Tom Lane [2015-06-28 18:06:14 -0400]: Improve design and imp
[d661532] Heikki Linnakangas [2015-06-29 00:09:10 +0300]: Also trigger
[6ab4d38] Heikki Linnakangas [2015-06-29 00:01:26 +0300]: Fix markup i
[a32c3ec] Heikki Linnakangas [2015-06-28 22:30:39 +0300]: Promote the 
[a45c70a] Heikki Linnakangas [2015-06-28 22:16:21 +0300]: Fix double-X
[b36805f] Heikki Linnakangas [2015-06-28 21:35:51 +0300]: Don't choke 
[cb2acb1] Heikki Linnakangas [2015-06-28 21:35:46 +0300]: Add missing_
[cca8ba9] Kevin Grittner [2015-06-28 12:43:59 -0500]: Fix comment for 
[527e6d3] Tatsuo Ishii [2015-06-28 18:54:27 +0900]: Fix function decla
[0a52d37] Tom Lane [2015-06-27 17:47:39 -0400]: Avoid passing NULL to 
[d47a113] Andres Freund [2015-06-27 19:00:45 +0200]: Fix test_decoding
[604e993] Kevin Grittner [2015-06-27 09:55:06 -0500]: Add opaque decla
[7845db2] Heikki Linnakangas [2015-06-27 10:17:42 +0300]: Fix typo in 
[66fbcb0] Simon Riggs [2015-06-27 00:41:47 +0100]: Avoid hot standby c
[7d60b2a] Alvaro Herrera [2015-06-26 18:17:54 -0300]: Fix DDL command 
[4028222] Alvaro Herrera [2015-06-26 18:13:05 -0300]: Fix BRIN xlog re
[7c02d48] Robert Haas [2015-06-26 16:04:46 -0400]: Fix grammar.
[8f15f74] Robert Haas [2015-06-26 15:53:13 -0400]: Be more conservativ
[c66bc72] Robert Haas [2015-06-26 14:49:37 -0400]: release notes: Add 
[8a8c581] Robert Haas [2015-06-26 14:46:48 -0400]: Remove unnecessary 
[31c018e] Robert Haas [2015-06-26 14:20:29 -0400]: release notes: Comb
[9043ef3] Robert Haas [2015-06-26 11:37:32 -0400]: Don't warn about cr

That might be all right but it seems like the timestamp is taking up
an undue amount of space.  If we could get git log to put out only
the date, it would be better IMO, but I don't see a format option
that does that :-(.

I wonder if it's worth teaching git_changelog to have an option
to put out summary lines in the specific format we select.
        regards, tom lane



Re: 9.5 release notes

From
Andres Freund
Date:
On 2015-06-29 17:30:57 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > Haven't yet thought much about the format, maybe in the style of
> > git log --pretty='format:[%h] %aN [%ci]: %s' upstream/master
> > I'd personally like to see the hash and the time, the rest isn't
> > particularly important to me.
> 
> [ ... plays with that ... ]  Hm.  To keep down the bulk of the release
> notes, I'd kind of like to fit this info into single SGML comment
> lines, ie

> and if that's to fit into 80 columns without wrapping, the "info" is
> pretty constrained.  I get something like this when running git log
> in a 70-column window:

> [f78329d] Tom Lane [2015-06-29 15:42:18 -0400]: Stamp 9.5alpha1.

I agree this is a bit long, but I don't particularly care about 70/80
cols.

How about:
git log --pretty='format:%cd [%h] %cN: %s' --date=short upstream/master
2015-06-29 [f78329d] Tom Lane: Stamp 9.5alpha1.
2015-06-29 [85c25fd] Tom Lane: Desultory review of 9.5 release notes.
2015-06-29 [cbc8d65] Tom Lane: Code + docs review for escaping of option values (commit 11a020eb6).
2015-06-29 [07cb8b0] Andres Freund: Replace ia64 S_UNLOCK compiler barrier with a full memory barrier.
2015-06-28 [c5e5d44] Peter Eisentraut: Translation updates
2015-06-28 [2bdc51a] Tom Lane: Run the C portions of guc-file.l through pgindent.
2015-06-28 [62d16c7] Tom Lane: Improve design and implementation of pg_file_settings view.

Greetings,

Andres Freund



Re: 9.5 release notes

From
Tom Lane
Date:
Andres Freund <andres@anarazel.de> writes:
> How about:
> git log --pretty='format:%cd [%h] %cN: %s' --date=short upstream/master

Ah yes, didn't see that option.  That's definitely better.  I'd still vote
for restricting it to fit in an 80-column window though.
        regards, tom lane



Re: 9.5 release notes

From
Andres Freund
Date:
On 2015-06-29 17:44:06 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > How about:
> > git log --pretty='format:%cd [%h] %cN: %s' --date=short upstream/master
> 
> Ah yes, didn't see that option.  That's definitely better.  I'd still vote
> for restricting it to fit in an 80-column window though.

I don't see the benefit of truncating away the end of the commit message
- that'll just mean more manual work and harder to understand
heading... It's also not like we're generally very religious about the
line length in sgml?



Re: 9.5 release notes

From
Tom Lane
Date:
Andres Freund <andres@anarazel.de> writes:
> On 2015-06-29 17:44:06 -0400, Tom Lane wrote:
>> Ah yes, didn't see that option.  That's definitely better.  I'd still vote
>> for restricting it to fit in an 80-column window though.

> I don't see the benefit of truncating away the end of the commit message
> - that'll just mean more manual work and harder to understand
> heading... It's also not like we're generally very religious about the
> line length in sgml?

Yeah we are.  The only places you'll find where we aren't formatting to 77
or 78 columns or so are where it would require breaking SGML tags in weird
places.  If we use this format without truncating the log lines then it'll
become practically impossible to edit release notes in a window less than
about 120 characters wide, and I for one will object to that.  It doesn't
fit well in my usual screen layout.  And it would be very wasteful of
screen space because even if you let regular text flow all the way to a
120-character margin, there are enough short lines like "<para>" that
there'd just be huge amounts of white space on your screen.

As for manual work to generate the format, we could fix that by getting
git_changelog to emit what we want.  I'd think the best thing to start
from would be the summary lines interspersed with full commit messages
anyway; the raw output of git log is going to be near unusable for this
purpose, with or without truncation.

(There are two different things to worry about here.  One is how you're
going to reverse-engineer the annotations into the release notes Bruce
already made.  Even un-truncated first lines of commit messages will often
not be enough for matching up commits to those notes.  But I'm thinking
more about how we do this in future release cycles, and for that, getting
git_changelog to help is clearly the ticket.)
        regards, tom lane



Re: 9.5 release notes

From
Andres Freund
Date:
On 2015-06-29 17:58:54 -0400, Tom Lane wrote:
> Yeah we are.  The only places you'll find where we aren't formatting to 77
> or 78 columns or so are where it would require breaking SGML tags in weird
> places.

Which isn't exactly infrequent...

Anyway, how about:
format:%cd [%h] %<(8,trunc)%cN: %<(49,trunc)%s

(which you can configure as pretty.pgmajor or so in .gitconfig btw.)

> (There are two different things to worry about here.  One is how you're
> going to reverse-engineer the annotations into the release notes Bruce
> already made.  Even un-truncated first lines of commit messages will often
> not be enough for matching up commits to those notes.  But I'm thinking
> more about how we do this in future release cycles, and for that, getting
> git_changelog to help is clearly the ticket.)

I'm not against doing that.



Re: 9.5 release notes

From
Andres Freund
Date:
I've gone through the release notes and added comments referencing
commits as discussed earlier. Additionally I've improved and added a
bunch of items.

Further stuff that came up while looking:
* 2014-09-25 [b0d81ad] Heikki..: Add -D option to specify data directory to pg_c.. new options, should possibly be
documented?

* 2014-10-02 [3acc10c9] Robert..: Increase the number of buffer mapping partition.. should we mention this? This has
beenpatched by a number of downstream vendors and users, so it's probably worth calling out?
 

* 2014-11-18 [606c012] Simon ..: Reduce btree scan overhead for < and > strategies Worthy of a note in the performance
section.

* 2014-11-22 [eca2b9b] Andrew..: Rework echo_hidden for \sf and \ef from commit .. Seems far too minor in comparison to
otherstuff left out.
 

* 2014-11-07 [7516f52] Alvaro..: BRIN: Block Range Indexes Looks to me like that should just be Alvaro.

* 2014-11-22 [b62f94c] Tom Lane: Allow simplification of EXISTS() subqueries con.. Planner change, I think it's good to
mentionthose.
 

* 2014-11-28 [e384ed6] Tom Lane: Improve typcache: cache negative lookup results.. Should perhaps, together with other
cacheenhancing entries, be mentioned?
 

* 2014-12-08 [519b075] Simon ..: Use GetSystemTimeAsFileTime directly in win32 2014-12-08 [8001fe6] Simon ..: Windows:
useGetSystemTimePreciseAsFileTime if .. Timer resolution isn't a unimportant thing for people using explain?
 

* 2014-12-12 [7e354ab] Andrew..: Add several generator functions for jsonb that .. 2015-05-12 [c694701] Andrew..:
Additionalfunctions and operators for jsonb 2015-05-31 [37def42] Andrew..: Rename jsonb_replace to jsonb_set and allow
it.. 2014-12-12 [237a882] Andrew..: Add json_strip_nulls and jsonb_strip_nulls fun.. should probably be merged? Similar
setof authors and too many similar release note entries.
 

* 2014-12-23 [d7ee82e] Alvaro..: Add SQL-callable pg_get_object_address
* 2014-12-30 [a676201] Alvaro..: Add pg_identify_object_as_address should be merged.

* 2015-01-13 [4bad60e] Andres..: Allow latches to wait for socket writability wi.. 2015-01-14 [59f71a0] Andres..: Add a
defaultlocal latch for use in signal han.. 2015-01-17 [ff44fba] Andres..: Replace walsender's latch with the general
shar..2015-02-03 [387da18] Andres..: Use a nonblocking socket for FE/BE communicatio.. 2015-02-03 [4f85fde] Andres..:
Introduceand use infrastructure for interrupt .. 2015-02-03 [4fe384b] Andres..: Process 'die' interrupts while
reading/writing.. 2015-02-03 [6647248] Andres..: Don't allow immediate interrupts during authent.. 2015-02-03 [6753333]
Andres..:Move deadlock and other interrupt handling in p.. 2015-02-13 [80788a4] Heikki..: Simplify waiting logic in
readingfrom /writin..
 

* 2015-01-17 [9402869] Heikki..: Advance backend's advertised xmin more aggressi.. This is a pretty big efficiency boon
forsystems with longer nontrivial transactions.
 

* 2015-01-29 [ed12700] Andres..: Align buffer descriptors to cache line boundari.. Maybe worthwhile to mention?

* 2015-02-16 [9e3ad1a] Tom Lane: Use fast path in plpgsql's RETURN/RETURN NEXT i.. 2015-02-28 [e524cbd] Tom Lane: Track
typmodsin plpgsql expression evaluation .. 2015-03-04 [1345cc6] Tom Lane: Use standard casting mechanism to convert
types..2015-03-11 [21dcda2] Tom Lane: Allocate ParamListInfo once per plpgsql functio..
 
 Maybe reformulate to generalize the array performance into plpgsql and mention arrays and RETURN? And combine with the
castingchange entry, because that's also about performance?
 

* 2015-02-20 [09d8d11] Tom Lane: Use FLEXIBLE_ARRAY_MEMBER in a bunch more places. 2015-02-20 [5740be6] Tom Lane: Some
moreFLEXIBLE_ARRAY_MEMBER hacking. 2015-02-20 [e38b1eb] Tom Lane: Use FLEXIBLE_ARRAY_MEMBER in struct varlena.
2015-02-20[c110eff] Tom Lane: Use FLEXIBLE_ARRAY_MEMBER in struct RecordIOData. 2015-02-20 [33a3b03] Tom Lane: Use
FLEXIBLE_ARRAY_MEMBERin some more places. 2015-02-20 [33b2a2c] Tom Lane: Fix statically allocated struct with
FLEXIBLE_A..2015-02-21 [f2874fe] Tom Lane: Some more FLEXIBLE_ARRAY_MEMBER fixes. 2015-02-21 [e1a11d9] Tom Lane: Use
FLEXIBLE_ARRAY_MEMBERfor HeapTupleHeaderDa.. 2015-02-21 [2e21121] Tom Lane: Use FLEXIBLE_ARRAY_MEMBER in a number of
other.. Maybe add a note in the source code section? Not sure if relevant enough.
 

* 2015-02-21 [b419865] Jeff D..: In array_agg(), don't create a new context for .. Peformance improvement?

* 2015-02-27 [f65e827] Tom Lane: Invent a memory context reset/delete callback m.. Interesting for devs, maybe add?

* 2015-03-11 [b557226] Tom Lane: Improve planner's cost estimation in the presen.. 2015-03-03 [b989619] Tom Lane: Fix
costestimation for indexscans on expensive.. 2015-03-11 [f4abd02] Tom Lane: Support flattening of empty-FROM subqueries
and..2015-05-04 [2503982] Tom Lane: Improve procost estimates for some text search .. Mention? Notes about planner
changesrather helpful when trying to pin down performance regression.s
 


* 2015-03-18 [f9dead5] Alvaro..: Install shared libraries to bin/ in Windows und.. Shouldn't this be mentioned rather
thanunimportant stuff like
 

* 2015-03-23 [e5f455f] Tom Lane: Apply table and domain CHECK constraints in nam.. I think this should be dropped, it
hasessentially no consequence to users.
 

* 2015-03-24 [1d8198b] Bruce ..: Add support for ALTER TABLE IF EXISTS ... RENAM.. Should imo be wrapped together with
theother IF EXISTS changes
 

* 2014-12-09 [ae4e688] Simon ..: Silence REINDEX Doesn't seem to warrant a separate release note entry.

* 2015-05-19 [0b28ea7] Tom Lane: Avoid collation dependence in indexes of syste.. Doesn't seem to warrant a separate
releasenote entry. We've left of many more significant things.
 

* 2015-04-14 [9fa8b0e] Peter ..: Move pg_upgrade from contrib/ to src/bin/ 2015-04-13 [81134af] Peter ..: Move pgbench
fromcontrib/ to src/bin/ Should imo merged with all the other moved utilities.
 

* 2014-07-14 [91f03ba] Noah M..: MSVC: Recognize PGFILEDESC in contrib and conv.. 2015-04-16 [22d0053] Alvaro..: MSVC:
installsrc/test/modules together with c.. Don't seem to warrant a release note entry.
 

Greetings,

Andres Freund



Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Fri, Jun 26, 2015 at 3:39 PM, Peter Geoghegan <pg@heroku.com> wrote:
> I attach a compatibility note that is clearly needed; adding this is
> an open item of mine for 9.5. This concerns foreign data wrappers and
> UPSERT.

Are you going to review this, Bruce? It is an open item for 9.5.

I would also like you to look into the other items I've highlighted.

-- 
Peter Geoghegan



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Sun, Jun 14, 2015 at 12:05:54PM +0100, Dean Rasheed wrote:
> On 11 June 2015 at 05:15, Bruce Momjian <bruce@momjian.us> wrote:
> > I have committed the first draft of the 9.5 release notes.  You can view
> > the output here:
> >
> >         http://momjian.us/pgsql_docs/release-9-5.html
> >
> 
> I think it's worth mentioning
> dcbf5948e12aa60b4d6ab65b6445897dfc971e01, probably under "General
> Performance". It's an optimisation, and also a user-visible change to
> the way LEAKPROOF works. Perhaps something like
> 
> Allow pushdown of non-leakproof functions into security barrier views
> if the function is not passed any arguments from the view.
> 
> Previously only functions marked as LEAKPROOF could be pushed down
> into security barrier views.

Sorry, just looking at this now.  Since RLS is new for 9.5, we wouldn't
mention the RLS change in the release notes because is is part of the
RLS new features, but we could mention the SB change --- the new text
would be:
Allow non-LEAKPROOF functions to be passed into security barrier viewsif the function does not reference any table
columns(Dean Rasheed)
 

However, this is usually a level of detail that I do not cover in the
release notes, so I need someone else to tell me it should be added.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Robert Haas
Date:
On Thu, Aug 6, 2015 at 10:33 AM, Bruce Momjian <bruce@momjian.us> wrote:
> On Sun, Jun 14, 2015 at 12:05:54PM +0100, Dean Rasheed wrote:
>> On 11 June 2015 at 05:15, Bruce Momjian <bruce@momjian.us> wrote:
>> > I have committed the first draft of the 9.5 release notes.  You can view
>> > the output here:
>> >
>> >         http://momjian.us/pgsql_docs/release-9-5.html
>> >
>>
>> I think it's worth mentioning
>> dcbf5948e12aa60b4d6ab65b6445897dfc971e01, probably under "General
>> Performance". It's an optimisation, and also a user-visible change to
>> the way LEAKPROOF works. Perhaps something like
>>
>> Allow pushdown of non-leakproof functions into security barrier views
>> if the function is not passed any arguments from the view.
>>
>> Previously only functions marked as LEAKPROOF could be pushed down
>> into security barrier views.
>
> Sorry, just looking at this now.  Since RLS is new for 9.5, we wouldn't
> mention the RLS change in the release notes because is is part of the
> RLS new features, but we could mention the SB change --- the new text
> would be:
>
>         Allow non-LEAKPROOF functions to be passed into security barrier views
>         if the function does not reference any table columns (Dean Rasheed)
>
> However, this is usually a level of detail that I do not cover in the
> release notes, so I need someone else to tell me it should be added.

+1 for including it.  That's a security-relevant backward incompatibility.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Fwd: 9.5 release notes

From
Jeff Janes
Date:
On Wed, Jun 10, 2015 at 9:15 PM, Bruce Momjian <bruce@momjian.us> wrote:
I have committed the first draft of the 9.5 release notes.  You can view
the output here:

        http://momjian.us/pgsql_docs/release-9-5.html

and it will eventually appear here:

        http://www.postgresql.org/docs/devel/static/release.html

I am ready to make suggested adjustments, though I am traveling for
conferences for the next ten days so there might a delay in my replies.


"Improve the speed of sorting character and numeric fields"

Here, does "numeric" refer specifically to the type "numeric", or to all fields which are numeric in nature?

If the former (which I think it is, as other numeric types are pass-by-value to start with), it should probably get some sgml markup.

(copy to list, accidentally sent just to Bruce initially)

Thanks,

Jeff

Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Aug  6, 2015 at 01:48:26PM -0400, Robert Haas wrote:
> On Thu, Aug 6, 2015 at 10:33 AM, Bruce Momjian <bruce@momjian.us> wrote:
> > On Sun, Jun 14, 2015 at 12:05:54PM +0100, Dean Rasheed wrote:
> >> On 11 June 2015 at 05:15, Bruce Momjian <bruce@momjian.us> wrote:
> >> > I have committed the first draft of the 9.5 release notes.  You can view
> >> > the output here:
> >> >
> >> >         http://momjian.us/pgsql_docs/release-9-5.html
> >> >
> >>
> >> I think it's worth mentioning
> >> dcbf5948e12aa60b4d6ab65b6445897dfc971e01, probably under "General
> >> Performance". It's an optimisation, and also a user-visible change to
> >> the way LEAKPROOF works. Perhaps something like
> >>
> >> Allow pushdown of non-leakproof functions into security barrier views
> >> if the function is not passed any arguments from the view.
> >>
> >> Previously only functions marked as LEAKPROOF could be pushed down
> >> into security barrier views.
> >
> > Sorry, just looking at this now.  Since RLS is new for 9.5, we wouldn't
> > mention the RLS change in the release notes because is is part of the
> > RLS new features, but we could mention the SB change --- the new text
> > would be:
> >
> >         Allow non-LEAKPROOF functions to be passed into security barrier views
> >         if the function does not reference any table columns (Dean Rasheed)
> >
> > However, this is usually a level of detail that I do not cover in the
> > release notes, so I need someone else to tell me it should be added.
>
> +1 for including it.  That's a security-relevant backward incompatibility.

Thanks, done.  I was not aware of the complexity of this issue.  Applied
patch attached.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + Everyone has their own god. +

Attachment

Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Fri, Jun 26, 2015 at 3:39 PM, Peter Geoghegan <pg@heroku.com> wrote:
> I attach a compatibility note that is clearly needed; adding this is
> an open item of mine for 9.5. This concerns foreign data wrappers and
> UPSERT.

Can you look at this please, Bruce?


-- 
Peter Geoghegan



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Fri, Jun 19, 2015 at 08:21:19PM +0200, Andres Freund wrote:
> Hi,
> 
> On 2015-06-11 00:15:21 -0400, Bruce Momjian wrote:
> > I have committed the first draft of the 9.5 release notes.  You can view
> > the output here:
> 
> So, I did a pass through master's state:
> 
> >      <listitem>
> >       <para>
> >        Add <link linkend="BRIN">Block Range Indexes</> (<acronym>BRIN</>)
> >        (Álvaro Herrera, Heikki Linnakangas, Emre Hasegeli)
> >       </para>
> >
> >       <para>
> >        <acronym>BRIN</> indexes are very compact and store the min/max
> >        values for a range of heap blocks.
> >       </para>
> >      </listitem>
> 
> Maybe we should mention that they're cheap to maintain?

I saw how you were able to get that idea into the release notes with
little additional text --- nice.  :-)  I couldn't figure out how to do
that.

> >      <listitem>
> >       <para>
> >        Improve in-memory hash performance (Tomas Vondra, Robert Haas)
> >       </para>
> >      </listitem>
> 
> "hash performance" is pretty general, there's lot of places where we use
> hashing that aren't affected.

Well, how many of our users even know what a hash is, or when we use
hashing internally, and care where the speedup is, or will understand
it?

The guiding principal is that these release notes are written for our
users, not for us.  I don't see any way to give more details here
without being confusing or overly verbose.

> >      <listitem>
> >       <para>
> >        Improve concurrency of <link linkend="guc-shared-buffers">shared
> >        buffer</> replacement (Robert Haas, Amit Kapila)
> >       </para>
> >      </listitem>
> 
> I think in the end that patch was enhanced to a significant degree by
> making it lockless in d72731a70450b. I think the three (?) involved
> patches should just combined under one entry.

Did you address this already in
c0b0501925eacbf2d9c10cd231bf8a14e7c9ef4c?  I can't tell.

> >      <listitem>
> >       <para>
> >        Improve concurrent locking and buffer scan performance (Andres
> >        Freund, Kevin Grittner)
> >       </para>
> >      </listitem>
> 
> If this is ab5194e6f, I don't think it makes sense to mention "buffer
> scan" - it's just any lwlock, and buffer locks aren't the primary
> benefit (ProcArrayLock, buffer mapping lock probably are that). I also
> don't think Kevin was involved?
> 
> I think ed127002d8 and 4b4b680c should be mentioned in this section as
> well. 4b4b680c will considerably reduce the per backend memory usage for
> servers with large shared buffers.

I think you addressed this, right?

> >    <sect4>
> >     <title>Server Settings</title>
> >
> >     <itemizedlist>
> >
> >      <listitem>
> >       <para>
> >        Replace <varname>checkpoint_segments</> with <link
> >        linkend="guc-min-wal-size"><varname>min_wal_size</></> and
> >        <link linkend="guc-max-wal-size"><varname>max_wal_size</></>
> >        (Heikki Linnakangas)
> >       </para>
> >
> >       <para>
> >        This allows the allocation of a large number of <acronym>WAL</>
> >        files without keeping them if they are not needed.
> >       </para>
> >      </listitem>
> 
> Hm. This affects performance significantly, should we also list it there?

Uh, I saw this as more of a configuration improvement in that you don't
need to waste lots of WAL files if you don't need them.

> >      <listitem>
> >       <para>
> >        Add <acronym>GUC</> <link
> >        linkend="guc-wal-compression"><varname>wal_compression</></> to
> >        enable compression of full page images stored in <acronym>WAL</>
> >        (Rahila Syed, Michael Paquier)
> >       </para>
> >      </listitem>
> 
> Also rather performance relevant?

Well, it is both, but we already have a configuration section, so it
seems most logical there.

> >      <listitem>
> >       <para>
> >        Archive <acronym>WAL</> files with suffix <literal>.partial</>
> >        during standby promotion (Heikki Linnakangas)
> >       </para>
> >      </listitem>
> 
> This should be expanded, will mention to Heikki. Possibly also need to
> be mentioned in the backward incompat section.

Uh, it seemed obscure enough to not mention it as a backward
incompatibility.  I am a little afraid of briefkly trying to explain why
we did it.

> >      <listitem>
> >       <para>
> >        Allow the <link linkend="pg-replication-origin-create">labeling</>
> >        of the origin of logical replication changes (Andres Freund)
> >       </para>
> >
> >       <para>
> >        This helps with change tracking.
> >       </para>
> >      </listitem>
> 
> I think it should be 'origin and progress'. The explanation should
> probably rather be 'This is helpful when implementing replication
> solutions" or something like it.

Done.

> >      <listitem>
> >       <para>
> >        Allow control of table <acronym>WAL</> logging after table creation
> >        with <link linkend="SQL-ALTERTABLE"><command>ALTER TABLE .. SET
> >        LOGGED / UNLOGGED</></> (Fabrízio de Royes Mello)
> >       </para>
> >      </listitem>
> 
> This sounds a bit confusing. Maybe "Allow to convert a WAL logged table
> to an UNLOGGED one, and the other way round"?

I saw you used you wording, but I changed it to this:
       Allow changing of the <acronym>WAL</acronym>       logging status of a table ater creation with <link
linkend="SQL-ALTERTABLE"><command>ALTERTABLE .. SET LOGGED /       UNLOGGED</></> (Fabrízio de Royes Mello)
 

> >    <sect4>
> >     <title>System Information Functions and Views</title>
> 
> I wonder if
> 
> >      <listitem>
> >       <para>
> >        Report the backends holding replication slots in <link
> >        linkend="catalog-pg-replication-slots"><structname>pg_replication_slots</></>
> >        (Craig Ringer)
> >       </para>
> >
> >       <para>
> >        The new output column is <structname>active_pid</>.
> >       </para>
> >      </listitem>
> 
> shouldn't be moved her?

OK, makes sense, I guess, though it fits in two categories so it is
unclear exactly which one is best.

> >      <listitem>
> >       <para>
> >        Allow <application>pg_dump</> to share a snapshot taken by another
> >        session using <option>--snapshot</> (Simon Riggs, Michael Paquier)
> >       </para>
> >
> >       <para>
> >        The remote snapshot must have been exported by
> >        <function>pg_export_snapshot()</> or been defined when creating
> >        a logical replication slot.
> 
> 'or exported by logical replication slot creation'?

OK, I used that now.

> >        This can be used by parallel
> >        <application>pg_dump</> to use a consistent snapshot across
> >        <application>pg_dump</> processes.
> >       </para>
> 
> What do you mean by this comment? Parallel pg_dump internally does all
> the snapshot stuff already, and it's independent of this option.

Sorry, parallel was unclear here. I was referring to another pg_dump
running in parallel, not pg_dump -j.  Updated text:
This can be used by another <application>pg_dump</> to use a sharea consistent snapshot across <application>pg_dump</>
processes.

> >      <listitem>
> >       <para>
> >        Change <application>pg_ctl</> default shutdown mode from
> >        <literal>smart</> to <literal>fast</> (Bruce Momjian)
> >       </para>
> >      </listitem>
> 
> Wonder if this should be listed as an incompatibility. This does have
> have impact on existing setups/scripts.

Oh, yes! This is a big incompatibility --- sorry I missed that ---
moved.

> >      <listitem>
> >       <para>
> >        Add basic atomics <acronym>API</> support (Andres Freund, Oskari
> >        Saarenmaa)
> >       </para>
> >      </listitem>
> 
> Out of fairness I think either Oskari shouldn't be listed, or Amit
> should as well. Amit surely has spent more time on the patch than
> Oskari.

I see you fixed this --- good.

> >      <listitem>
> >       <para>
> >        Add native compiler and memory barriers for <productname>Solaris
> >        Studio</> (Oskari Saarenmaa)
> >       </para>
> >
> >       <para>
> >        IS THIS PART OF ATOMICS?
> >       </para>
> >      </listitem>
> 
> Not really, barriers are independent. But I guess we could just combine
> it nonetheless.

I see you took care of this.

> >   <sect3>
> >    <title>Additional Modules</title>
> >
> >    <itemizedlist>
> >
> >     <listitem>
> >      <para>
> >       Add <link linkend="app-pgrewind"><application>pg_rewind</></>,
> >       which allows re-synchronizing a master server after failback
> >       (Heikki Linnakangas)
> >      </para>
> >     </listitem>
> 
> Hm. pg_rewind isn't in contrib, so is this the right place for this?

Tom moved it in this commit, 85c25fdbd7d624928bb8a1f1fd1e5043ec35dd74.
> I think 647248e3708, 4fe384bd85, 4f85fde8, 59f71a0d0 should also be

I couldn't look up 647248e3708, I got "unknown revision or path not in
the working tree."

For 4fe384bd85:
Process 'die' interrupts while reading/writing from the clientsocket.

I am unclear if this is something to mention, or just a rare event that
we now avoid.

For 4f85fde8:
Introduce and use infrastructure for interrupt processing duringclient reads.

I had the same feeling.  The same with 59f71a0d0:
Add a default local latch for use in signal handlers.

These feel very internal to me.

> mentioned. They're amongst the most invasive changes in 9.5, and they do
> have implications for developers and users.

Again, the question is whether it is general enough to inform our user
community about.

> f5ae3ba4/779fdcd should be listed as improving bitmap index scan
> performance.

That is these two:
Make tbm_add_tuples more efficient by caching the last acccessed page.Improve new caching logic in tbm_add_tuples().

Again, as mentioned above, are these of enough general interest?

I am sure I have made some mistakes above, so please let me know what
else I need to do.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Wed, Jun 24, 2015 at 11:34:27AM +0200, Andres Freund wrote:
> On 2015-06-23 21:08:36 -0400, Robert Haas wrote:
> > On Tue, Jun 23, 2015 at 5:48 PM, Kevin Grittner <kgrittn@ymail.com> wrote:
> > > Andres Freund <andres@anarazel.de> wrote:
> > >>>      <listitem>
> > >>>       <para>
> > >>>        Improve concurrent locking and buffer scan performance (Andres
> > >>>        Freund, Kevin Grittner)
> > >>>       </para>
> > >>>      </listitem>
> > >>
> > >> If this is ab5194e6f, I don't think it makes sense to mention "buffer
> > >> scan" - it's just any lwlock, and buffer locks aren't the primary
> > >> benefit (ProcArrayLock, buffer mapping lock probably are that). I also
> > >
> > >> don't think Kevin was involved?
> > >
> > > It seems likely that 2ed5b87f9 was combined with something else in
> > > this reference.  By reducing buffer pins and buffer content locking
> > > during btree index scans it shows a slight performance gain in
> > > btree scans and avoids some blocking of btree index vacuuming.
> 
> Oh. That's what it was combined with. I don't think it makes sense to
> throw these three items together into one note. Their benefit/risk
> potential is pretty different.

I believe Andres has made all these adjustments suggested.  If not,
please let me know.

> > I think maybe we should separate that back out.  The list needs to be
> > user-accessible, but if it's hard to understand what it's referring
> > to, that's not good either.
> 
> Yea. And if then Bruce goes and compares feature counts... :)

:-)

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Alvaro Herrera
Date:
Bruce Momjian wrote:
> On Fri, Jun 19, 2015 at 08:21:19PM +0200, Andres Freund wrote:

> > I think 647248e3708, 4fe384bd85, 4f85fde8, 59f71a0d0 should also be
> 
> I couldn't look up 647248e3708, I got "unknown revision or path not in
> the working tree."

a "6" is missing.  6647248e3708

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Aug  6, 2015 at 06:44:40PM -0300, Alvaro Herrera wrote:
> Bruce Momjian wrote:
> > On Fri, Jun 19, 2015 at 08:21:19PM +0200, Andres Freund wrote:
> 
> > > I think 647248e3708, 4fe384bd85, 4f85fde8, 59f71a0d0 should also be
> > 
> > I couldn't look up 647248e3708, I got "unknown revision or path not in
> > the working tree."
> 
> a "6" is missing.  6647248e3708

Thanks.  It was this one:
    Don't allow immediate interrupts during authentication anymore.

Falls into the same category, but if someone can write up an entry that
covers them all, I think it would make sense to add it.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Fri, Jun 19, 2015 at 09:44:04PM +0200, Andres Freund wrote:
> Hi,
> 
> On 2015-06-11 00:15:21 -0400, Bruce Momjian wrote:
> > I have committed the first draft of the 9.5 release notes.  You can view
> > the output here:
> 
> I'm looking through all the commits, checking which I think should
> possibly be mentioned additionally:
> - 9f03ca91 - Speed up CREATE INDEX by avoiding to copy index tuples

I can't see this information being useful to our general users.

> - 9da86753 - Reject duplicate column names in foreign key referenced-columns lists.
>   behavioural change, should be documented as intentional

Uh, well, I didn't think such uses were common, and we now issue a very
clear error message, so I didn't think it was worth mentioning.

> - 680513ab - Break out OpenSSL-specific code to separate files.
>   should perhaps be mentioned in the developer section.

I usually only mention code changes that would affect extension authors,
or massive changes.

> - 0709b7ee7 - Change the spinlock primitives to function as compiler barriers.
>   Significant behavioural change for developers.

Uh, again, not really for the group of developers I usually worry about
in the release notes.

> - 94028691 - Advance backend's advertised xmin more aggressively.
>   Pretty good feature imo.

Yeah, but again, do our general users understand enough to even
understand the explaination, and even if they do, do they care.

> - 5028f22f6 - Switch to CRC-32C in WAL and other places.
>   This might have compability impact in some external tools.

Agreed, text updated:
       Speed up <acronym>CRC</> (cyclic redundancy check) computations       and switch to CRC-32C (Abhijit Menon-Sen,
HeikkiLinnakangas)
 

> - 4f1b890b1 -
>   Unsure if this should be mentioned, at least it's a fairly
>   large change.

This is:
   Merge the various forms of transaction commit & abort records.

Again, much too internal-focused for our users to care.

> - 27846f02 - Optimize locking a tuple already locked by another subxact
>   Several users ran into this slowness, so I think we should mention the
>   optimization.

I usually don't mention speedup of edge cases.  There are problably a
few people who care about it though, but this will be read by thousands
of people.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Mon, Jun 22, 2015 at 09:14:01PM +0000, Rajeev rastogi wrote:
> On 11 June 2015 09:45, Bruce Momjian Wrote:
> 
> > 
> > I have committed the first draft of the 9.5 release notes.  You can
> > view the output here:
> > 
> >     http://momjian.us/pgsql_docs/release-9-5.html
> > 
> > and it will eventually appear here:
> > 
> >     http://www.postgresql.org/docs/devel/static/release.html
> > 
> > I am ready to make suggested adjustments, though I am traveling for
> > conferences for the next ten days so there might a delay in my replies.
> 
> Below performance improvement in the "General Performance" category is missing:
> 
>     Reduce btree scan overhead for < and > strategies
>     
>     For <, <=, > and >= strategies, mark the first scan key
>     as already matched if scanning in an appropriate direction.
>     If index tuple contains no nulls we can skip the first
>     re-check for each tuple.

While this is a nice 9.5 feature, it really is "btree is faster", and
users usually don't need to know that, unless the change is massive that
they would change their use of the feature.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Thu, Aug 6, 2015 at 3:06 PM, Bruce Momjian <bruce@momjian.us> wrote:
>> Below performance improvement in the "General Performance" category is missing:
>>
>>     Reduce btree scan overhead for < and > strategies
>>
>>     For <, <=, > and >= strategies, mark the first scan key
>>     as already matched if scanning in an appropriate direction.
>>     If index tuple contains no nulls we can skip the first
>>     re-check for each tuple.
>
> While this is a nice 9.5 feature, it really is "btree is faster", and
> users usually don't need to know that, unless the change is massive that
> they would change their use of the feature.

I think that Rajeev is entitled to be credited for his work,
especially as a less experienced contributor.

Sure, users are not likely to care too much about incremental progress
like this (however, I would be willing to bet that more users care
about this item than about existing items like "Make initdb issue a
warning about placing the data directory at the top of a file system
mount point"). IMV it is the role of the release notes to document
what went into a release fairly methodically. I often look at release
notes to determine what might have gone wrong in a part of the code
that I'm less familiar with, for example.

Users mostly only specifically care about one or two big ticket items,
and in any case are not likely to learn about them from the release
notes. The release notes seem shorter than previous years, and I don't
think it's because 9.5 is a smaller release.

-- 
Peter Geoghegan



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Aug  6, 2015 at 03:32:43PM -0700, Peter Geoghegan wrote:
> On Thu, Aug 6, 2015 at 3:06 PM, Bruce Momjian <bruce@momjian.us> wrote:
> >> Below performance improvement in the "General Performance" category is missing:
> >>
> >>     Reduce btree scan overhead for < and > strategies
> >>
> >>     For <, <=, > and >= strategies, mark the first scan key
> >>     as already matched if scanning in an appropriate direction.
> >>     If index tuple contains no nulls we can skip the first
> >>     re-check for each tuple.
> >
> > While this is a nice 9.5 feature, it really is "btree is faster", and
> > users usually don't need to know that, unless the change is massive that
> > they would change their use of the feature.
> 
> I think that Rajeev is entitled to be credited for his work,
> especially as a less experienced contributor.
> 
> Sure, users are not likely to care too much about incremental progress
> like this (however, I would be willing to bet that more users care
> about this item than about existing items like "Make initdb issue a
> warning about placing the data directory at the top of a file system
> mount point"). IMV it is the role of the release notes to document
> what went into a release fairly methodically. I often look at release
> notes to determine what might have gone wrong in a part of the code
> that I'm less familiar with, for example.
> 
> Users mostly only specifically care about one or two big ticket items,
> and in any case are not likely to learn about them from the release
> notes. The release notes seem shorter than previous years, and I don't
> think it's because 9.5 is a smaller release.

I am using the same criteria I have always used.  If you would like it
changed, we need to discuss it at a macro level, not for individual
cases where we feel someone didn't get enough _credit_.

And, I will restate this again, the release note are not for _us_, or
for _credit_ --- they are for our general users.  If you would like that
changed, you need to make a case for that change.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Thu, Aug 6, 2015 at 3:39 PM, Bruce Momjian <bruce@momjian.us> wrote:
> I am using the same criteria I have always used.  If you would like it
> changed, we need to discuss it at a macro level, not for individual
> cases where we feel someone didn't get enough _credit_.

I don't know how you can say that no *user* cares about Rajeev's
B-Tree contribution on the one hand, but on the other hand add items
about things like hash index crash-safety warnings, or an entire
section on obscure source code changes, with commentary about memory
ordering considerations, for example.

> And, I will restate this again, the release note are not for _us_, or
> for _credit_ --- they are for our general users.  If you would like that
> changed, you need to make a case for that change.

Certainly, that's mostly true. But if it was entirely true, then no
individual would be named on the release notes at all. That would be a
mistake, but not because any ordinary user would care one bit.

I am not going to comment further on this matter.

-- 
Peter Geoghegan



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Fri, Jun 26, 2015 at 03:39:19PM -0700, Peter Geoghegan wrote:
> On Wed, Jun 10, 2015 at 9:15 PM, Bruce Momjian <bruce@momjian.us> wrote:
> > I am ready to make suggested adjustments
> 
> I attach a compatibility note that is clearly needed; adding this is
> an open item of mine for 9.5. This concerns foreign data wrappers and
> UPSERT.

I though tabout this, and it is really an issue for FDW authors, not for
end users, so I put this text in the Source Code changes section:

+        <para>
+         Foreign tables can now take part in <command>INSERT ... ON CONFLICT
+         DO NOTHING</> queries (Peter Geoghegan, Heikki Linnakangas,
+         Andres Freund)
+        </para>
+
+        <para>
+         Foreign data wrappers must be modified to handle this.
+         <command>INSERT ... ON CONFLICT DO UPDATE</> is not supported on
+         foreign tables.
+        </para>
+       </listitem>

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Mon, Jun 29, 2015 at 05:05:59PM -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > I'm working on integrating the suggestions I made last week to the
> > release notes. Would anybody mind if I start to add commit ids in
> > comments, similar to what Tom has done for minor releases, to news
> > items?
> 
> +1.  Helps confirm which items are meant to correspond to which commits.
> 
> In case you didn't realize it already, the stuff I put into the minor
> release notes is from src/tools/git_changelog.  Dunno whether we need
> to use that exact format for major releases though; there's no need to
> identify branches in major release notes.

Gee, if I had known we wanted to do this for major releases I could have
done it at the time I created the 9.5 release notes.  It has to be
harder to do it after the fact.  Should I do it for 9.6?

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Tue, Jun 30, 2015 at 12:12:16AM +0200, Andres Freund wrote:
> On 2015-06-29 17:58:54 -0400, Tom Lane wrote:
> > Yeah we are.  The only places you'll find where we aren't formatting to 77
> > or 78 columns or so are where it would require breaking SGML tags in weird
> > places.
> 
> Which isn't exactly infrequent...
> 
> Anyway, how about:
> format:%cd [%h] %<(8,trunc)%cN: %<(49,trunc)%s
> 
> (which you can configure as pretty.pgmajor or so in .gitconfig btw.)

Should we add this to src/tools/git_changelog?  It currently uses "git
log --format=fuller --date=iso".

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Thu, Aug 6, 2015 at 6:08 PM, Bruce Momjian <bruce@momjian.us> wrote:
> I though tabout this, and it is really an issue for FDW authors, not for
> end users, so I put this text in the Source Code changes section:

I carefully considered where to put it, and chose the compatibility
section based on the precedent of this 9.4 item:

"Foreign data wrappers that support updating foreign tables must
consider the possible presence of AFTER ROW triggers (Noah Misch)"

I don't suppose it matters much, though. I can close out that open item now.

Thanks

-- 
Peter Geoghegan



Re: 9.5 release notes

From
Tom Lane
Date:
Bruce Momjian <bruce@momjian.us> writes:
> On Tue, Jun 30, 2015 at 12:12:16AM +0200, Andres Freund wrote:
>> Anyway, how about:
>> format:%cd [%h] %<(8,trunc)%cN: %<(49,trunc)%s
>> (which you can configure as pretty.pgmajor or so in .gitconfig btw.)

> Should we add this to src/tools/git_changelog?  It currently uses "git
> log --format=fuller --date=iso".

I don't think that format can easily be merged into what git_changelog
does: there would be no place to put branch or release annotations.
But it was a real PITA to add that format into the alpha2 release notes,
so we should probably think of a way to produce a suitable report.
        regards, tom lane



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Tue, Jun 30, 2015 at 09:00:44PM +0200, Andres Freund wrote:
> I've gone through the release notes and added comments referencing
> commits as discussed earlier. Additionally I've improved and added a
> bunch of items.
> 
> Further stuff that came up while looking:
> * 2014-09-25 [b0d81ad] Heikki..: Add -D option to specify data directory to pg_c..
>   new options, should possibly be documented?

As I remember it was added just for consistency with other tools and is
not expected to be used because the data directory is required, so
documenting it seems like it would encourage nonsensical use.

> * 2014-10-02 [3acc10c9] Robert..: Increase the number of buffer mapping partition..
>   should we mention this? This has been patched by a number of
>   downstream vendors and users, so it's probably worth calling out?

Uh, I am not sure why general users would care.

> * 2014-11-18 [606c012] Simon ..: Reduce btree scan overhead for < and > strategies
>   Worthy of a note in the performance section.

I commented on this already.

> * 2014-11-22 [eca2b9b] Andrew..: Rework echo_hidden for \sf and \ef from commit ..
>   Seems far too minor in comparison to other stuff left out.

This is a user-visible change.

> * 2014-11-07 [7516f52] Alvaro..: BRIN: Block Range Indexes
>   Looks to me like that should just be Alvaro.

OK, other names removed.

> * 2014-11-22 [b62f94c] Tom Lane: Allow simplification of EXISTS() subqueries con..
>   Planner change, I think it's good to mention those.

I have talked to Tom about this in the past and unless the item has a
user-visible componient, in general he doesn't think they should be
mentioned.  I added a bunch of planner items in the major release notes
years ago and he thought they should be removed, and they were.

> 
> * 2014-11-28 [e384ed6] Tom Lane: Improve typcache: cache negative lookup results..
>   Should perhaps, together with other cache enhancing entries, be mentioned?

Uh, I am not seeing it.

> * 2014-12-08 [519b075] Simon ..: Use GetSystemTimeAsFileTime directly in win32
>   2014-12-08 [8001fe6] Simon ..: Windows: use GetSystemTimePreciseAsFileTime if ..
>   Timer resolution isn't a unimportant thing for people using explain?

This all seemed very internals-only, e.g.:
   On most Windows systems this change will actually have no significant effect on   timestamp resolution as the system
timertick is typically between 1ms and 15ms   depending on what timer resolution currently running applications have
requested.You can check this with clockres.exe from sysinternals. Despite the   platform limiation this change still
permitscapture of finer timestamps where   the system is capable of producing them and it gets rid of an unnecessary
syscall.

Was I wrong?

> * 2014-12-12 [7e354ab] Andrew..: Add several generator functions for jsonb that ..
>   2015-05-12 [c694701] Andrew..: Additional functions and operators for jsonb
>   2015-05-31 [37def42] Andrew..: Rename jsonb_replace to jsonb_set and allow it ..
>   2014-12-12 [237a882] Andrew..: Add json_strip_nulls and jsonb_strip_nulls fun..
>   should probably be merged? Similar set of authors and too many similar release note
>   entries.

They all do different types of things, hance the different entries.

> * 2014-12-23 [d7ee82e] Alvaro..: Add SQL-callable pg_get_object_address
> * 2014-12-30 [a676201] Alvaro..: Add pg_identify_object_as_address
>   should be merged.

OK, merged.

> * 2015-01-13 [4bad60e] Andres..: Allow latches to wait for socket writability wi..
>   2015-01-14 [59f71a0] Andres..: Add a default local latch for use in signal han..
>   2015-01-17 [ff44fba] Andres..: Replace walsender's latch with the general shar..
>   2015-02-03 [387da18] Andres..: Use a nonblocking socket for FE/BE communicatio..
>   2015-02-03 [4f85fde] Andres..: Introduce and use infrastructure for interrupt ..
>   2015-02-03 [4fe384b] Andres..: Process 'die' interrupts while reading/writing ..
>   2015-02-03 [6647248] Andres..: Don't allow immediate interrupts during authent..
>   2015-02-03 [6753333] Andres..: Move deadlock and other interrupt handling in p..
>   2015-02-13 [80788a4] Heikki..: Simplify waiting logic in reading from /writin..

Again, I am willing to add an combined entry for these, but I can't
figure out accurate text for it.

> * 2015-01-17 [9402869] Heikki..: Advance backend's advertised xmin more aggressi..
>   This is a pretty big efficiency boon for systems with longer
>   nontrivial transactions.

What is the user-visible behavior here?  More aggressive cleanup?

> * 2015-01-29 [ed12700] Andres..: Align buffer descriptors to cache line boundari..
>   Maybe worthwhile to mention?

Uh, I think you and I worked on that.  Odd I didn't list it, but I don't
think it fit in with any user-visible behavior, and was rare in that it
only happened on larger systems.

> * 2015-02-16 [9e3ad1a] Tom Lane: Use fast path in plpgsql's RETURN/RETURN NEXT i..
>   2015-02-28 [e524cbd] Tom Lane: Track typmods in plpgsql expression evaluation ..
>   2015-03-04 [1345cc6] Tom Lane: Use standard casting mechanism to convert types..
>   2015-03-11 [21dcda2] Tom Lane: Allocate ParamListInfo once per plpgsql functio..
> 
>   Maybe reformulate to generalize the array performance into plpgsql and
>   mention arrays and RETURN? And combine with the casting change entry,
>   because that's also about performance?

Sure.  Can you suggest some text?

> * 2015-02-20 [09d8d11] Tom Lane: Use FLEXIBLE_ARRAY_MEMBER in a bunch more places.
>   2015-02-20 [5740be6] Tom Lane: Some more FLEXIBLE_ARRAY_MEMBER hacking.
>   2015-02-20 [e38b1eb] Tom Lane: Use FLEXIBLE_ARRAY_MEMBER in struct varlena.
>   2015-02-20 [c110eff] Tom Lane: Use FLEXIBLE_ARRAY_MEMBER in struct RecordIOData.
>   2015-02-20 [33a3b03] Tom Lane: Use FLEXIBLE_ARRAY_MEMBER in some more places.
>   2015-02-20 [33b2a2c] Tom Lane: Fix statically allocated struct with FLEXIBLE_A..
>   2015-02-21 [f2874fe] Tom Lane: Some more FLEXIBLE_ARRAY_MEMBER fixes.
>   2015-02-21 [e1a11d9] Tom Lane: Use FLEXIBLE_ARRAY_MEMBER for HeapTupleHeaderDa..
>   2015-02-21 [2e21121] Tom Lane: Use FLEXIBLE_ARRAY_MEMBER in a number of other ..
>   Maybe add a note in the source code section? Not sure if relevant enough.

I am thinking no.  Anyone else?

> * 2015-02-21 [b419865] Jeff D..: In array_agg(), don't create a new context for ..
>   Peformance improvement?

Again, I didn't think so.  We don't document all performance
improvements, only big ones that would change user behavior or choices,
in general.

> * 2015-02-27 [f65e827] Tom Lane: Invent a memory context reset/delete callback m..
>   Interesting for devs, maybe add?

No idea.

> * 2015-03-11 [b557226] Tom Lane: Improve planner's cost estimation in the presen..
>   2015-03-03 [b989619] Tom Lane: Fix cost estimation for indexscans on expensive..
>   2015-03-11 [f4abd02] Tom Lane: Support flattening of empty-FROM subqueries and..
>   2015-05-04 [2503982] Tom Lane: Improve procost estimates for some text search ..
>   Mention? Notes about planner changes rather helpful when trying to pin
>   down performance regression.s

Again, see above on when planner changes are mentioned.

> * 2015-03-18 [f9dead5] Alvaro..: Install shared libraries to bin/ in Windows und..
>   Shouldn't this be mentioned rather than unimportant stuff like

Uh, we do mention this:
       Install shared libraries to <filename>bin</> in <systemitem       class="osname">MS Windows</> (Peter
Eisentraut,Michael Paquier)
 

> * 2015-03-23 [e5f455f] Tom Lane: Apply table and domain CHECK constraints in nam..
>   I think this should be dropped, it has essentially no consequence to users.

Actually, the ordering of CHECK constraint checks was previously
indeterminate, and users might have been depending on the order of
failure, while now it is consistent --- this is a user-visible change.

> * 2015-03-24 [1d8198b] Bruce ..: Add support for ALTER TABLE IF EXISTS ... RENAM..
>   Should imo be wrapped together with the other IF EXISTS changes

Uh, there is only one IF EXIST entry;  there is an IF NOT EXISTS, but
that is different.

> * 2014-12-09 [ae4e688] Simon ..: Silence REINDEX
>   Doesn't seem to warrant a separate release note entry.

The original behavior now requires use of the new VERBOSE option.

> * 2015-05-19 [0b28ea7] Tom Lane: Avoid collation dependence in indexes of syste..
>   Doesn't seem to warrant a separate release note entry. We've left of
>   many more significant things.

It is a bug fix.

> * 2015-04-14 [9fa8b0e] Peter ..: Move pg_upgrade from contrib/ to src/bin/
>   2015-04-13 [81134af] Peter ..: Move pgbench from contrib/ to src/bin/
>   Should imo merged with all the other moved utilities.

These items already had their own release note sections.

> * 2014-07-14 [91f03ba] Noah M..: MSVC: Recognize PGFILEDESC in contrib and conv..
>   2015-04-16 [22d0053] Alvaro..: MSVC: install src/test/modules together with c..
>   Don't seem to warrant a release note entry.

User-visible changes.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Aug  6, 2015 at 10:21:29PM -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > On Tue, Jun 30, 2015 at 12:12:16AM +0200, Andres Freund wrote:
> >> Anyway, how about:
> >> format:%cd [%h] %<(8,trunc)%cN: %<(49,trunc)%s
> >> (which you can configure as pretty.pgmajor or so in .gitconfig btw.)
> 
> > Should we add this to src/tools/git_changelog?  It currently uses "git
> > log --format=fuller --date=iso".
> 
> I don't think that format can easily be merged into what git_changelog
> does: there would be no place to put branch or release annotations.
> But it was a real PITA to add that format into the alpha2 release notes,
> so we should probably think of a way to produce a suitable report.

Agreed.  If it is already output by git_changelog it would be much
easier to add them to the release notes.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: Fwd: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Aug  6, 2015 at 11:19:46AM -0700, Jeff Janes wrote:
> On Wed, Jun 10, 2015 at 9:15 PM, Bruce Momjian <bruce@momjian.us> wrote:
> 
>     I have committed the first draft of the 9.5 release notes.  You can view
>     the output here:
> 
>             http://momjian.us/pgsql_docs/release-9-5.html
> 
>     and it will eventually appear here:
> 
>             http://www.postgresql.org/docs/devel/static/release.html
> 
>     I am ready to make suggested adjustments, though I am traveling for
>     conferences for the next ten days so there might a delay in my replies.
> 
> 
> 
> "Improve the speed of sorting character and numeric fields"
> 
> Here, does "numeric" refer specifically to the type "numeric", or to all fields
> which are numeric in nature?
> 
> If the former (which I think it is, as other numeric types are pass-by-value to
> start with), it should probably get some sgml markup.

You are right, is only NUMERIC.  It was this commit:
commit abd94bcac4582903765be7be959d1dbc121df0d0Author: Robert Haas <rhaas@postgresql.org>Date:   Thu Apr 2 14:02:55
2015-0400    Use abbreviated keys for faster sorting of numeric datums.    Andrew Gierth, reviewed by Peter Geoghegan,
withfurther tweaks by me.
 

I changed it to <type>NUMERIC</>.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Aug  6, 2015 at 06:42:38PM -0700, Peter Geoghegan wrote:
> On Thu, Aug 6, 2015 at 6:08 PM, Bruce Momjian <bruce@momjian.us> wrote:
> > I though tabout this, and it is really an issue for FDW authors, not for
> > end users, so I put this text in the Source Code changes section:
> 
> I carefully considered where to put it, and chose the compatibility
> section based on the precedent of this 9.4 item:
> 
> "Foreign data wrappers that support updating foreign tables must
> consider the possible presence of AFTER ROW triggers (Noah Misch)"
> 
> I don't suppose it matters much, though. I can close out that open item now.

Oh, I think that 9.4 is in the wrong section, but good you researched it.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Robert Haas
Date:
On Thu, Aug 6, 2015 at 10:24 PM, Bruce Momjian <bruce@momjian.us> wrote:
>> * 2014-10-02 [3acc10c9] Robert..: Increase the number of buffer mapping partition..
>>   should we mention this? This has been patched by a number of
>>   downstream vendors and users, so it's probably worth calling out?
>
> Uh, I am not sure why general users would care.

Because it significantly improves performance on workloads that don't
fit into shared_buffers and have concurrency.

>> * 2014-07-14 [91f03ba] Noah M..: MSVC: Recognize PGFILEDESC in contrib and conv..
>>   2015-04-16 [22d0053] Alvaro..: MSVC: install src/test/modules together with c..
>>   Don't seem to warrant a release note entry.
>
> User-visible changes.

It seems really strange to me to say that things which improve
performance or change query plans are somehow not user-visible, but
applying a file description to Windows binaries is user-visible.  I am
willing to bet you a beverage of your choice that a lot more people
are going to notice the performance improvements and planner changes
than will ever notice that stuff.

(I realize now that compiling the release notes must be a somewhat
thankless task, so let me just say that I do appreciate the work
you've put into this very much and the comments above shouldn't be
understood to take anything away from that.  The fact that we don't
agree on what the criteria ought to be does not mean that I don't
appreciate you doing the work.)

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: 9.5 release notes

From
Andres Freund
Date:
On 2015-08-07 11:53:30 -0400, Robert Haas wrote:
> On Thu, Aug 6, 2015 at 10:24 PM, Bruce Momjian <bruce@momjian.us> wrote:
> >> * 2014-10-02 [3acc10c9] Robert..: Increase the number of buffer mapping partition..
> >>   should we mention this? This has been patched by a number of
> >>   downstream vendors and users, so it's probably worth calling out?
> >
> > Uh, I am not sure why general users would care.
> 
> Because it significantly improves performance on workloads that don't
> fit into shared_buffers and have concurrency.

Exactly.

> >> * 2014-07-14 [91f03ba] Noah M..: MSVC: Recognize PGFILEDESC in contrib and conv..
> >>   2015-04-16 [22d0053] Alvaro..: MSVC: install src/test/modules together with c..
> >>   Don't seem to warrant a release note entry.
> >
> > User-visible changes.
> 
> It seems really strange to me to say that things which improve
> performance or change query plans are somehow not user-visible, but
> applying a file description to Windows binaries is user-visible.  I am
> willing to bet you a beverage of your choice that a lot more people
> are going to notice the performance improvements and planner changes
> than will ever notice that stuff.

+ many.

> (I realize now that compiling the release notes must be a somewhat
> thankless task, so let me just say that I do appreciate the work
> you've put into this very much and the comments above shouldn't be
> understood to take anything away from that.  The fact that we don't
> agree on what the criteria ought to be does not mean that I don't
> appreciate you doing the work.)

Same here.



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Fri, Aug  7, 2015 at 11:53:30AM -0400, Robert Haas wrote:
> On Thu, Aug 6, 2015 at 10:24 PM, Bruce Momjian <bruce@momjian.us> wrote:
> >> * 2014-10-02 [3acc10c9] Robert..: Increase the number of buffer mapping partition..
> >>   should we mention this? This has been patched by a number of
> >>   downstream vendors and users, so it's probably worth calling out?
> >
> > Uh, I am not sure why general users would care.
> 
> Because it significantly improves performance on workloads that don't
> fit into shared_buffers and have concurrency.
> 
> >> * 2014-07-14 [91f03ba] Noah M..: MSVC: Recognize PGFILEDESC in contrib and conv..
> >>   2015-04-16 [22d0053] Alvaro..: MSVC: install src/test/modules together with c..
> >>   Don't seem to warrant a release note entry.
> >
> > User-visible changes.
> 
> It seems really strange to me to say that things which improve
> performance or change query plans are somehow not user-visible, but
> applying a file description to Windows binaries is user-visible.  I am
> willing to bet you a beverage of your choice that a lot more people
> are going to notice the performance improvements and planner changes
> than will ever notice that stuff.

Well, we could just throw a "Postgres 9.5 is faster" release note item
in there and call it a day.  ;-)  Of course, there are certain
performance items we have to list:  user-visible changes, e.g. new
features (BRIN), and faster behavior for common operations, i.e. things
that will cause users to try things that were too slow in the past.

On the opposite end, we have many changes that shave 1% off of query
execution time with no change in user behavior, which we probably don't
want to list.

So, the line is between those somewhere, and the question is where is
that line?  If two people think this item is above that line, then let's
add it.  If you can provide text, I can add it.  If you have others, we
can add those too.

What I _am_ saying is that you should use the same criteria I am using,
and just disagree on the place for the line, rather than use a different
criteria, which will lead to perpetual complaints.  We can change the
criteria, but that is a different discussion.

> (I realize now that compiling the release notes must be a somewhat
> thankless task, so let me just say that I do appreciate the work
> you've put into this very much and the comments above shouldn't be
> understood to take anything away from that.  The fact that we don't
> agree on what the criteria ought to be does not mean that I don't
> appreciate you doing the work.)

Considering the number of almost-arbitrary decisions I have to make to
write the major release notes, I am surprised at how few complaints I
get.  Of course, I have been clearly told by core that no one else wants
this job.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Andres Freund
Date:
On 2015-08-07 14:43:11 -0400, Bruce Momjian wrote:
> Well, we could just throw a "Postgres 9.5 is faster" release note item
> in there and call it a day.  ;-)

Based on my experience one of the prime reason people move to a new
version of postgres is performance. And it's not like 'faster!!!' is
really helpful for them to evaluate the benefits appropriately. A
scalability improvement isn't going to help somebody concerned with
single query performance. Somebody concerned about OLTP write
performance isn't going to be overly excited about the sort performance
improvements, but somebody doing OLAP style queries very much so.

The consequence of not listing that is that we're essentially asking our
users to read through all the changes between two releases. Something
that takes a long while even for people like you and me who are very
familiar with the project..


The *by far* biggest complaint about upgrades with postgres isn't binary
compatibility (even before pg_upgrade), it's not that there's minor
incompatibilities (at least not since 8.3, and maybe bytea_output). It's
that previously working queries don't work anymore. It's always just a
minority, but they're there. And it's very hard to figure out what's
up. Is it stats? Different settings? Planner changes? If we then don't
list changes to the planner, they're screwed.

In comparison to that just about nobody cares about the rest of the
update but new SQL level stuff and a few other major things? A new field
in EXPLAIN, debug_assertions read only,  effective_io_concurrency
settable without effect, VACUUM logs new one more data point, an
10+ year old obsolete undocumented option being removed, new icons for
binaries. They just don't matter.


With regard to the point about the number of buffer mappings: This has
forced peoples sites down. People have found this out themselves and
patched postgres. That's an entirely different league.

> What I _am_ saying is that you should use the same criteria I am using,
> and just disagree on the place for the line, rather than use a different
> criteria, which will lead to perpetual complaints.  We can change the
> criteria, but that is a different discussion.

We need to change that criteria then.

Andres



Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Fri, Aug 7, 2015 at 12:02 PM, Andres Freund <andres@anarazel.de> wrote:
> Based on my experience one of the prime reason people move to a new
> version of postgres is performance. And it's not like 'faster!!!' is
> really helpful for them to evaluate the benefits appropriately. A
> scalability improvement isn't going to help somebody concerned with
> single query performance. Somebody concerned about OLTP write
> performance isn't going to be overly excited about the sort performance
> improvements, but somebody doing OLAP style queries very much so.
>
> The consequence of not listing that is that we're essentially asking our
> users to read through all the changes between two releases. Something
> that takes a long while even for people like you and me who are very
> familiar with the project..

Well put. I don't see much of a downside to having smaller performance
improvements listed, provided they're towards the end of the
performance section.

It is true that many users don't really care much about performance
stuff, but among those that give the release notes any more than a
cursory look, I bet most care a lot. It's not as if the release notes
are the only way, or even the best way of learning about new features
for those with a more casual interest. Keeping the features list short
and sweet is more of a job for advocacy people that prepare press
releases and so on.

>> What I _am_ saying is that you should use the same criteria I am using,
>> and just disagree on the place for the line, rather than use a different
>> criteria, which will lead to perpetual complaints.  We can change the
>> criteria, but that is a different discussion.
>
> We need to change that criteria then.

I strongly agree.

-- 
Peter Geoghegan



Re: 9.5 release notes

From
Gavin Flower
Date:
On 08/08/15 06:43, Bruce Momjian wrote:
> On Fri, Aug  7, 2015 at 11:53:30AM -0400, Robert Haas wrote:
[...]
> Well, we could just throw a "Postgres 9.5 is faster" release note item
> in there and call it a day.  ;-)
Nah! Just say it is Much Shinier, I'll buy it.

Unfortunately, we have to have much more professional sounding reasons 
to upgrade, to tell our managers - so I guess you DO need more details...

[...]
>
>> (I realize now that compiling the release notes must be a somewhat
>> thankless task, so let me just say that I do appreciate the work
>> you've put into this very much and the comments above shouldn't be
>> understood to take anything away from that.  The fact that we don't
>> agree on what the criteria ought to be does not mean that I don't
>> appreciate you doing the work.)
> Considering the number of almost-arbitrary decisions I have to make to
> write the major release notes, I am surprised at how few complaints I
> get.  Of course, I have been clearly told by core that no one else wants
> this job.
>
All joking aside, I appreciate your efforts.  I read the release notes, 
even though currently I don't have an immediate need to use PostgreSQL.


Cheers,
Gavin



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Aug  6, 2015 at 10:35:50PM -0400, Bruce Momjian wrote:
> On Thu, Aug  6, 2015 at 06:42:38PM -0700, Peter Geoghegan wrote:
> > On Thu, Aug 6, 2015 at 6:08 PM, Bruce Momjian <bruce@momjian.us> wrote:
> > > I though tabout this, and it is really an issue for FDW authors, not for
> > > end users, so I put this text in the Source Code changes section:
> > 
> > I carefully considered where to put it, and chose the compatibility
> > section based on the precedent of this 9.4 item:
> > 
> > "Foreign data wrappers that support updating foreign tables must
> > consider the possible presence of AFTER ROW triggers (Noah Misch)"
> > 
> > I don't suppose it matters much, though. I can close out that open item now.
> 
> Oh, I think that 9.4 is in the wrong section, but good you researched it.

Actually, 9.4 and 9.5 are both in the right sections.  Users create
triggers, but only people modifying the source code create FDWs, so they
do belong in different sections.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
David Rowley
Date:

On 7 August 2015 at 14:24, Bruce Momjian <bruce@momjian.us> wrote:
On Tue, Jun 30, 2015 at 09:00:44PM +0200, Andres Freund wrote:
> * 2014-12-08 [519b075] Simon ..: Use GetSystemTimeAsFileTime directly in win32
>   2014-12-08 [8001fe6] Simon ..: Windows: use GetSystemTimePreciseAsFileTime if ..
>   Timer resolution isn't a unimportant thing for people using explain?

This all seemed very internals-only, e.g.:

    On most Windows systems this change will actually have no significant effect on
    timestamp resolution as the system timer tick is typically between 1ms and 15ms
    depending on what timer resolution currently running applications have
    requested. You can check this with clockres.exe from sysinternals. Despite the
    platform limiation this change still permits capture of finer timestamps where
    the system is capable of producing them and it gets rid of an unnecessary
    syscall.

Was I wrong?


This does have a user visible change. Timestamps are now likely to have 6 digits after the decimal point, if they're on a version of windows which supports GetSystemTimePreciseAsFileTime();

Master:

postgres=# select now();
              now
-------------------------------
 2015-08-09 01:14:01.959645+12
(1 row)

9.4.4
postgres=# select now();
            now
----------------------------
 2015-08-09 01:15:09.783+12
(1 row)

Regards

David Rowley

--
 David Rowley                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services
 

Re: 9.5 release notes

From
Bruce Momjian
Date:
On Fri, Aug  7, 2015 at 09:02:09PM +0200, Andres Freund wrote:
> With regard to the point about the number of buffer mappings: This has
> forced peoples sites down. People have found this out themselves and
> patched postgres. That's an entirely different league.

I was not aware of the magnitude of this issue.  9.5 release note item
attached and applied.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + Everyone has their own god. +

Attachment

Re: 9.5 release notes

From
Bruce Momjian
Date:
On Fri, Aug  7, 2015 at 09:02:09PM +0200, Andres Freund wrote:
> On 2015-08-07 14:43:11 -0400, Bruce Momjian wrote:
> > Well, we could just throw a "Postgres 9.5 is faster" release note item
> > in there and call it a day.  ;-)
> 
> Based on my experience one of the prime reason people move to a new
> version of postgres is performance. And it's not like 'faster!!!' is
> really helpful for them to evaluate the benefits appropriately. A
> scalability improvement isn't going to help somebody concerned with
> single query performance. Somebody concerned about OLTP write
> performance isn't going to be overly excited about the sort performance
> improvements, but somebody doing OLAP style queries very much so.
> 
> The consequence of not listing that is that we're essentially asking our
> users to read through all the changes between two releases. Something
> that takes a long while even for people like you and me who are very
> familiar with the project..

Well, considering I have used the same criteria for years, and I am only
now hearing complaints, I am unsure about the statement that our
existing criteria isn't generally meeting people's needs.

> The *by far* biggest complaint about upgrades with postgres isn't binary
> compatibility (even before pg_upgrade), it's not that there's minor
> incompatibilities (at least not since 8.3, and maybe bytea_output). It's
> that previously working queries don't work anymore. It's always just a
> minority, but they're there. And it's very hard to figure out what's
> up. Is it stats? Different settings? Planner changes? If we then don't
> list changes to the planner, they're screwed.

Well, if we do list them, is that going to help people?  You can say,
"well it might", but we are then in the same logic we use to decide on
adding GUC entries  --- we have to weigh the value of having the entry
vs the overhead of everyone reading the entry.  Now, in this case, it is
a one-time read vs. something that we will keep for years, but the basic
decision process is the same.

And, again, I will say, we are not writing this for ourselves, but for
the general user.

> In comparison to that just about nobody cares about the rest of the
> update but new SQL level stuff and a few other major things? A new field
> in EXPLAIN, debug_assertions read only,  effective_io_concurrency
> settable without effect, VACUUM logs new one more data point, an
> 10+ year old obsolete undocumented option being removed, new icons for
> binaries. They just don't matter.

Well, if we have user-visible behavior, and we don't tell them about it,
they are never going to be able to use it, so it is hard to see how we
could avoid mentioning those.

> > What I _am_ saying is that you should use the same criteria I am using,
> > and just disagree on the place for the line, rather than use a different
> > criteria, which will lead to perpetual complaints.  We can change the
> > criteria, but that is a different discussion.
> 
> We need to change that criteria then.

Then you need to start a new thread on that topic to get community
agreement that I can implement, and we can probably change 9.5 to match.

You might also want to address the fact we don't list all bug fixes in
the release notes either if the bug is a rare, minor event, and/or if
the new error message is sufficient communication to users.

One way of minimizing the downside of any new such entries is to have a
"Minor performance improvements" or "Internal performance improvements"
section in the release notes so people will realize they are not of the
same import as other items --- same for possible new bug fix listings.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Sat, Aug  8, 2015 at 02:49:21PM -0400, Bruce Momjian wrote:
> > > What I _am_ saying is that you should use the same criteria I am using,
> > > and just disagree on the place for the line, rather than use a different
> > > criteria, which will lead to perpetual complaints.  We can change the
> > > criteria, but that is a different discussion.
> > 
> > We need to change that criteria then.
> 
> Then you need to start a new thread on that topic to get community
> agreement that I can implement, and we can probably change 9.5 to match.
> 
> You might also want to address the fact we don't list all bug fixes in
> the release notes either if the bug is a rare, minor event, and/or if
> the new error message is sufficient communication to users.
> 
> One way of minimizing the downside of any new such entries is to have a
> "Minor performance improvements" or "Internal performance improvements"
> section in the release notes so people will realize they are not of the
> same import as other items --- same for possible new bug fix listings.

I have updated src/tools/RELEASE_CHANGES to document the criteria I use
to create the major release notes:
               o  new features and options               o  major performance improvements               o  bug fixes
forserious or common bugs               o  incompatibilities               o  major source code changes
 

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Sun, Aug  9, 2015 at 01:24:33AM +1200, David Rowley wrote:
> 
> On 7 August 2015 at 14:24, Bruce Momjian <bruce@momjian.us> wrote:
> 
>     On Tue, Jun 30, 2015 at 09:00:44PM +0200, Andres Freund wrote:
>     > * 2014-12-08 [519b075] Simon ..: Use GetSystemTimeAsFileTime directly in
>     win32
>     >   2014-12-08 [8001fe6] Simon ..: Windows: use
>     GetSystemTimePreciseAsFileTime if ..
>     >   Timer resolution isn't a unimportant thing for people using explain?
> 
>     This all seemed very internals-only, e.g.:
> 
>         On most Windows systems this change will actually have no significant
>     effect on
>         timestamp resolution as the system timer tick is typically between 1ms
>     and 15ms
>         depending on what timer resolution currently running applications have
>         requested. You can check this with clockres.exe from sysinternals.
>     Despite the
>         platform limiation this change still permits capture of finer
>     timestamps where
>         the system is capable of producing them and it gets rid of an
>     unnecessary
>         syscall.
> 
>     Was I wrong?
> 
> 
> 
> This does have a user visible change. Timestamps are now likely to have 6
> digits after the decimal point, if they're on a version of windows which
> supports GetSystemTimePreciseAsFileTime();
> 
> Master:
> 
> postgres=# select now();
>               now
> -------------------------------
>  2015-08-09 01:14:01.959645+12
> (1 row)
> 
> 9.4.4
> postgres=# select now();
>             now
> ----------------------------
>  2015-08-09 01:15:09.783+12
> (1 row)

Yes, this was already in the release notes:
       Allow higher-precision timestamp resolution on <systemitem       class="osname">Windows 8</> or <systemitem
class="osname">Windows      Server 2012</> and later Windows systems (Craig Ringer)
 

I am not sure why people were saying it was missing.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Arthur Silva
Date:


On Sat, Aug 8, 2015 at 11:04 PM, Bruce Momjian <bruce@momjian.us> wrote:
On Sun, Aug  9, 2015 at 01:24:33AM +1200, David Rowley wrote:
>
> On 7 August 2015 at 14:24, Bruce Momjian <bruce@momjian.us> wrote:
>
>     On Tue, Jun 30, 2015 at 09:00:44PM +0200, Andres Freund wrote:
>     > * 2014-12-08 [519b075] Simon ..: Use GetSystemTimeAsFileTime directly in
>     win32
>     >   2014-12-08 [8001fe6] Simon ..: Windows: use
>     GetSystemTimePreciseAsFileTime if ..
>     >   Timer resolution isn't a unimportant thing for people using explain?
>
>     This all seemed very internals-only, e.g.:
>
>         On most Windows systems this change will actually have no significant
>     effect on
>         timestamp resolution as the system timer tick is typically between 1ms
>     and 15ms
>         depending on what timer resolution currently running applications have
>         requested. You can check this with clockres.exe from sysinternals.
>     Despite the
>         platform limiation this change still permits capture of finer
>     timestamps where
>         the system is capable of producing them and it gets rid of an
>     unnecessary
>         syscall.
>
>     Was I wrong?
>
>
>
> This does have a user visible change. Timestamps are now likely to have 6
> digits after the decimal point, if they're on a version of windows which
> supports GetSystemTimePreciseAsFileTime();
>
> Master:
>
> postgres=# select now();
>               now
> -------------------------------
>  2015-08-09 01:14:01.959645+12
> (1 row)
>
> 9.4.4
> postgres=# select now();
>             now
> ----------------------------
>  2015-08-09 01:15:09.783+12
> (1 row)

Yes, this was already in the release notes:

        Allow higher-precision timestamp resolution on <systemitem
        class="osname">Windows 8</> or <systemitem class="osname">Windows
        Server 2012</> and later Windows systems (Craig Ringer)

I am not sure why people were saying it was missing.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + Everyone has their own god. +


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Are we landing pg_tgrm 1.2 in pg 9.5?

If yes (we should), up to an order of magnitude improvements is a worthy inclusion in the release notes.

--
Arthur Silva

Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Sat, Jun 13, 2015 at 3:53 PM, Peter Geoghegan <pg@heroku.com> wrote:
> I think we should really address this. Attached patch adds a new
> release note item for it. It also adds to the documentation that
> explains why users should prefer varchar(n)/text to character(n); the
> lack of abbreviated key support now becomes a huge disadvantage for
> character(n), whereas in previous versions the disadvantages were
> fairly minor.
>
> In passing, I updated the existing sort item to reflect that only
> varchar(n), text, and numeric benefit from the abbreviation
> optimization (not character types more generally + numeric), and added
> a note on the effectiveness of the abbreviation optimization alone.

A recent e-mail from Kaigai-san [1] reminded me of this item. I really
think this limitation of char(n) needs to be documented along the
lines I proposed here back in June. Benchmarks like TPC-H use char(n)
extensively, since it's faster in other systems. However, PostgreSQL
now has hugely inferior sort performance for that type as compared to
text/varchar(n). This needs to be highlighted.

[1]
http://www.postgresql.org/message-id/flat/CAM3SWZRRCs6KAyN-bDsh0_pG=8xm3fvcF1X9dLsVd3wVbt1pHw@mail.gmail.com#CAM3SWZRRCs6KAyN-bDsh0_pG=8xm3fvcF1X9dLsVd3wVbt1pHw@mail.gmail.com
-- 
Peter Geoghegan



Re: 9.5 release notes

From
Tom Lane
Date:
Arthur Silva <arthurprs@gmail.com> writes:
> Are we landing pg_tgrm 1.2 in pg 9.5?

No, we aren't.

And please don't quote 70 lines of unrelated stuff before making
your point.
        regards, tom lane



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Thu, Aug 20, 2015 at 04:07:36PM -0700, Peter Geoghegan wrote:
> On Sat, Jun 13, 2015 at 3:53 PM, Peter Geoghegan <pg@heroku.com> wrote:
> > I think we should really address this. Attached patch adds a new
> > release note item for it. It also adds to the documentation that
> > explains why users should prefer varchar(n)/text to character(n); the
> > lack of abbreviated key support now becomes a huge disadvantage for
> > character(n), whereas in previous versions the disadvantages were
> > fairly minor.
> >
> > In passing, I updated the existing sort item to reflect that only
> > varchar(n), text, and numeric benefit from the abbreviation
> > optimization (not character types more generally + numeric), and added
> > a note on the effectiveness of the abbreviation optimization alone.
>
> A recent e-mail from Kaigai-san [1] reminded me of this item. I really
> think this limitation of char(n) needs to be documented along the
> lines I proposed here back in June. Benchmarks like TPC-H use char(n)
> extensively, since it's faster in other systems. However, PostgreSQL
> now has hugely inferior sort performance for that type as compared to
> text/varchar(n). This needs to be highlighted.
>
> [1]
http://www.postgresql.org/message-id/flat/CAM3SWZRRCs6KAyN-bDsh0_pG=8xm3fvcF1X9dLsVd3wVbt1pHw@mail.gmail.com#CAM3SWZRRCs6KAyN-bDsh0_pG=8xm3fvcF1X9dLsVd3wVbt1pHw@mail.gmail.com

I have applied the attached patch to document this in the data type docs.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + Everyone has their own god. +

Attachment

Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Wed, Aug 26, 2015 at 7:33 AM, Bruce Momjian <bruce@momjian.us> wrote:
> I have applied the attached patch to document this in the data type docs.

Thanks, but shouldn't varchar/text also be mentioned in the release
notes, rather than "character fields"?

-- 
Peter Geoghegan



Re: 9.5 release notes

From
Bruce Momjian
Date:
On Wed, Aug 26, 2015 at 10:10:01AM -0700, Peter Geoghegan wrote:
> On Wed, Aug 26, 2015 at 7:33 AM, Bruce Momjian <bruce@momjian.us> wrote:
> > I have applied the attached patch to document this in the data type docs.
>
> Thanks, but shouldn't varchar/text also be mentioned in the release
> notes, rather than "character fields"?

Good point. I have applied the attached patch.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + Everyone has their own god. +

Attachment

Re: 9.5 release notes

From
Bruce Momjian
Date:
On Wed, Aug 26, 2015 at 02:47:14PM -0400, Bruce Momjian wrote:
> On Wed, Aug 26, 2015 at 10:10:01AM -0700, Peter Geoghegan wrote:
> > On Wed, Aug 26, 2015 at 7:33 AM, Bruce Momjian <bruce@momjian.us> wrote:
> > > I have applied the attached patch to document this in the data type docs.
> > 
> > Thanks, but shouldn't varchar/text also be mentioned in the release
> > notes, rather than "character fields"?
> 
> Good point. I have applied the attached patch.

I am sorry it so long for me to address this.  Peter brought it up in
June, but I just wasn't around to address it cleanly before now.  I am
glad he reminded me.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



Re: 9.5 release notes

From
Peter Geoghegan
Date:
On Wed, Aug 26, 2015 at 11:58 AM, Bruce Momjian <bruce@momjian.us> wrote:
> I am sorry it so long for me to address this.  Peter brought it up in
> June, but I just wasn't around to address it cleanly before now.  I am
> glad he reminded me.

Well, you got around to it eventually. Thanks.

-- 
Peter Geoghegan