Thread: PG 12 draft release notes
I have posted a draft copy of the PG 12 release notes here:
http://momjian.us/pgsql_docs/release-12.html
They are committed to git. It includes links to the main docs, where
appropriate. Our official developer docs will rebuild in a few hours.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
Bruce, I noticed that jsonpath in your version is mentioned only in functions chapter, but commit 72b6460336e86ad5cafd3426af6013c7d8457367 is about implementation of SQL-2016 standard. We implemented JSON Path language as a jsonpath datatype with a bunch of support functions, our implementation supports 14 out of 15 features and it's the most complete implementation (we compared oracle, mysql and ms sql). On Sat, May 11, 2019 at 11:33 PM Bruce Momjian <bruce@momjian.us> wrote: > > I have posted a draft copy of the PG 12 release notes here: > > http://momjian.us/pgsql_docs/release-12.html > > They are committed to git. It includes links to the main docs, where > appropriate. Our official developer docs will rebuild in a few hours. > > -- > Bruce Momjian <bruce@momjian.us> http://momjian.us > EnterpriseDB http://enterprisedb.com > > + As you are, so once was I. As I am, so you will be. + > + Ancient Roman grave inscription + > > -- Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
On Sun, May 12, 2019 at 10:00:38AM +0300, Oleg Bartunov wrote: > Bruce, > > I noticed that jsonpath in your version is mentioned only in functions > chapter, but commit > 72b6460336e86ad5cafd3426af6013c7d8457367 is about implementation of > SQL-2016 standard. We implemented JSON Path language as a jsonpath > datatype with a bunch of support functions, our implementation > supports 14 out of 15 features and it's the most complete > implementation (we compared oracle, mysql and ms sql). Glad you asked. I was very confused about why a data type was added for a new path syntax. Is it a new storage format for JSON, or something else? I need help on this. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Hi Bruce, On 5/11/19 4:33 PM, Bruce Momjian wrote: > I have posted a draft copy of the PG 12 release notes here: > > http://momjian.us/pgsql_docs/release-12.html > > They are committed to git. It includes links to the main docs, where > appropriate. Our official developer docs will rebuild in a few hours. Thank you for working on this, I know it's a gargantuan task. I have a small modification for a section entitled "Source Code" which is a repeat of the previous section. Based on the bullet points in that part, I thought "Documentation" might be a more appropriate name; please see attached. Thanks, Jonathan
Attachment
On Sun, 12 May 2019 at 08:33, Bruce Momjian <bruce@momjian.us> wrote: > > I have posted a draft copy of the PG 12 release notes here: > > http://momjian.us/pgsql_docs/release-12.html I noticed a couple of different spellings of Álvaro's name. Loading the file line by line into a table and crudely performing: select distinct name from (select trim(regexp_split_to_table(substring(a, '\((.*?)\)'),',')) as name from r where a like '%(%)%')a order by name; turned up variations in Michaël and Pavel's names The attached fixes. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Attachment
On Sun, May 12, 2019 at 10:49:07AM -0400, Jonathan Katz wrote: > Hi Bruce, > > On 5/11/19 4:33 PM, Bruce Momjian wrote: > > I have posted a draft copy of the PG 12 release notes here: > > > > http://momjian.us/pgsql_docs/release-12.html > > > > They are committed to git. It includes links to the main docs, where > > appropriate. Our official developer docs will rebuild in a few hours. > > Thank you for working on this, I know it's a gargantuan task. > > I have a small modification for a section entitled "Source Code" which > is a repeat of the previous section. Based on the bullet points in that > part, I thought "Documentation" might be a more appropriate name; please > see attached. Yes, I saw that myself and just updated it. Thanks. --------------------------------------------------------------------------- > > Thanks, > > Jonathan > diff --git a/doc/src/sgml/release-12.sgml b/doc/src/sgml/release-12.sgml > index 5f5d1da33d..1bbd91a02e 100644 > --- a/doc/src/sgml/release-12.sgml > +++ b/doc/src/sgml/release-12.sgml > @@ -2617,7 +2617,7 @@ Require a C99-supported compiler, and <acronym>MSCV</acronym> 2013 or later on < > </sect3> > > <sect3> > - <title>Source Code</title> > + <title>Documentation</title> > > <itemizedlist> > -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Sun, May 12, 2019 at 09:49:40AM -0400, Bruce Momjian wrote: > On Sun, May 12, 2019 at 10:00:38AM +0300, Oleg Bartunov wrote: > > Bruce, > > > > I noticed that jsonpath in your version is mentioned only in functions > > chapter, but commit > > 72b6460336e86ad5cafd3426af6013c7d8457367 is about implementation of > > SQL-2016 standard. We implemented JSON Path language as a jsonpath > > datatype with a bunch of support functions, our implementation > > supports 14 out of 15 features and it's the most complete > > implementation (we compared oracle, mysql and ms sql). > > Glad you asked. I was very confused about why a data type was added for > a new path syntax. Is it a new storage format for JSON, or something > else? I need help on this. I talked to Alexander Korotkov on chat about this. The data types are used as arguments to the functions, similar to how tsquery and tsvector are used for full text search. Therefore, the data types are not really useful on their own, but as support for path functions. However, path functions are more like JSON queries, rather than traditional functions, so it odd to list them under functions, but there isn't a more reasonable place to put them. Alexander researched how we listed full text search in the release notes that added the feature, but we had "General" category at that time that we don't have now. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Mon, May 13, 2019 at 01:37:25PM +1200, David Rowley wrote: > On Sun, 12 May 2019 at 08:33, Bruce Momjian <bruce@momjian.us> wrote: > > > > I have posted a draft copy of the PG 12 release notes here: > > > > http://momjian.us/pgsql_docs/release-12.html > > I noticed a couple of different spellings of Álvaro's name. Loading > the file line by line into a table and crudely performing: > > select distinct name from (select > trim(regexp_split_to_table(substring(a, '\((.*?)\)'),',')) as name > from r where a like '%(%)%')a order by name; > > turned up variations in Michaël and Pavel's names That is a big help, thanks, applied. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Hello Bruce, > I have posted a draft copy of the PG 12 release notes here: > > http://momjian.us/pgsql_docs/release-12.html > > They are committed to git. It includes links to the main docs, where > appropriate. Our official developer docs will rebuild in a few hours. Pgbench entry "Improve pgbench error reporting with clearer messages and return codes" is by Peter Eisentraut, not me. I just reviewed it. -- Fabien.
On Mon, May 13, 2019 at 6:52 AM Bruce Momjian <bruce@momjian.us> wrote: > > On Sun, May 12, 2019 at 09:49:40AM -0400, Bruce Momjian wrote: > > On Sun, May 12, 2019 at 10:00:38AM +0300, Oleg Bartunov wrote: > > > Bruce, > > > > > > I noticed that jsonpath in your version is mentioned only in functions > > > chapter, but commit > > > 72b6460336e86ad5cafd3426af6013c7d8457367 is about implementation of > > > SQL-2016 standard. We implemented JSON Path language as a jsonpath > > > datatype with a bunch of support functions, our implementation > > > supports 14 out of 15 features and it's the most complete > > > implementation (we compared oracle, mysql and ms sql). > > > > Glad you asked. I was very confused about why a data type was added for > > a new path syntax. Is it a new storage format for JSON, or something > > else? I need help on this. > > I talked to Alexander Korotkov on chat about this. The data types are > used as arguments to the functions, similar to how tsquery and tsvector > are used for full text search. > > Therefore, the data types are not really useful on their own, but as > support for path functions. However, path functions are more like JSON > queries, rather than traditional functions, so it odd to list them under > functions, but there isn't a more reasonable place to put them. > > Alexander researched how we listed full text search in the release notes > that added the feature, but we had "General" category at that time that > we don't have now. I attached slide about our Jsonpath implementation in Postgres, it summarizes the reasons to have jsonpath data type. But my point was: JSON Path is a part of SQL-2016 standard and I think it's worth to mention it, not just a set of jsonb functions. > > -- > Bruce Momjian <bruce@momjian.us> http://momjian.us > EnterpriseDB http://enterprisedb.com > > + As you are, so once was I. As I am, so you will be. + > + Ancient Roman grave inscription + -- Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
Attachment
On Mon, May 13, 2019 at 08:41:25AM +0200, Fabien COELHO wrote: > > Hello Bruce, > > > I have posted a draft copy of the PG 12 release notes here: > > > > http://momjian.us/pgsql_docs/release-12.html > > > > They are committed to git. It includes links to the main docs, where > > appropriate. Our official developer docs will rebuild in a few hours. > > Pgbench entry "Improve pgbench error reporting with clearer messages and > return codes" is by Peter Eisentraut, not me. I just reviewed it. Thanks, fixed. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Mon, May 13, 2019 at 10:08:57AM +0300, Oleg Bartunov wrote: > On Mon, May 13, 2019 at 6:52 AM Bruce Momjian <bruce@momjian.us> wrote: > > > > On Sun, May 12, 2019 at 09:49:40AM -0400, Bruce Momjian wrote: > > > On Sun, May 12, 2019 at 10:00:38AM +0300, Oleg Bartunov wrote: > > > > Bruce, > > > > > > > > I noticed that jsonpath in your version is mentioned only in functions > > > > chapter, but commit > > > > 72b6460336e86ad5cafd3426af6013c7d8457367 is about implementation of > > > > SQL-2016 standard. We implemented JSON Path language as a jsonpath > > > > datatype with a bunch of support functions, our implementation > > > > supports 14 out of 15 features and it's the most complete > > > > implementation (we compared oracle, mysql and ms sql). > > > > > > Glad you asked. I was very confused about why a data type was added for > > > a new path syntax. Is it a new storage format for JSON, or something > > > else? I need help on this. > > > > I talked to Alexander Korotkov on chat about this. The data types are > > used as arguments to the functions, similar to how tsquery and tsvector > > are used for full text search. > > > > Therefore, the data types are not really useful on their own, but as > > support for path functions. However, path functions are more like JSON > > queries, rather than traditional functions, so it odd to list them under > > functions, but there isn't a more reasonable place to put them. > > > > Alexander researched how we listed full text search in the release notes > > that added the feature, but we had "General" category at that time that > > we don't have now. > > I attached slide about our Jsonpath implementation in Postgres, it > summarizes the reasons to have jsonpath data type. But my point was: > JSON Path is a part of SQL-2016 standard and I think it's worth to > mention it, not just a set of jsonb functions. So, are you suggesting we mention the jsonpath data type in the Data Type section, even though it is useless without jsonpath, which is in another section, or are you suggesting to move the jsonpath item to the Data Type section? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Sat, May 11, 2019, at 22:33, Bruce Momjian wrote: > http://momjian.us/pgsql_docs/release-12.html There is a typo in E.1.3.1.1.: > Expressions are evaluated at table partitioned table creation time. First "table" seems to be excessive. Regards, Nick.
On Tue, May 14, 2019 at 11:53:23AM +0200, nickb wrote: > On Sat, May 11, 2019, at 22:33, Bruce Momjian wrote: > > http://momjian.us/pgsql_docs/release-12.html > > There is a typo in E.1.3.1.1.: > > Expressions are evaluated at table partitioned table creation time. > First "table" seems to be excessive. Yep, fixed, thanks. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Hi, Note that I've added a few questions to individuals involved with specific points. If you're in the To: list, please search for your name. On 2019-05-11 16:33:24 -0400, Bruce Momjian wrote: > I have posted a draft copy of the PG 12 release notes here: > > http://momjian.us/pgsql_docs/release-12.html > They are committed to git. Thanks! <title>Migration to Version 12</title> There's a number of features in the compat section that are more general improvements with a side of incompatibility. Won't it be confusing to e.g. have have the ryu floating point conversion speedups in the compat section, but not in the "General Performance" section? <para> Remove the special behavior of <link linkend="datatype-oid">OID</link> columns (Andres Freund, John Naylor) </para> Should we mention that tables with OIDs have to have their oids removed before they can be upgraded? <para> Refactor <link linkend="functions-geometry">geometric functions</link> and operators (Emre Hasegeli) </para> <para> This could lead to more accurate, but slightly different, results from previous releases. </para> </listitem> <listitem> <!-- Author: Tomas Vondra <tomas.vondra@postgresql.org> 2018-08-16 [c4c340088] Use the built-in float datatypes to implement geometric --> <para> Restructure <link linkend="datatype-geometric">geometric types</link> to handle NaN, underflow, overflow and division by zero more consistently (Emre Hasegeli) </para> </listitem> <listitem> <!-- Author: Tomas Vondra <tomas.vondra@postgresql.org> 2018-09-26 [2e2a392de] Fix problems in handling the line data type --> <para> Improve behavior and error reporting for the <link linkend="datatype-geometric">line data type</link> (Emre Hasegeli) </para> </listitem> Is that sufficient explanation? Feels like we need to expand a bit more. In particular, is it possible that a subset of the changes here require reindexing? Also, aren't three different entries a bit too much? <para> Avoid performing unnecessary rounding of <link linkend="datatype-float"><type>REAL</type></link> and <type>DOUBLE PRECISION</type> values (Andrew Gierth) </para> <para> This dramatically speeds up processing of floating-point values but causes additional trailing digits to potentially be displayed. Users wishing to have output that is rounded to match the previous behavior can set <link linkend="guc-extra-float-digits"><literal>extra_float_digits=0</literal></link>, which is no longer the default. </para> </listitem> Isn't it exactly the *other* way round? *Previously* we'd output additional trailing digits. The new algorithm instead will instead have *exactly* the required number of digits? <listitem> <!-- Author: Tom Lane <tgl@sss.pgh.pa.us> 2019-02-11 [1d92a0c9f] Redesign the partition dependency mechanism. --> <para> Improve handling of partition dependency (Tom Lane) </para> <para> This prevents the creation of inconsistent partition hierarchies in rare cases. </para> </listitem> That seems not very informative for users? <listitem> <!-- Author: Alexander Korotkov <akorotkov@postgresql.org> 2018-07-28 [d2086b08b] Reduce path length for locking leaf B-tree pages during Author: Peter Geoghegan <pg@bowt.ie> 2019-03-25 [f21668f32] Add "split after new tuple" nbtree optimization. --> <para> Improve speed of btree index insertions (Peter Geoghegan, Alexander Korotkov) </para> <para> The new code improves the space-efficiency of page splits, reduces locking overhead, and gives better performance for <command>UPDATE</command>s and <command>DELETE</command>s on indexes with many duplicates. </para> </listitem> <listitem> <!-- Author: Peter Geoghegan <pg@bowt.ie> 2019-03-20 [dd299df81] Make heap TID a tiebreaker nbtree index column. Author: Peter Geoghegan <pg@bowt.ie> 2019-03-20 [fab250243] Consider secondary factors during nbtree splits. --> <para> Have new btree indexes sort duplicate index entries in heap-storage order (Peter Geoghegan, Heikki Linnakangas) </para> <para> Indexes <application>pg_upgraded</application> from previous releases will not have this ordering. </para> </listitem> I'm not sure that the grouping here is quite right. And the second entry probably should have some explanation about the benefits? <listitem> <!-- Author: Peter Eisentraut <peter_e@gmx.net> 2018-11-14 [1b5d797cd] Lower lock level for renaming indexes --> <para> Reduce locking requirements for index renaming (Peter Eisentraut) </para> </listitem> Should we specify the newly required lock level? Because it's quire relevant for users what exactly they're now able to do concurrently in operation? <para> Allow <link linkend="queries-with">common table expressions</link> (<acronym>CTE</acronym>) to be inlined in later parts of the query (Andreas Karlsson, Andrew Gierth, David Fetter, Tom Lane) </para> <para> Specifically, <acronym>CTE</acronym>s are inlined if they are not recursive and are referenced only once later in the query. Inlining can be prevented by specifying <literal>MATERIALIZED</literal>, and forced by specifying <literal>NOT MATERIALIZED</literal>. Previously, <acronym>CTE</acronym>s were never inlined and were always evaluated before the rest of the query. </para> Hm. Is it actually correct to say that "were always evaluated before the rest of the query."? My understanding is that that's not actually how they behaved. Materialization for CTE scans was on-demand (i.e. when needed by a CTE scan), and even for DML CTEs we'd only force the underlying query to completion at the end of the query? <listitem> <!-- Author: Tom Lane <tgl@sss.pgh.pa.us> 2019-02-09 [1fb57af92] Create the infrastructure for planner support functions. --> <para> Add support for <link linkend="sql-createfunction">function selectivity</link> (Tom Lane) </para> </listitem> Hm, that message doesn't seem like an accurate description of that commit (if anything it's a391ff3c?). Given that it all requires C hackery, perhaps we ought to move it to the source code section? And isn't the most important part of this set of changes commit 74dfe58a5927b22c744b29534e67bfdd203ac028 Author: Tom Lane <tgl@sss.pgh.pa.us> Date: 2019-02-11 21:26:08 -0500 Allow extensions to generate lossy index conditions. <listitem> <!-- Author: Tomas Vondra <tomas.vondra@postgresql.org> 2019-01-29 [36a1281f8] Separate per-batch and per-tuple memory contexts in COPY Author: Heikki Linnakangas <heikki.linnakangas@iki.fi> 2019-01-25 [9556aa01c] Use single-byte Boyer-Moore-Horspool search even with mu Author: Andres Freund <andres@anarazel.de> 2019-01-26 [a9c35cf85] Change function call information to be variable length. --> <para> Greatly reduce memory consumption of <xref linkend="sql-copy"/> and function calls (Andres Freund, Tomas Vondra, Tom Lane) </para> </listitem> Grouping these three changes together makes no sense to me. I think the first commit just ought not to be mentioned separately, it's just a fix for a memory leak in 31f3817402, essentially a 12 only bugfix? The second commit is about position() etc, which seems not to match that description either? The third is probably more appropriate to be in the source code section. While it does speed up function calls a bit (in particular plpgsql which is very function call heavy), it also is a breaking change for some external code? Not sure why Tom is listed with this entry? <listitem> <!-- Author: Heikki Linnakangas <heikki.linnakangas@iki.fi> 2019-01-25 [9556aa01c] Use single-byte Boyer-Moore-Horspool search even with mu --> <para> Improve search performance for multi-byte characters (Heikki Linnakangas) </para> </listitem> That's the second reference to the commit. I suspect this is much better separate, so I'd just remove it from above. <listitem> <!-- Author: Stephen Frost <sfrost@snowman.net> 2019-04-02 [4d0e994ee] Add support for partial TOAST decompression --> <para> Allow <link linkend="storage-toast"><literal>TOAST</literal></link> values to be minimally decompressed (Paul Ramsey) </para> I'd s/minimal/partial/ - I don't think the code guarantees anything about it being minimal? And "minimally decompressed" also is somewhat confusing, because it sounds like it's about the compression quality rather than only decompressing part of the data. <listitem> <!-- Author: Michael Paquier <michael@paquier.xyz> 2018-08-10 [f841ceb26] Improve TRUNCATE by avoiding early lock queue --> <para> Prevent <xref linkend="sql-truncate"/> from requesting a lock on tables for which it lacks permission (Michaël Paquier) </para> <para> This prevents unauthorized locking delays. </para> </listitem> <listitem> <!-- Author: Michael Paquier <michael@paquier.xyz> 2018-08-27 [a556549d7] Improve VACUUM and ANALYZE by avoiding early lock queue --> <para> Prevent <command>VACUUM</command> and <command>ANALYZE</command> from requesting a lock on tables for which it lacks permission (Michaël Paquier) </para> <para> This prevents unauthorized locking delays. </para> </listitem> I don't think this should be in the <title><acronym>Authentication</acronym></title> section. Also perhaps, s/it/the user/, or "the caller"? <listitem> <!-- Author: Tom Lane <tgl@sss.pgh.pa.us> 2019-03-10 [cbccac371] Reduce the default value of autovacuum_vacuum_cost_delay --> <para> Reduce the default value of <xref linkend="guc-autovacuum-vacuum-cost-delay"/> to 2ms (Tom Lane) </para> </listitem> I think this needs to explain that this can increase autovacuum's IO throughput considerably. <listitem> <!-- Author: Tom Lane <tgl@sss.pgh.pa.us> 2019-03-10 [caf626b2c] Convert [autovacuum_]vacuum_cost_delay into floating-poi --> <para> Allow <xref linkend="guc-vacuum-cost-delay"/> to specify sub-millisecond delays (Tom Lane) </para> <para> Floating-point values can also now be specified. </para> </listitem> And this should be merged with the previous entry? <listitem> <!-- Author: Tom Lane <tgl@sss.pgh.pa.us> 2019-03-10 [caf626b2c] Convert [autovacuum_]vacuum_cost_delay into floating-poi --> <para> Allow time-based server variables to use <link linkend="config-setting">micro-seconds</link> (us) (Tom Lane) </para> </listitem> <listitem> <!-- Author: Tom Lane <tgl@sss.pgh.pa.us> 2019-03-11 [1a83a80a2] Allow fractional input values for integer GUCs, and impr --> <para> Allow fractional input for integer server variables (Tom Lane) </para> <para> For example, <command>SET work_mem = '30.1GB'</command>. </para> </listitem> <listitem> <!-- Author: Tom Lane <tgl@sss.pgh.pa.us> 2019-03-10 [caf626b2c] Convert [autovacuum_]vacuum_cost_delay into floating-poi --> <para> Allow units to be specified for floating-point server variables (Tom Lane) </para> </listitem> Can't we combine these? Seems excessively detailed in comparison to the rest of the entries. <listitem> <!-- Author: Peter Eisentraut <peter@eisentraut.org> 2019-01-11 [ff8530605] Add value 'current' for recovery_target_timeline --> <para> Add an explicit value of <literal>current</literal> for <xref linkend="guc-recovery-target-time"/> (Peter Eisentraut) </para> </listitem> Seems like this should be combined with the earlier "Cause recovery to advance to the latest timeline by default" entry. <listitem> <!-- Author: Peter Eisentraut <peter@eisentraut.org> 2019-03-30 [fc22b6623] Generated columns --> <para> Add support for <link linkend="sql-createtable">generated columns</link> (Peter Eisentraut) </para> <para> Rather than storing a value only at row creation time, generated columns are also modified during updates, and can reference other table columns. </para> </listitem> I find this description confusing. How about cribbing from the commit? Roughly like This allows creating columns that are computed from expressions, including references to other columns in the same table, rather than having to be specified by the inserter/updater. Think we also ought to mention that this is only stored generated columns, given that the SQL feature also includes virtual columns? <listitem> <!-- Author: Fujii Masao <fujii@postgresql.org> 2019-04-08 [119dcfad9] Add vacuum_truncate reloption. Author: Fujii Masao <fujii@postgresql.org> 2019-05-08 [b84dbc8eb] Add TRUNCATE parameter to VACUUM. --> <para> Add <xref linkend="sql-vacuum"/> and <command>CREATE TABLE</command> options to prevent <command>VACUUM</command> from truncating trailing empty pages (Tsunakawa Takayuki) </para> <para> The options are <varname>vacuum_truncate</varname> and <varname>toast.vacuum_truncate</varname>. This reduces vacuum locking requirements. </para> </listitem> Maybe add something like: "This can be helpful to avoid query cancellations on standby that are not avoided by hot_standby_feedback."? <listitem> <!-- Author: Robert Haas <rhaas@postgresql.org> 2019-04-04 [a96c41fee] Allow VACUUM to be run with index cleanup disabled. --> <para> Allow vacuum to avoid index cleanup with the <literal>INDEX_CLEANUP</literal> option (Masahiko Sawada) </para> </listitem> I think we ought to expand a bit more on why one would do that, including perhaps some caveat? <listitem> <!-- Author: Peter Eisentraut <peter@eisentraut.org> 2019-03-19 [590a87025] Ignore attempts to add TOAST table to shared or catalog --> <para> Allow modifications of system tables using <xref linkend="sql-altertable"/> (Peter Eisentraut) </para> <para> This allows modifications of <literal>reloptions</literal> and autovacuum settings. </para> </listitem> I think the first paragraph is a bit dangerous. This does *not* generally allow modifications of system tables using ALTER TABLE. <listitem> <!-- Author: Tom Lane <tgl@sss.pgh.pa.us> 2019-01-30 [5f5c01459] Allow RECORD and RECORD[] to be specified in function co --> <para> Allow <type>RECORD</type> and <type>RECORD[]</type> to be specified as a function <link linkend="sql-createfunction">return-value record</link> (Elvis Pranskevichus) </para> <para> DETAIL? </para> </listitem> This description doesn't sound accurate to me. Tom? <listitem> <!-- Author: Tom Lane <tgl@sss.pgh.pa.us> 2018-09-25 [5b7e03670] Avoid unnecessary precision loss for pgbench's - -rate ta --> <para> Compute behavior based on pgbench's <option>--rate</option> value more precisely (Tom Lane) </para> </listitem> "Computing behavior" sounds a bit odd. Maybe "Improve precision of pgbench's <option>--rate</option>" option? <listitem> <!-- Author: Thomas Munro <tmunro@postgresql.org> 2018-07-13 [387a5cfb9] Add pg_dump - -on-conflict-do-nothing option. --> <para> Allow restoration of an <command>INSERT</command>-statement dump to skip rows which would cause conflicts (Surafel Temesgen) </para> <para> The <application>pg_dump</application> option is <option>--on-conflict-do-nothing</option>. </para> </listitem> Hm, this doesn't seem that clear. It's not really a restoration time option, and it sounds a bit like that in the above. How about instead saying something like: Allow pg_dump to emit INSERT ... ON CONFLICT DO NOTHING (Surafel). <listitem> <!-- Author: Andrew Dunstan <andrew@dunslane.net> 2019-02-18 [af25bc03e] Provide an extra-float-digits setting for pg_dump / pg_d --> <para> Allow the number of float digits to be specified for <application>pg_dump</application> and <application>pg_dumpall</application> (Andrew Dunstan) </para> <para> This allows the float digit output to match previous dumps. </para> Hm, feels like that should be combined with the ryu compat entry? <para> Add <xref linkend="sql-create-access-method"/> command to create new table types (Haribabu Kommi, Andres Freund, Álvaro Herrera, Dimitri Dolgov) </para> A few points: 1) Is this really source code, given that CREATE ACCESS METHOD TYPE TABLE is a DDL command, and USING (...) for CREATE TABLE etc is an option to DDL commands? 2) I think the description sounds a bit too much like it's about new forms of tables, rather than their storage. How about something roughly like: Allow different <link linkend="tableam">table access methods</> to be <link linkend="sql-create-access-method>created</> and <link linkend="sql-createtable-method">used</>. This allows to develop and use new ways of storing and accessing table data, optimized for different use-cases, without having to modify PostgreSQL. The existing <literal>heap</literal> access method remains the default. 3) This misses a large set of commits around making tableam possible, in particular the commits around commit 4da597edf1bae0cf0453b5ed6fc4347b6334dfe1 Author: Andres Freund <andres@anarazel.de> Date: 2018-11-16 16:35:11 -0800 Make TupleTableSlots extensible, finish split of existing slot type. Given that those commits entail an API break relevant for extensions, should we have them as a separate "source code" note? 4) I think the attribution isn't quite right. For one, a few names with substantial work are missing (Amit Khandekar, Ashutosh Bapat, Alexander Korotkov), and the order doesn't quite seem right. On the latter part I might be somewhat petty, but I spend *many* months of my life on this. How about: Andres Freund, Haribabu Kommi, Alvaro Herrera, Alexander Korotkov, David Rowley, Dimitri Golgov if we keep 3) separate and Andres Freund, Haribabu Kommi, Alvaro Herrera, Ashutosh Bapat, Alexander Korotkov, Amit Khandekar, David Rowley, DimitriGolgov otherwise? I think it might actually make sense to take David off this list, because his tableam work is essentially part of it's own entry, as <!-- Author: Peter Eisentraut <peter_e@gmx.net> 2018-08-01 [0d5f05cde] Allow multi-inserts during COPY into a partitioned table --> <para> Improve speed of <command>COPY</command> into partitioned tables (David Rowley) </para> since his copy.c portions of 86b85044e823a largely are a rewrite of the above commit. <listitem> <!-- Author: Greg Stark <stark@mit.edu> 2018-10-09 [36e9d413a] Add "B" suffix for bytes to docs --> <para> Document that the <literal>B</literal>/bytes units can be specified for <link linkend="config-setting">server variables</link> (Greg Stark) </para> </listitem> Given how large changes we skip over in the release notes, I don't really see a point in including changes like this. Feels like we'd at the very least also have to include larger changes with typo/grammar fixes etc? Greetings, Andres Freund
[ To: header pruned ] >>>>> "Andres" == Andres Freund <andres@anarazel.de> writes: Andres> <para> Andres> Avoid performing unnecessary rounding of <link Andres> linkend="datatype-float"><type>REAL</type></link> and <type>DOUBLE Andres> PRECISION</type> values (Andrew Gierth) Andres> </para> Andres> <para> Andres> This dramatically speeds up processing of floating-point Andres> values but causes additional trailing digits to Andres> potentially be displayed. Users wishing to have output Andres> that is rounded to match the previous behavior can set <link Andres> linkend="guc-extra-float-digits"><literal>extra_float_digits=0</literal></link>, Andres> which is no longer the default. Andres> </para> Andres> </listitem> Andres> Isn't it exactly the *other* way round? *Previously* we'd Andres> output additional trailing digits. The new algorithm instead Andres> will instead have *exactly* the required number of digits? Yeah, this wording is not right. But your description is also wrong. Previously we output values rounded to 6+d or 15+d digits where d=extra_float_digits, with d=0 being the default. Only clients that wanted exact results would set that to 3 instead. Now we output the minimum digits to get an exact result, which is usually 8 or 17 digits (sometimes less depending on the value, or 9 for the relatively rare float4 values that need it) for any extra_float_digits value > 0. Clients that set d=3 will therefore usually get one less digit than before, and the value they get will usually be slightly different (i.e. not the same value that they would have seen with d=2), but it should give them the same binary value after going through strtod() or strtof(). -- Andrew (irc:RhodiumToad)
Andres Freund <andres@anarazel.de> writes:
> Note that I've added a few questions to individuals involved with
> specific points. If you're in the To: list, please search for your name.
I'm not sure which of my commits you want me to opine on, other than
> <listitem>
> <!--
> Author: Tom Lane <tgl@sss.pgh.pa.us>
> 2019-01-30 [5f5c01459] Allow RECORD and RECORD[] to be specified in function co
> -->
> <para>
> Allow <type>RECORD</type> and <type>RECORD[]</type> to be specified
> as a function <link linkend="sql-createfunction">return-value
> record</link> (Elvis Pranskevichus)
> </para>
> <para>
> DETAIL?
> </para>
> </listitem>
> This description doesn't sound accurate to me. Tom?
Yeah, maybe better
Allow <type>RECORD</type> and <type>RECORD[]</type> to be used
as column types in a query's column definition list for a
table function that is declared to return <type>RECORD</type>
(Elvis Pranskevichus)
You could link to "queries-tablefunctions" which describes the column
definition business; it's much more specific than "sql-createfunction".
regards, tom lane
Hi, On 2019-05-20 23:56:33 +0100, Andrew Gierth wrote: > [ To: header pruned ] > > >>>>> "Andres" == Andres Freund <andres@anarazel.de> writes: > > Andres> <para> > Andres> Avoid performing unnecessary rounding of <link > Andres> linkend="datatype-float"><type>REAL</type></link> and <type>DOUBLE > Andres> PRECISION</type> values (Andrew Gierth) > Andres> </para> > > Andres> <para> > Andres> This dramatically speeds up processing of floating-point > Andres> values but causes additional trailing digits to > Andres> potentially be displayed. Users wishing to have output > Andres> that is rounded to match the previous behavior can set <link > Andres> linkend="guc-extra-float-digits"><literal>extra_float_digits=0</literal></link>, > Andres> which is no longer the default. > Andres> </para> > Andres> </listitem> > > Andres> Isn't it exactly the *other* way round? *Previously* we'd > Andres> output additional trailing digits. The new algorithm instead > Andres> will instead have *exactly* the required number of digits? > > Yeah, this wording is not right. But your description is also wrong. > > Previously we output values rounded to 6+d or 15+d digits where > d=extra_float_digits, with d=0 being the default. Only clients that > wanted exact results would set that to 3 instead. > > Now we output the minimum digits to get an exact result, which is > usually 8 or 17 digits (sometimes less depending on the value, or 9 for > the relatively rare float4 values that need it) for any > extra_float_digits value > 0. Clients that set d=3 will therefore > usually get one less digit than before, and the value they get will > usually be slightly different (i.e. not the same value that they would > have seen with d=2), but it should give them the same binary value after > going through strtod() or strtof(). Any chance for you to propose a text? Greetings, Andres Freund
>>>>> "Andres" == Andres Freund <andres@anarazel.de> writes: Andres> Any chance for you to propose a text? This is what I posted before; I'm not 100% happy with it but it's still better than any of the other versions: * Output REAL and DOUBLE PRECISION values in shortest-exact format by default, and change the behavior of extra_float_digits Previously, float values were output rounded to 6 or 15 decimals by default, with the number of decimals adjusted by extra_float_digits. The previous rounding behavior is no longer the default, and is now done only if extra_float_digits is set to zero or less; if the value is greater than zero (which it is by default), a shortest-precise representation is output (for a substantial performance improvement). This representation preserves the exact binary value when correctly read back in, even though the trailing digits will usually differ from the output generated by previous versions when extra_float_digits=3. -- Andrew (irc:RhodiumToad)
On 2019-05-20 18:56:50 -0400, Tom Lane wrote: > Andres Freund <andres@anarazel.de> writes: > > Note that I've added a few questions to individuals involved with > > specific points. If you're in the To: list, please search for your name. > > I'm not sure which of my commits you want me to opine on, other than That was one of the main ones. I'm also specifically wondering about: > <listitem> > <!-- > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2019-02-09 [1fb57af92] Create the infrastructure for planner support functions. > --> > > <para> > Add support for <link linkend="sql-createfunction">function > selectivity</link> (Tom Lane) > </para> > </listitem> > > Hm, that message doesn't seem like an accurate description of that > commit (if anything it's a391ff3c?). Given that it all requires C > hackery, perhaps we ought to move it to the source code section? And > isn't the most important part of this set of changes > > commit 74dfe58a5927b22c744b29534e67bfdd203ac028 > Author: Tom Lane <tgl@sss.pgh.pa.us> > Date: 2019-02-11 21:26:08 -0500 > > Allow extensions to generate lossy index conditions. and perhaps you could opine on whether we ought to include > <listitem> > <!-- > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2019-02-11 [1d92a0c9f] Redesign the partition dependency mechanism. > --> > > <para> > Improve handling of partition dependency (Tom Lane) > </para> > > <para> > This prevents the creation of inconsistent partition hierarchies > in rare cases. > </para> > </listitem> > > That seems not very informative for users? and if so provide a better description? Because no user is going to make sense of that. And lastly, opine on the int GUC fractions, microsoecond, and cost_delay items? > Yeah, maybe better > > Allow <type>RECORD</type> and <type>RECORD[]</type> to be used > as column types in a query's column definition list for a > table function that is declared to return <type>RECORD</type> > (Elvis Pranskevichus) > > You could link to "queries-tablefunctions" which describes the column > definition business; it's much more specific than "sql-createfunction". Yea, that's much better. Greetings, Andres Freund
Hi, On 2019-05-21 00:08:25 +0100, Andrew Gierth wrote: > >>>>> "Andres" == Andres Freund <andres@anarazel.de> writes: > > Andres> Any chance for you to propose a text? > > This is what I posted before; I'm not 100% happy with it but it's still > better than any of the other versions: > * Output REAL and DOUBLE PRECISION values in shortest-exact format by > default, and change the behavior of extra_float_digits > > Previously, float values were output rounded to 6 or 15 decimals by > default, with the number of decimals adjusted by extra_float_digits. > The previous rounding behavior is no longer the default, and is now > done only if extra_float_digits is set to zero or less; if the value > is greater than zero (which it is by default), a shortest-precise > representation is output (for a substantial performance improvement). > This representation preserves the exact binary value when correctly > read back in, even though the trailing digits will usually differ > from the output generated by previous versions when > extra_float_digits=3. Definitely better from what's there in my opinion. Shortening it if reasonable wouldn't hurt. Perhaps Output REAL and DOUBLE PRECISION values in shortest-exact format by default, and change the behavior of extra_float_digits (...) When extra_float_digits is set to a value greater than zero (the default), a shortest-precise representation is output (for a substantial performance improvement). This representation preserves the exact binary value when correctly read back in, even though the trailing digits will usually differ from the output generated by previous versions when extra_float_digits=3. Previously, float values were output rounded to 6 or 15 decimals by default, with the number of decimals adjusted by extra_float_digits. This behaviour can be restored by setting extra_float_digits is set to zero or less. Or something in that vein? Greetings, Andres Freund
On Tue, 21 May 2019 at 10:17, Andres Freund <andres@anarazel.de> wrote: > commit 4da597edf1bae0cf0453b5ed6fc4347b6334dfe1 > Author: Andres Freund <andres@anarazel.de> > Date: 2018-11-16 16:35:11 -0800 > > Make TupleTableSlots extensible, finish split of existing slot type. > > Given that those commits entail an API break relevant for extensions, > should we have them as a separate "source code" note? > > 4) I think the attribution isn't quite right. For one, a few names with > substantial work are missing (Amit Khandekar, Ashutosh Bapat, > Alexander Korotkov), and the order doesn't quite seem right. On the > latter part I might be somewhat petty, but I spend *many* months of > my life on this. > > How about: > Andres Freund, Haribabu Kommi, Alvaro Herrera, Alexander Korotkov, David Rowley, Dimitri Golgov > if we keep 3) separate and > Andres Freund, Haribabu Kommi, Alvaro Herrera, Ashutosh Bapat, Alexander Korotkov, Amit Khandekar, David Rowley, DimitriGolgov > otherwise? > > I think it might actually make sense to take David off this list, > because his tableam work is essentially part of it's own entry, as Yeah, please take me off that one. My focus there was mostly on keeping COPY fast with partitioned tables, to which, as Andres mentioned is listed somewhere else. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Andres Freund <andres@anarazel.de> writes:
> On 2019-05-20 18:56:50 -0400, Tom Lane wrote:
>> I'm not sure which of my commits you want me to opine on, other than
> That was one of the main ones. I'm also specifically wondering about:
>> Author: Tom Lane <tgl@sss.pgh.pa.us>
>> 2019-02-09 [1fb57af92] Create the infrastructure for planner support functions.
>> <para>
>> Add support for <link linkend="sql-createfunction">function
>> selectivity</link> (Tom Lane)
>> </para>
>> </listitem>
>>
>> Hm, that message doesn't seem like an accurate description of that
>> commit (if anything it's a391ff3c?). Given that it all requires C
>> hackery, perhaps we ought to move it to the source code section?
Yes, this should be in "source code". I think it should be merged
with a391ff3c and 74dfe58a into something like
Allow extensions to create planner support functions that
can provide function-specific selectivity, cost, and
row-count estimates that can depend on the function arguments.
Support functions can also transform WHERE clauses involving
an extension's functions and operators into indexable clauses
in ways that the core code cannot for lack of detailed semantic
knowledge of those functions/operators.
> and perhaps you could opine on whether we ought to include
>> <listitem>
>> <!--
>> Author: Tom Lane <tgl@sss.pgh.pa.us>
>> 2019-02-11 [1d92a0c9f] Redesign the partition dependency mechanism.
>> -->
>>
>> <para>
>> Improve handling of partition dependency (Tom Lane)
>> </para>
>>
>> <para>
>> This prevents the creation of inconsistent partition hierarchies
>> in rare cases.
>> </para>
>> </listitem>
It's probably worth mentioning, but I'd say something like
Fix bugs that could cause ALTER TABLE DETACH PARTITION
to not drop objects that should be dropped, such as
automatically-created child indexes.
The rest of it is not terribly interesting from a user's standpoint,
I think.
> And lastly, opine on the int GUC fractions, microsoecond, and cost_delay
> items?
I agree with your comments on those.
regards, tom lane
On Mon, May 20, 2019 at 3:17 PM Andres Freund <andres@anarazel.de> wrote: > <!-- > Author: Alexander Korotkov <akorotkov@postgresql.org> > 2018-07-28 [d2086b08b] Reduce path length for locking leaf B-tree pages during > Author: Peter Geoghegan <pg@bowt.ie> > 2019-03-25 [f21668f32] Add "split after new tuple" nbtree optimization. > --> > > <para> > Improve speed of btree index insertions (Peter Geoghegan, > Alexander Korotkov) > </para> My concern here (which I believe Alexander shares) is that it doesn't make sense to group these two items together. They're two totally unrelated pieces of work. Alexander's work does more or less help with lock contention with writes, whereas the feature that that was merged with is about preventing index bloat, which is mostly helpful for reads (it helps writes to the extent that writes are also reads). The release notes go on to say that this item "gives better performance for UPDATEs and DELETEs on indexes with many duplicates", which is wrong. That is something that should have been listed below, under the "duplicate index entries in heap-storage order" item. > Author: Peter Geoghegan <pg@bowt.ie> > 2019-03-20 [dd299df81] Make heap TID a tiebreaker nbtree index column. > Author: Peter Geoghegan <pg@bowt.ie> > 2019-03-20 [fab250243] Consider secondary factors during nbtree splits. > --> > > <para> > Have new btree indexes sort duplicate index entries in heap-storage > order (Peter Geoghegan, Heikki Linnakangas) > </para> > I'm not sure that the grouping here is quite right. And the second entry > probably should have some explanation about the benefits? It could stand to say something about the benefits. As I said, there is already a little bit about the benefits, but that ended up being tied to the "Improve speed of btree index insertions" item. Moving that snippet to the correct item would be a good start. -- Peter Geoghegan
On Tue, May 21, 2019 at 8:17 AM Andres Freund <andres@anarazel.de> wrote:
<para>
Add <xref linkend="sql-create-access-method"/> command to create
new table types (Haribabu Kommi, Andres Freund, Álvaro Herrera,
Dimitri Dolgov)
</para>
A few points:
1) Is this really source code, given that CREATE ACCESS METHOD TYPE
TABLE is a DDL command, and USING (...) for CREATE TABLE etc is an
option to DDL commands?
+1
It would be better to provide a description of the newly added syntax.
Do we need to provide any 'Note' explaining that currently there are no other
alternatives to the heap?
2) I think the description sounds a bit too much like it's about new
forms of tables, rather than their storage. How about something
roughly like:
Allow different <link linkend="tableam">table access methods</> to be
<link linkend="sql-create-access-method>created</> and <link
linkend="sql-createtable-method">used</>. This allows to develop and
use new ways of storing and accessing table data, optimized for
different use-cases, without having to modify
PostgreSQL. The existing <literal>heap</literal> access method
remains the default.
3) This misses a large set of commits around making tableam possible, in
particular the commits around
commit 4da597edf1bae0cf0453b5ed6fc4347b6334dfe1
Author: Andres Freund <andres@anarazel.de>
Date: 2018-11-16 16:35:11 -0800
Make TupleTableSlots extensible, finish split of existing slot type.
Given that those commits entail an API break relevant for extensions,
should we have them as a separate "source code" note?
+1 to add, but I am not sure whether we need to list all the breakage that
has introduced by Tableam needs to be described separately or with some
combined note to explain it to extension developers is fine?
4) I think the attribution isn't quite right. For one, a few names with
substantial work are missing (Amit Khandekar, Ashutosh Bapat,
Alexander Korotkov), and the order doesn't quite seem right. On the
latter part I might be somewhat petty, but I spend *many* months of
my life on this.
How about:
Andres Freund, Haribabu Kommi, Alvaro Herrera, Alexander Korotkov, David Rowley, Dimitri Golgov
if we keep 3) separate and
Andres Freund, Haribabu Kommi, Alvaro Herrera, Ashutosh Bapat, Alexander Korotkov, Amit Khandekar, David Rowley, Dimitri Golgov
otherwise?
+1 to either of the above.
Without Andres enormous efforts, Tableam couldn't have been possible into v12.
Regards,
Haribabu Kommi
Fujitsu Australia
On 2019-05-21 00:17, Andres Freund wrote:
> <listitem>
> <!--
> Author: Peter Eisentraut <peter_e@gmx.net>
> 2018-11-14 [1b5d797cd] Lower lock level for renaming indexes
> -->
>
> <para>
> Reduce locking requirements for index renaming (Peter Eisentraut)
> </para>
> </listitem>
>
> Should we specify the newly required lock level? Because it's quire
> relevant for users what exactly they're now able to do concurrently in
> operation?
Yes, more information is in the commit message. We could expand the
release note item with:
"""
Renaming an index now requires only a ShareUpdateExclusiveLock instead
of a AccessExclusiveLock. This allows index renaming without blocking
access to the table.
"""
Note also that this functionality later became part of REINDEX
CONCURRENTLY, which is presumably where most people will make use of it.
> <listitem>
> <!--
> Author: Peter Eisentraut <peter@eisentraut.org>
> 2019-01-11 [ff8530605] Add value 'current' for recovery_target_timeline
> -->
>
> <para>
> Add an explicit value of <literal>current</literal> for <xref
> linkend="guc-recovery-target-time"/> (Peter Eisentraut)
> </para>
> </listitem>
>
> Seems like this should be combined with the earlier "Cause recovery to
> advance to the latest timeline by default" entry.
It could be combined or kept separate or not mentioned at all. Either
way is fine.
> <listitem>
> <!--
> Author: Peter Eisentraut <peter@eisentraut.org>
> 2019-03-30 [fc22b6623] Generated columns
> -->
>
> <para>
> Add support for <link linkend="sql-createtable">generated
> columns</link> (Peter Eisentraut)
> </para>
>
> <para>
> Rather than storing a value only at row creation time, generated
> columns are also modified during updates, and can reference other
> table columns.
> </para>
> </listitem>
>
> I find this description confusing. How about cribbing from the commit?
> Roughly like
>
> This allows creating columns that are computed from expressions,
> including references to other columns in the same table, rather than
> having to be specified by the inserter/updater.
Yeah, that's better.
> Think we also ought to mention that this is only stored generated
> columns, given that the SQL feature also includes virtual columns?
The SQL standard doesn't specify whether generated columns are stored,
but reading between the lines suggest that they expect them to be. So
we don't need to get into more detail there in the release notes. The
main documentation does address this point.
> <listitem>
> <!--
> Author: Peter Eisentraut <peter@eisentraut.org>
> 2019-03-19 [590a87025] Ignore attempts to add TOAST table to shared or catalog
> -->
>
> <para>
> Allow modifications of system tables using <xref
> linkend="sql-altertable"/> (Peter Eisentraut)
> </para>
>
> <para>
> This allows modifications of <literal>reloptions</literal> and
> autovacuum settings.
> </para>
> </listitem>
>
> I think the first paragraph is a bit dangerous. This does *not*
> generally allow modifications of system tables using ALTER TABLE.
Yes, it's overly broad. The second paragraph is really the gist of the
change, so we could write
Allow modifications of reloptions of system tables
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Sat, May 11, 2019 at 04:33:24PM -0400, Bruce Momjian wrote: > I have posted a draft copy of the PG 12 release notes here: > > http://momjian.us/pgsql_docs/release-12.html > > They are committed to git. It includes links to the main docs, where > appropriate. Our official developer docs will rebuild in a few hours. > Thank you for doing this. I didn't see [1] in the release notes, should it be included in the "Source Code" section? [1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=3eb77eba5a51780d5cf52cd66a9844cd4d26feb0 -- Shawn Debnath Amazon Web Services (AWS)
On Tue, May 21, 2019 at 09:09:10AM -0700, Shawn Debnath wrote: > On Sat, May 11, 2019 at 04:33:24PM -0400, Bruce Momjian wrote: > > I have posted a draft copy of the PG 12 release notes here: > > > > http://momjian.us/pgsql_docs/release-12.html > > > > They are committed to git. It includes links to the main docs, where > > appropriate. Our official developer docs will rebuild in a few hours. > > > > Thank you for doing this. I didn't see [1] in the release notes, should > it be included in the "Source Code" section? > > [1] > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=3eb77eba5a51780d5cf52cd66a9844cd4d26feb0 Uh, this is an internals change that is usually not listed in the release notes since it mostly affects internals developers. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
qOn Mon, May 20, 2019 at 03:17:19PM -0700, Andres Freund wrote: > Hi, > > Note that I've added a few questions to individuals involved with > specific points. If you're in the To: list, please search for your name. > > > On 2019-05-11 16:33:24 -0400, Bruce Momjian wrote: > > I have posted a draft copy of the PG 12 release notes here: > > > > http://momjian.us/pgsql_docs/release-12.html > > They are committed to git. > > Thanks! > > <title>Migration to Version 12</title> > > There's a number of features in the compat section that are more general > improvements with a side of incompatibility. Won't it be confusing to > e.g. have have the ryu floating point conversion speedups in the compat > section, but not in the "General Performance" section? Yes, it can be. What I did with the btree item was to split out the max length change with the larger changes. We can do the same for other items. As you rightly stated, it is for cases where the incompatibility is minor compared to the change. Do you have a list of the ones that need this treatment? > <para> > Remove the special behavior of <link > linkend="datatype-oid">OID</link> columns (Andres Freund, > John Naylor) > </para> > > Should we mention that tables with OIDs have to have their oids removed > before they can be upgraded? Uh, is that true? pg_upgrade? pg_dump? > <para> > Refactor <link linkend="functions-geometry">geometric > functions</link> and operators (Emre Hasegeli) > </para> > > <para> > This could lead to more accurate, but slightly different, results > from previous releases. > </para> > </listitem> > <listitem> > <!-- > Author: Tomas Vondra <tomas.vondra@postgresql.org> > 2018-08-16 [c4c340088] Use the built-in float datatypes to implement geometric > --> > > <para> > Restructure <link linkend="datatype-geometric">geometric > types</link> to handle NaN, underflow, overflow and division by > zero more consistently (Emre Hasegeli) > </para> > </listitem> > > <listitem> > <!-- > Author: Tomas Vondra <tomas.vondra@postgresql.org> > 2018-09-26 [2e2a392de] Fix problems in handling the line data type > --> > > <para> > Improve behavior and error reporting for the <link > linkend="datatype-geometric">line data type</link> (Emre Hasegeli) > </para> > </listitem> > > Is that sufficient explanation? Feels like we need to expand a bit > more. In particular, is it possible that a subset of the changes here > require reindexing? > > Also, aren't three different entries a bit too much? The 'line' item related to more errors than just the ones listed for the geometric data types, so I was not clear how to do that as a single entry. I think there is a much larger compatibility breakage possibility with 'line'. > <listitem> > <!-- > Author: Alexander Korotkov <akorotkov@postgresql.org> > 2018-07-28 [d2086b08b] Reduce path length for locking leaf B-tree pages during > Author: Peter Geoghegan <pg@bowt.ie> > 2019-03-25 [f21668f32] Add "split after new tuple" nbtree optimization. > --> > > <para> > Improve speed of btree index insertions (Peter Geoghegan, > Alexander Korotkov) > </para> > > <para> > The new code improves the space-efficiency of page splits, > reduces locking overhead, and gives better performance for > <command>UPDATE</command>s and <command>DELETE</command>s on > indexes with many duplicates. > </para> > </listitem> > > <listitem> > <!-- > Author: Peter Geoghegan <pg@bowt.ie> > 2019-03-20 [dd299df81] Make heap TID a tiebreaker nbtree index column. > Author: Peter Geoghegan <pg@bowt.ie> > 2019-03-20 [fab250243] Consider secondary factors during nbtree splits. > --> > > <para> > Have new btree indexes sort duplicate index entries in heap-storage > order (Peter Geoghegan, Heikki Linnakangas) > </para> > > <para> > Indexes <application>pg_upgraded</application> from previous > releases will not have this ordering. > </para> > </listitem> > > I'm not sure that the grouping here is quite right. And the second entry > probably should have some explanation about the benefits? Agreed. > <listitem> > <!-- > Author: Peter Eisentraut <peter_e@gmx.net> > 2018-11-14 [1b5d797cd] Lower lock level for renaming indexes > --> > > <para> > Reduce locking requirements for index renaming (Peter Eisentraut) > </para> > </listitem> > > Should we specify the newly required lock level? Because it's quire > relevant for users what exactly they're now able to do concurrently in > operation? Sure. > <listitem> > <!-- > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2019-02-09 [1fb57af92] Create the infrastructure for planner support functions. > --> > > <para> > Add support for <link linkend="sql-createfunction">function > selectivity</link> (Tom Lane) > </para> > </listitem> > > Hm, that message doesn't seem like an accurate description of that > commit (if anything it's a391ff3c?). Given that it all requires C > hackery, perhaps we ought to move it to the source code section? And > isn't the most important part of this set of changes > > commit 74dfe58a5927b22c744b29534e67bfdd203ac028 > Author: Tom Lane <tgl@sss.pgh.pa.us> > Date: 2019-02-11 21:26:08 -0500 > > Allow extensions to generate lossy index conditions. Uh, I missed that as an important item. Can someone give me some text? > <listitem> > <!-- > Author: Tomas Vondra <tomas.vondra@postgresql.org> > 2019-01-29 [36a1281f8] Separate per-batch and per-tuple memory contexts in COPY > Author: Heikki Linnakangas <heikki.linnakangas@iki.fi> > 2019-01-25 [9556aa01c] Use single-byte Boyer-Moore-Horspool search even with mu > Author: Andres Freund <andres@anarazel.de> > 2019-01-26 [a9c35cf85] Change function call information to be variable length. > --> > > <para> > Greatly reduce memory consumption of <xref linkend="sql-copy"/> > and function calls (Andres Freund, Tomas Vondra, Tom Lane) > </para> > </listitem> > > Grouping these three changes together makes no sense to me. > > I think the first commit just ought not to be mentioned separately, it's > just a fix for a memory leak in 31f3817402, essentially a 12 only bugfix? Oh, I was not aware of that. > The second commit is about position() etc, which seems not to match that > description either? Ugh. > The third is probably more appropriate to be in the source code > section. While it does speed up function calls a bit (in particular > plpgsql which is very function call heavy), it also is a breaking change > for some external code? Not sure why Tom is listed with this entry? The order of names is just a guess when multiple commits are merged --- this needs help. > <listitem> > <!-- > Author: Heikki Linnakangas <heikki.linnakangas@iki.fi> > 2019-01-25 [9556aa01c] Use single-byte Boyer-Moore-Horspool search even with mu > --> > > <para> > Improve search performance for multi-byte characters (Heikki > Linnakangas) > </para> > </listitem> > > That's the second reference to the commit. I suspect this is much better > separate, so I'd just remove it from above. Done. > <listitem> > <!-- > Author: Stephen Frost <sfrost@snowman.net> > 2019-04-02 [4d0e994ee] Add support for partial TOAST decompression > --> > > <para> > Allow <link linkend="storage-toast"><literal>TOAST</literal></link> > values to be minimally decompressed (Paul Ramsey) > </para> > > I'd s/minimal/partial/ - I don't think the code guarantees anything > about it being minimal? And "minimally decompressed" also is somewhat > confusing, because it sounds like it's about the compression quality > rather than only decompressing part of the data. It is confusing. Is "partially decompressed" better? > <listitem> > <!-- > Author: Michael Paquier <michael@paquier.xyz> > 2018-08-10 [f841ceb26] Improve TRUNCATE by avoiding early lock queue > --> > > <para> > Prevent <xref linkend="sql-truncate"/> from requesting a lock on > tables for which it lacks permission (Michaël Paquier) > </para> > > <para> > This prevents unauthorized locking delays. > </para> > </listitem> > > <listitem> > <!-- > Author: Michael Paquier <michael@paquier.xyz> > 2018-08-27 [a556549d7] Improve VACUUM and ANALYZE by avoiding early lock queue > --> > > <para> > Prevent <command>VACUUM</command> and <command>ANALYZE</command> > from requesting a lock on tables for which it lacks permission > (Michaël Paquier) > </para> > > <para> > This prevents unauthorized locking delays. > </para> > </listitem> > > > I don't think this should be in the <title><acronym>Authentication</acronym></title> > section. I put it in that section since I thought the motivation was to prevent people from locking up connecting to the database if someone has a pending VACUUM/ANALYZE. No? > Also perhaps, s/it/the user/, or "the caller"? Agreed, "the user". > <listitem> > <!-- > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2019-03-10 [cbccac371] Reduce the default value of autovacuum_vacuum_cost_delay > --> > > <para> > Reduce the default value of <xref > linkend="guc-autovacuum-vacuum-cost-delay"/> to 2ms (Tom Lane) > </para> > </listitem> > > I think this needs to explain that this can increase autovacuum's IO > throughput considerably. Uh, well, do we normally document the effect of a change like this? It will cause vacuum to be more agressive, and increase I/O? Do we want to re-educate on what this paramater does? > <listitem> > <!-- > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2019-03-10 [caf626b2c] Convert [autovacuum_]vacuum_cost_delay into floating-poi > --> > > <para> > Allow <xref linkend="guc-vacuum-cost-delay"/> to specify > sub-millisecond delays (Tom Lane) > </para> > > <para> > Floating-point values can also now be specified. > </para> > </listitem> > > And this should be merged with the previous entry? Uh, I thought the change of default and its range were different enough that combining them would add confusion. > <listitem> > <!-- > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2019-03-10 [caf626b2c] Convert [autovacuum_]vacuum_cost_delay into floating-poi > --> > > <para> > Allow time-based server variables to use <link > linkend="config-setting">micro-seconds</link> (us) (Tom Lane) > </para> > </listitem> > > <listitem> > <!-- > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2019-03-11 [1a83a80a2] Allow fractional input values for integer GUCs, and impr > --> > > <para> > Allow fractional input for integer server variables (Tom Lane) > </para> > > <para> > For example, <command>SET work_mem = '30.1GB'</command>. > </para> > </listitem> > > <listitem> > <!-- > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2019-03-10 [caf626b2c] Convert [autovacuum_]vacuum_cost_delay into floating-poi > --> > > <para> > Allow units to be specified for floating-point server variables > (Tom Lane) > </para> > </listitem> > > Can't we combine these? Seems excessively detailed in comparison to the > rest of the entries. See above. It seems confusing to combine them but please propose text if you think it is possible. > <listitem> > <!-- > Author: Peter Eisentraut <peter@eisentraut.org> > 2019-01-11 [ff8530605] Add value 'current' for recovery_target_timeline > --> > > <para> > Add an explicit value of <literal>current</literal> for <xref > linkend="guc-recovery-target-time"/> (Peter Eisentraut) > </para> > </listitem> > > Seems like this should be combined with the earlier "Cause recovery to > advance to the latest timeline by default" entry. The odd part is that the old default was 'current' but there was no way to specify current --- you just specified nothing. That seemed confusing enough that having them combined would add confusion, but if you have some suggested text? > <listitem> > <!-- > Author: Peter Eisentraut <peter@eisentraut.org> > 2019-03-30 [fc22b6623] Generated columns > --> > > <para> > Add support for <link linkend="sql-createtable">generated > columns</link> (Peter Eisentraut) > </para> > > <para> > Rather than storing a value only at row creation time, generated > columns are also modified during updates, and can reference other > table columns. > </para> > </listitem> > > I find this description confusing. How about cribbing from the commit? > Roughly like > > This allows creating columns that are computed from expressions, > including references to other columns in the same table, rather than > having to be specified by the inserter/updater. > > Think we also ought to mention that this is only stored generated > columns, given that the SQL feature also includes virtual columns? OK, new text is: The content of generated columns are computed from expressions (including references to other columns in the same table) rather than being specified by <command>INSERT</command> or <command>UPDATE</command> commands. > > <listitem> > <!-- > Author: Fujii Masao <fujii@postgresql.org> > 2019-04-08 [119dcfad9] Add vacuum_truncate reloption. > Author: Fujii Masao <fujii@postgresql.org> > 2019-05-08 [b84dbc8eb] Add TRUNCATE parameter to VACUUM. > --> > > <para> > Add <xref linkend="sql-vacuum"/> and <command>CREATE > TABLE</command> options to prevent <command>VACUUM</command> > from truncating trailing empty pages (Tsunakawa Takayuki) > </para> > > <para> > The options are <varname>vacuum_truncate</varname> and > <varname>toast.vacuum_truncate</varname>. This reduces vacuum > locking requirements. > </para> > </listitem> > > Maybe add something like: "This can be helpful to avoid query > cancellations on standby that are not avoided by hot_standby_feedback."? So you turn off truncate on the primary becaues the replay of the truncate on the standby might cause a cancelation? I was not aware that was a common problem. > <listitem> > <!-- > Author: Robert Haas <rhaas@postgresql.org> > 2019-04-04 [a96c41fee] Allow VACUUM to be run with index cleanup disabled. > --> > > <para> > Allow vacuum to avoid index cleanup with the > <literal>INDEX_CLEANUP</literal> option (Masahiko Sawada) > </para> > </listitem> > > I think we ought to expand a bit more on why one would do that, > including perhaps some caveat? I actually have no idea why someone would want to do that. > <listitem> > <!-- > Author: Peter Eisentraut <peter@eisentraut.org> > 2019-03-19 [590a87025] Ignore attempts to add TOAST table to shared or catalog > --> > > <para> > Allow modifications of system tables using <xref > linkend="sql-altertable"/> (Peter Eisentraut) > </para> > > <para> > This allows modifications of <literal>reloptions</literal> and > autovacuum settings. > </para> > </listitem> > > I think the first paragraph is a bit dangerous. This does *not* > generally allow modifications of system tables using ALTER TABLE. OK, new text added "options": Allow modifications of system table options using <xref linkend="sql-altertable"/> (Peter Eisentraut) > <listitem> > <!-- > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2018-09-25 [5b7e03670] Avoid unnecessary precision loss for pgbench's - -rate ta > --> > > <para> > Compute behavior based on pgbench's <option>--rate</option> > value more precisely (Tom Lane) > </para> > </listitem> > > "Computing behavior" sounds a bit odd. Maybe "Improve precision of > pgbench's <option>--rate</option>" option? Done. > <listitem> > <!-- > Author: Thomas Munro <tmunro@postgresql.org> > 2018-07-13 [387a5cfb9] Add pg_dump - -on-conflict-do-nothing option. > --> > > <para> > Allow restoration of an <command>INSERT</command>-statement dump > to skip rows which would cause conflicts (Surafel Temesgen) > </para> > > <para> > The <application>pg_dump</application> option is > <option>--on-conflict-do-nothing</option>. > </para> > </listitem> > > Hm, this doesn't seem that clear. It's not really a restoration time > option, and it sounds a bit like that in the above. How about instead saying something > like: > Allow pg_dump to emit INSERT ... ON CONFLICT DO NOTHING (Surafel). Done. > <listitem> > <!-- > Author: Andrew Dunstan <andrew@dunslane.net> > 2019-02-18 [af25bc03e] Provide an extra-float-digits setting for pg_dump / pg_d > --> > > <para> > Allow the number of float digits to be specified > for <application>pg_dump</application> and > <application>pg_dumpall</application> (Andrew Dunstan) > </para> > > <para> > This allows the float digit output to match previous dumps. > </para> > > Hm, feels like that should be combined with the ryu compat entry? Uh, but it relates to this specific command, and it is a new feature rather than a compatibility. > <para> > Add <xref linkend="sql-create-access-method"/> command to create > new table types (Haribabu Kommi, Andres Freund, Álvaro Herrera, > Dimitri Dolgov) > </para> > > A few points: > > 1) Is this really source code, given that CREATE ACCESS METHOD TYPE > TABLE is a DDL command, and USING (...) for CREATE TABLE etc is an > option to DDL commands? I struggled with this. It is a new command, but it has no use yet to users, so if we move it out of "source code" we need to be clear it has no useful purpose yet. Can we do that clearly? > 2) I think the description sounds a bit too much like it's about new > forms of tables, rather than their storage. How about something > roughly like: > > Allow different <link linkend="tableam">table access methods</> to be > <link linkend="sql-create-access-method>created</> and <link > linkend="sql-createtable-method">used</>. This allows to develop and > use new ways of storing and accessing table data, optimized for > different use-cases, without having to modify > PostgreSQL. The existing <literal>heap</literal> access method > remains the default. I added a new detail paragraph: This enables the development of new <link linkend="tableam">table access methods</>, which can optimize storage for different use-cases. The existing <literal>heap</literal> access method remains the default. > 3) This misses a large set of commits around making tableam possible, in > particular the commits around > > commit 4da597edf1bae0cf0453b5ed6fc4347b6334dfe1 > Author: Andres Freund <andres@anarazel.de> > Date: 2018-11-16 16:35:11 -0800 > > Make TupleTableSlots extensible, finish split of existing slot type. > > Given that those commits entail an API break relevant for extensions, > should we have them as a separate "source code" note? I have added this commit to the table-am item. I don't know if this is something that extension people care about, but if so, we should certainly add it. > 4) I think the attribution isn't quite right. For one, a few names with > substantial work are missing (Amit Khandekar, Ashutosh Bapat, > Alexander Korotkov), and the order doesn't quite seem right. On the > latter part I might be somewhat petty, but I spend *many* months of > my life on this. > > How about: > Andres Freund, Haribabu Kommi, Alvaro Herrera, Alexander Korotkov, David Rowley, Dimitri Golgov > if we keep 3) separate and I used the above list since I combined 3 so far. > Andres Freund, Haribabu Kommi, Alvaro Herrera, Ashutosh Bapat, Alexander Korotkov, Amit Khandekar, David Rowley, DimitriGolgov > otherwise? > > I think it might actually make sense to take David off this list, > because his tableam work is essentially part of it's own entry, as > <!-- > Author: Peter Eisentraut <peter_e@gmx.net> > 2018-08-01 [0d5f05cde] Allow multi-inserts during COPY into a partitioned table > --> > > <para> > Improve speed of <command>COPY</command> into partitioned tables > (David Rowley) > </para> > > since his copy.c portions of 86b85044e823a largely are a rewrite of > the above commit. > OK, David removed. > <!-- > Author: Greg Stark <stark@mit.edu> > 2018-10-09 [36e9d413a] Add "B" suffix for bytes to docs > --> > > <para> > Document that the <literal>B</literal>/bytes units can be specified > for <link linkend="config-setting">server variables</link> > (Greg Stark) > </para> > </listitem> > > Given how large changes we skip over in the release notes, I don't > really see a point in including changes like this. Feels like we'd at > the very least also have to include larger changes with typo/grammar > fixes etc? I mentioned it since it was added in a prior release, but was not documented, so effectively there was no way for someone to know it was possible before, so I thought it made sense to mention it. I have only corrected a small number of issues above and look for guidance to finish the rest. I will reply to the other emails in this thread now. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Hi, On 2019-05-21 15:47:34 -0400, Bruce Momjian wrote: > On Mon, May 20, 2019 at 03:17:19PM -0700, Andres Freund wrote: > > Hi, > > > > Note that I've added a few questions to individuals involved with > > specific points. If you're in the To: list, please search for your name. > > > > > > On 2019-05-11 16:33:24 -0400, Bruce Momjian wrote: > > > I have posted a draft copy of the PG 12 release notes here: > > > > > > http://momjian.us/pgsql_docs/release-12.html > > > They are committed to git. > > > > Thanks! > > > > <title>Migration to Version 12</title> > > > > There's a number of features in the compat section that are more general > > improvements with a side of incompatibility. Won't it be confusing to > > e.g. have have the ryu floating point conversion speedups in the compat > > section, but not in the "General Performance" section? > > Yes, it can be. What I did with the btree item was to split out the max > length change with the larger changes. We can do the same for other > items. As you rightly stated, it is for cases where the incompatibility > is minor compared to the change. Do you have a list of the ones that > need this treatment? I was concretely thinking of: - floating point output changes, which are primarily about performance - recovery.conf changes where I'd merge: - Do not allow multiple different recovery_target specificatios - Allow some recovery parameters to be changed with reload - Cause recovery to advance to the latest timeline by default - Add an explicit value of current for guc-recovery-target-time into on entry on the feature side. After having to move recovery settings to a different file, disallowing multiple targets isn't really a separate config break imo. And all the other changes are also fallout from the recovery.conf GUCification. > > <para> > > Remove the special behavior of <link > > linkend="datatype-oid">OID</link> columns (Andres Freund, > > John Naylor) > > </para> > > > > Should we mention that tables with OIDs have to have their oids removed > > before they can be upgraded? > > Uh, is that true? pg_upgrade? pg_dump? Yes. > > <para> > > Refactor <link linkend="functions-geometry">geometric > > functions</link> and operators (Emre Hasegeli) > > </para> > > > > <para> > > This could lead to more accurate, but slightly different, results > > from previous releases. > > </para> > > </listitem> > > <listitem> > > <!-- > > Author: Tomas Vondra <tomas.vondra@postgresql.org> > > 2018-08-16 [c4c340088] Use the built-in float datatypes to implement geometric > > --> > > > > <para> > > Restructure <link linkend="datatype-geometric">geometric > > types</link> to handle NaN, underflow, overflow and division by > > zero more consistently (Emre Hasegeli) > > </para> > > </listitem> > > > > <listitem> > > <!-- > > Author: Tomas Vondra <tomas.vondra@postgresql.org> > > 2018-09-26 [2e2a392de] Fix problems in handling the line data type > > --> > > > > <para> > > Improve behavior and error reporting for the <link > > linkend="datatype-geometric">line data type</link> (Emre Hasegeli) > > </para> > > </listitem> > > > > Is that sufficient explanation? Feels like we need to expand a bit > > more. In particular, is it possible that a subset of the changes here > > require reindexing? > > > > Also, aren't three different entries a bit too much? > > The 'line' item related to more errors than just the ones listed for the > geometric data types, so I was not clear how to do that as a single > entry. I think there is a much larger compatibility breakage > possibility with 'line'. > > > <listitem> > > <!-- > > Author: Alexander Korotkov <akorotkov@postgresql.org> > > 2018-07-28 [d2086b08b] Reduce path length for locking leaf B-tree pages during > > Author: Peter Geoghegan <pg@bowt.ie> > > 2019-03-25 [f21668f32] Add "split after new tuple" nbtree optimization. > > --> > > > > <para> > > Improve speed of btree index insertions (Peter Geoghegan, > > Alexander Korotkov) > > </para> > > > > <para> > > The new code improves the space-efficiency of page splits, > > reduces locking overhead, and gives better performance for > > <command>UPDATE</command>s and <command>DELETE</command>s on > > indexes with many duplicates. > > </para> > > </listitem> > > > > <listitem> > > <!-- > > Author: Peter Geoghegan <pg@bowt.ie> > > 2019-03-20 [dd299df81] Make heap TID a tiebreaker nbtree index column. > > Author: Peter Geoghegan <pg@bowt.ie> > > 2019-03-20 [fab250243] Consider secondary factors during nbtree splits. > > --> > > > > <para> > > Have new btree indexes sort duplicate index entries in heap-storage > > order (Peter Geoghegan, Heikki Linnakangas) > > </para> > > > > <para> > > Indexes <application>pg_upgraded</application> from previous > > releases will not have this ordering. > > </para> > > </listitem> > > > > I'm not sure that the grouping here is quite right. And the second entry > > probably should have some explanation about the benefits? > > Agreed. > > > <listitem> > > <!-- > > Author: Peter Eisentraut <peter_e@gmx.net> > > 2018-11-14 [1b5d797cd] Lower lock level for renaming indexes > > --> > > > > <para> > > Reduce locking requirements for index renaming (Peter Eisentraut) > > </para> > > </listitem> > > > > Should we specify the newly required lock level? Because it's quire > > relevant for users what exactly they're now able to do concurrently in > > operation? > > Sure. > > > <listitem> > > <!-- > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > 2019-02-09 [1fb57af92] Create the infrastructure for planner support functions. > > --> > > > > <para> > > Add support for <link linkend="sql-createfunction">function > > selectivity</link> (Tom Lane) > > </para> > > </listitem> > > > > Hm, that message doesn't seem like an accurate description of that > > commit (if anything it's a391ff3c?). Given that it all requires C > > hackery, perhaps we ought to move it to the source code section? And > > isn't the most important part of this set of changes > > > > commit 74dfe58a5927b22c744b29534e67bfdd203ac028 > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > Date: 2019-02-11 21:26:08 -0500 > > > > Allow extensions to generate lossy index conditions. > > Uh, I missed that as an important item. Can someone give me some text? > > > <listitem> > > <!-- > > Author: Tomas Vondra <tomas.vondra@postgresql.org> > > 2019-01-29 [36a1281f8] Separate per-batch and per-tuple memory contexts in COPY > > Author: Heikki Linnakangas <heikki.linnakangas@iki.fi> > > 2019-01-25 [9556aa01c] Use single-byte Boyer-Moore-Horspool search even with mu > > Author: Andres Freund <andres@anarazel.de> > > 2019-01-26 [a9c35cf85] Change function call information to be variable length. > > --> > > > > <para> > > Greatly reduce memory consumption of <xref linkend="sql-copy"/> > > and function calls (Andres Freund, Tomas Vondra, Tom Lane) > > </para> > > </listitem> > > > > Grouping these three changes together makes no sense to me. > > > > I think the first commit just ought not to be mentioned separately, it's > > just a fix for a memory leak in 31f3817402, essentially a 12 only bugfix? > > Oh, I was not aware of that. > > > The second commit is about position() etc, which seems not to match that > > description either? > > Ugh. > > > The third is probably more appropriate to be in the source code > > section. While it does speed up function calls a bit (in particular > > plpgsql which is very function call heavy), it also is a breaking change > > for some external code? Not sure why Tom is listed with this entry? > > The order of names is just a guess when multiple commits are merged --- > this needs help. > > > <listitem> > > <!-- > > Author: Heikki Linnakangas <heikki.linnakangas@iki.fi> > > 2019-01-25 [9556aa01c] Use single-byte Boyer-Moore-Horspool search even with mu > > --> > > > > <para> > > Improve search performance for multi-byte characters (Heikki > > Linnakangas) > > </para> > > </listitem> > > > > That's the second reference to the commit. I suspect this is much better > > separate, so I'd just remove it from above. > > Done. > > > <listitem> > > <!-- > > Author: Stephen Frost <sfrost@snowman.net> > > 2019-04-02 [4d0e994ee] Add support for partial TOAST decompression > > --> > > > > <para> > > Allow <link linkend="storage-toast"><literal>TOAST</literal></link> > > values to be minimally decompressed (Paul Ramsey) > > </para> > > > > I'd s/minimal/partial/ - I don't think the code guarantees anything > > about it being minimal? And "minimally decompressed" also is somewhat > > confusing, because it sounds like it's about the compression quality > > rather than only decompressing part of the data. > > It is confusing. Is "partially decompressed" better? > > > <listitem> > > <!-- > > Author: Michael Paquier <michael@paquier.xyz> > > 2018-08-10 [f841ceb26] Improve TRUNCATE by avoiding early lock queue > > --> > > > > <para> > > Prevent <xref linkend="sql-truncate"/> from requesting a lock on > > tables for which it lacks permission (Michaël Paquier) > > </para> > > > > <para> > > This prevents unauthorized locking delays. > > </para> > > </listitem> > > > > <listitem> > > <!-- > > Author: Michael Paquier <michael@paquier.xyz> > > 2018-08-27 [a556549d7] Improve VACUUM and ANALYZE by avoiding early lock queue > > --> > > > > <para> > > Prevent <command>VACUUM</command> and <command>ANALYZE</command> > > from requesting a lock on tables for which it lacks permission > > (Michaël Paquier) > > </para> > > > > <para> > > This prevents unauthorized locking delays. > > </para> > > </listitem> > > > > > > I don't think this should be in the <title><acronym>Authentication</acronym></title> > > section. > > I put it in that section since I thought the motivation was to prevent > people from locking up connecting to the database if someone has a > pending VACUUM/ANALYZE. No? > > > Also perhaps, s/it/the user/, or "the caller"? > > Agreed, "the user". > > > <listitem> > > <!-- > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > 2019-03-10 [cbccac371] Reduce the default value of autovacuum_vacuum_cost_delay > > --> > > > > <para> > > Reduce the default value of <xref > > linkend="guc-autovacuum-vacuum-cost-delay"/> to 2ms (Tom Lane) > > </para> > > </listitem> > > > > I think this needs to explain that this can increase autovacuum's IO > > throughput considerably. > > Uh, well, do we normally document the effect of a change like this? It > will cause vacuum to be more agressive, and increase I/O? Do we want to > re-educate on what this paramater does? > > > <listitem> > > <!-- > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > 2019-03-10 [caf626b2c] Convert [autovacuum_]vacuum_cost_delay into floating-poi > > --> > > > > <para> > > Allow <xref linkend="guc-vacuum-cost-delay"/> to specify > > sub-millisecond delays (Tom Lane) > > </para> > > > > <para> > > Floating-point values can also now be specified. > > </para> > > </listitem> > > > > And this should be merged with the previous entry? > > Uh, I thought the change of default and its range were different enough > that combining them would add confusion. > > > <listitem> > > <!-- > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > 2019-03-10 [caf626b2c] Convert [autovacuum_]vacuum_cost_delay into floating-poi > > --> > > > > <para> > > Allow time-based server variables to use <link > > linkend="config-setting">micro-seconds</link> (us) (Tom Lane) > > </para> > > </listitem> > > > > <listitem> > > <!-- > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > 2019-03-11 [1a83a80a2] Allow fractional input values for integer GUCs, and impr > > --> > > > > <para> > > Allow fractional input for integer server variables (Tom Lane) > > </para> > > > > <para> > > For example, <command>SET work_mem = '30.1GB'</command>. > > </para> > > </listitem> > > > > <listitem> > > <!-- > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > 2019-03-10 [caf626b2c] Convert [autovacuum_]vacuum_cost_delay into floating-poi > > --> > > > > <para> > > Allow units to be specified for floating-point server variables > > (Tom Lane) > > </para> > > </listitem> > > > > Can't we combine these? Seems excessively detailed in comparison to the > > rest of the entries. > > See above. It seems confusing to combine them but please propose text > if you think it is possible. > > > <listitem> > > <!-- > > Author: Peter Eisentraut <peter@eisentraut.org> > > 2019-01-11 [ff8530605] Add value 'current' for recovery_target_timeline > > --> > > > > <para> > > Add an explicit value of <literal>current</literal> for <xref > > linkend="guc-recovery-target-time"/> (Peter Eisentraut) > > </para> > > </listitem> > > > > Seems like this should be combined with the earlier "Cause recovery to > > advance to the latest timeline by default" entry. > > The odd part is that the old default was 'current' but there was no way > to specify current --- you just specified nothing. That seemed > confusing enough that having them combined would add confusion, but if > you have some suggested text? > > > <listitem> > > <!-- > > Author: Peter Eisentraut <peter@eisentraut.org> > > 2019-03-30 [fc22b6623] Generated columns > > --> > > > > <para> > > Add support for <link linkend="sql-createtable">generated > > columns</link> (Peter Eisentraut) > > </para> > > > > <para> > > Rather than storing a value only at row creation time, generated > > columns are also modified during updates, and can reference other > > table columns. > > </para> > > </listitem> > > > > I find this description confusing. How about cribbing from the commit? > > Roughly like > > > > This allows creating columns that are computed from expressions, > > including references to other columns in the same table, rather than > > having to be specified by the inserter/updater. > > > > Think we also ought to mention that this is only stored generated > > columns, given that the SQL feature also includes virtual columns? > > OK, new text is: > > The content of generated columns are computed from expressions > (including references to other columns in the same table) > rather than being specified by <command>INSERT</command> or > <command>UPDATE</command> commands. > > > > <listitem> > > <!-- > > Author: Fujii Masao <fujii@postgresql.org> > > 2019-04-08 [119dcfad9] Add vacuum_truncate reloption. > > Author: Fujii Masao <fujii@postgresql.org> > > 2019-05-08 [b84dbc8eb] Add TRUNCATE parameter to VACUUM. > > --> > > > > <para> > > Add <xref linkend="sql-vacuum"/> and <command>CREATE > > TABLE</command> options to prevent <command>VACUUM</command> > > from truncating trailing empty pages (Tsunakawa Takayuki) > > </para> > > > > <para> > > The options are <varname>vacuum_truncate</varname> and > > <varname>toast.vacuum_truncate</varname>. This reduces vacuum > > locking requirements. > > </para> > > </listitem> > > > > Maybe add something like: "This can be helpful to avoid query > > cancellations on standby that are not avoided by hot_standby_feedback."? > > So you turn off truncate on the primary becaues the replay of the > truncate on the standby might cause a cancelation? I was not aware that > was a common problem. > > > <listitem> > > <!-- > > Author: Robert Haas <rhaas@postgresql.org> > > 2019-04-04 [a96c41fee] Allow VACUUM to be run with index cleanup disabled. > > --> > > > > <para> > > Allow vacuum to avoid index cleanup with the > > <literal>INDEX_CLEANUP</literal> option (Masahiko Sawada) > > </para> > > </listitem> > > > > I think we ought to expand a bit more on why one would do that, > > including perhaps some caveat? > > I actually have no idea why someone would want to do that. > > > <listitem> > > <!-- > > Author: Peter Eisentraut <peter@eisentraut.org> > > 2019-03-19 [590a87025] Ignore attempts to add TOAST table to shared or catalog > > --> > > > > <para> > > Allow modifications of system tables using <xref > > linkend="sql-altertable"/> (Peter Eisentraut) > > </para> > > > > <para> > > This allows modifications of <literal>reloptions</literal> and > > autovacuum settings. > > </para> > > </listitem> > > > > I think the first paragraph is a bit dangerous. This does *not* > > generally allow modifications of system tables using ALTER TABLE. > > OK, new text added "options": > > Allow modifications of system table options using <xref > linkend="sql-altertable"/> (Peter Eisentraut) > > > <listitem> > > <!-- > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > 2018-09-25 [5b7e03670] Avoid unnecessary precision loss for pgbench's - -rate ta > > --> > > > > <para> > > Compute behavior based on pgbench's <option>--rate</option> > > value more precisely (Tom Lane) > > </para> > > </listitem> > > > > "Computing behavior" sounds a bit odd. Maybe "Improve precision of > > pgbench's <option>--rate</option>" option? > > Done. > > > <listitem> > > <!-- > > Author: Thomas Munro <tmunro@postgresql.org> > > 2018-07-13 [387a5cfb9] Add pg_dump - -on-conflict-do-nothing option. > > --> > > > > <para> > > Allow restoration of an <command>INSERT</command>-statement dump > > to skip rows which would cause conflicts (Surafel Temesgen) > > </para> > > > > <para> > > The <application>pg_dump</application> option is > > <option>--on-conflict-do-nothing</option>. > > </para> > > </listitem> > > > > Hm, this doesn't seem that clear. It's not really a restoration time > > option, and it sounds a bit like that in the above. How about instead saying something > > like: > > Allow pg_dump to emit INSERT ... ON CONFLICT DO NOTHING (Surafel). > > Done. > > > <listitem> > > <!-- > > Author: Andrew Dunstan <andrew@dunslane.net> > > 2019-02-18 [af25bc03e] Provide an extra-float-digits setting for pg_dump / pg_d > > --> > > > > <para> > > Allow the number of float digits to be specified > > for <application>pg_dump</application> and > > <application>pg_dumpall</application> (Andrew Dunstan) > > </para> > > > > <para> > > This allows the float digit output to match previous dumps. > > </para> > > > > Hm, feels like that should be combined with the ryu compat entry? > > Uh, but it relates to this specific command, and it is a new feature > rather than a compatibility. > > > <para> > > Add <xref linkend="sql-create-access-method"/> command to create > > new table types (Haribabu Kommi, Andres Freund, Álvaro Herrera, > > Dimitri Dolgov) > > </para> > > > > A few points: > > > > 1) Is this really source code, given that CREATE ACCESS METHOD TYPE > > TABLE is a DDL command, and USING (...) for CREATE TABLE etc is an > > option to DDL commands? > > I struggled with this. It is a new command, but it has no use yet to > users, so if we move it out of "source code" we need to be clear it has > no useful purpose yet. Can we do that clearly? > > > > 2) I think the description sounds a bit too much like it's about new > > forms of tables, rather than their storage. How about something > > roughly like: > > > > Allow different <link linkend="tableam">table access methods</> to be > > <link linkend="sql-create-access-method>created</> and <link > > linkend="sql-createtable-method">used</>. This allows to develop and > > use new ways of storing and accessing table data, optimized for > > different use-cases, without having to modify > > PostgreSQL. The existing <literal>heap</literal> access method > > remains the default. > > I added a new detail paragraph: > > This enables the development of new <link linkend="tableam">table > access methods</>, which can optimize storage for different > use-cases. The existing <literal>heap</literal> access method > remains the default. > > > 3) This misses a large set of commits around making tableam possible, in > > particular the commits around > > > > commit 4da597edf1bae0cf0453b5ed6fc4347b6334dfe1 > > Author: Andres Freund <andres@anarazel.de> > > Date: 2018-11-16 16:35:11 -0800 > > > > Make TupleTableSlots extensible, finish split of existing slot type. > > > > Given that those commits entail an API break relevant for extensions, > > should we have them as a separate "source code" note? > > I have added this commit to the table-am item. I don't know if this is > something that extension people care about, but if so, we should > certainly add it. > > > 4) I think the attribution isn't quite right. For one, a few names with > > substantial work are missing (Amit Khandekar, Ashutosh Bapat, > > Alexander Korotkov), and the order doesn't quite seem right. On the > > latter part I might be somewhat petty, but I spend *many* months of > > my life on this. > > > > How about: > > Andres Freund, Haribabu Kommi, Alvaro Herrera, Alexander Korotkov, David Rowley, Dimitri Golgov > > if we keep 3) separate and > > I used the above list since I combined 3 so far. > > > Andres Freund, Haribabu Kommi, Alvaro Herrera, Ashutosh Bapat, Alexander Korotkov, Amit Khandekar, David Rowley, DimitriGolgov > > otherwise? > > > > I think it might actually make sense to take David off this list, > > because his tableam work is essentially part of it's own entry, as > > > <!-- > > Author: Peter Eisentraut <peter_e@gmx.net> > > 2018-08-01 [0d5f05cde] Allow multi-inserts during COPY into a partitioned table > > --> > > > > <para> > > Improve speed of <command>COPY</command> into partitioned tables > > (David Rowley) > > </para> > > > > since his copy.c portions of 86b85044e823a largely are a rewrite of > > the above commit. > > > > OK, David removed. > > > > > <!-- > > Author: Greg Stark <stark@mit.edu> > > 2018-10-09 [36e9d413a] Add "B" suffix for bytes to docs > > --> > > > > <para> > > Document that the <literal>B</literal>/bytes units can be specified > > for <link linkend="config-setting">server variables</link> > > (Greg Stark) > > </para> > > </listitem> > > > > Given how large changes we skip over in the release notes, I don't > > really see a point in including changes like this. Feels like we'd at > > the very least also have to include larger changes with typo/grammar > > fixes etc? > > I mentioned it since it was added in a prior release, but was not > documented, so effectively there was no way for someone to know it was > possible before, so I thought it made sense to mention it. > > I have only corrected a small number of issues above and look for > guidance to finish the rest. I will reply to the other emails in this > thread now. > > -- > Bruce Momjian <bruce@momjian.us> http://momjian.us > EnterpriseDB http://enterprisedb.com > > + As you are, so once was I. As I am, so you will be. + > + Ancient Roman grave inscription + Greetings, Andres Freund
On Tue, May 21, 2019 at 12:08:25AM +0100, Andrew Gierth wrote:
> >>>>> "Andres" == Andres Freund <andres@anarazel.de> writes:
>
> Andres> Any chance for you to propose a text?
>
> This is what I posted before; I'm not 100% happy with it but it's still
> better than any of the other versions:
>
> * Output REAL and DOUBLE PRECISION values in shortest-exact format by
> default, and change the behavior of extra_float_digits
>
> Previously, float values were output rounded to 6 or 15 decimals by
> default, with the number of decimals adjusted by extra_float_digits.
> The previous rounding behavior is no longer the default, and is now
> done only if extra_float_digits is set to zero or less; if the value
> is greater than zero (which it is by default), a shortest-precise
> representation is output (for a substantial performance improvement).
> This representation preserves the exact binary value when correctly
> read back in, even though the trailing digits will usually differ
> from the output generated by previous versions when
> extra_float_digits=3.
How is this?
<para>
Improve performance by changing the default number of trailing digits
output for <link linkend="datatype-float"><type>REAL</type></link>
and <type>DOUBLE PRECISION</type> values (Andrew Gierth)
</para>
<para>
Previously, float values were output rounded to 6 or 15 decimals
by default. Now, only the number of digits required to preserve
the exact binary value is output. The previous behavior can be
restored by setting <xref linkend="guc-extra-float-digits"> to zero.
</para>
Am I missing something?
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
On Mon, May 20, 2019 at 06:56:50PM -0400, Tom Lane wrote: > Andres Freund <andres@anarazel.de> writes: > > Note that I've added a few questions to individuals involved with > > specific points. If you're in the To: list, please search for your name. > > I'm not sure which of my commits you want me to opine on, other than > > > <listitem> > > <!-- > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > 2019-01-30 [5f5c01459] Allow RECORD and RECORD[] to be specified in function co > > --> > > <para> > > Allow <type>RECORD</type> and <type>RECORD[]</type> to be specified > > as a function <link linkend="sql-createfunction">return-value > > record</link> (Elvis Pranskevichus) > > </para> > > <para> > > DETAIL? > > </para> > > </listitem> > > > This description doesn't sound accurate to me. Tom? > > Yeah, maybe better > > Allow <type>RECORD</type> and <type>RECORD[]</type> to be used > as column types in a query's column definition list for a > table function that is declared to return <type>RECORD</type> > (Elvis Pranskevichus) > > You could link to "queries-tablefunctions" which describes the column > definition business; it's much more specific than "sql-createfunction". Done. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, May 21, 2019 at 12:04:50PM +1200, David Rowley wrote: > On Tue, 21 May 2019 at 10:17, Andres Freund <andres@anarazel.de> wrote: > > commit 4da597edf1bae0cf0453b5ed6fc4347b6334dfe1 > > Author: Andres Freund <andres@anarazel.de> > > Date: 2018-11-16 16:35:11 -0800 > > > > Make TupleTableSlots extensible, finish split of existing slot type. > > > > Given that those commits entail an API break relevant for extensions, > > should we have them as a separate "source code" note? > > > > 4) I think the attribution isn't quite right. For one, a few names with > > substantial work are missing (Amit Khandekar, Ashutosh Bapat, > > Alexander Korotkov), and the order doesn't quite seem right. On the > > latter part I might be somewhat petty, but I spend *many* months of > > my life on this. > > > > How about: > > Andres Freund, Haribabu Kommi, Alvaro Herrera, Alexander Korotkov, David Rowley, Dimitri Golgov > > if we keep 3) separate and > > Andres Freund, Haribabu Kommi, Alvaro Herrera, Ashutosh Bapat, Alexander Korotkov, Amit Khandekar, David Rowley, DimitriGolgov > > otherwise? > > > > I think it might actually make sense to take David off this list, > > because his tableam work is essentially part of it's own entry, as > > Yeah, please take me off that one. My focus there was mostly on > keeping COPY fast with partitioned tables, to which, as Andres > mentioned is listed somewhere else. Done. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Mon, May 20, 2019 at 08:48:15PM -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2019-05-20 18:56:50 -0400, Tom Lane wrote:
> >> I'm not sure which of my commits you want me to opine on, other than
>
> > That was one of the main ones. I'm also specifically wondering about:
>
> >> Author: Tom Lane <tgl@sss.pgh.pa.us>
> >> 2019-02-09 [1fb57af92] Create the infrastructure for planner support functions.
> >> <para>
> >> Add support for <link linkend="sql-createfunction">function
> >> selectivity</link> (Tom Lane)
> >> </para>
> >> </listitem>
> >>
> >> Hm, that message doesn't seem like an accurate description of that
> >> commit (if anything it's a391ff3c?). Given that it all requires C
> >> hackery, perhaps we ought to move it to the source code section?
>
> Yes, this should be in "source code". I think it should be merged
> with a391ff3c and 74dfe58a into something like
>
> Allow extensions to create planner support functions that
> can provide function-specific selectivity, cost, and
> row-count estimates that can depend on the function arguments.
> Support functions can also transform WHERE clauses involving
> an extension's functions and operators into indexable clauses
> in ways that the core code cannot for lack of detailed semantic
> knowledge of those functions/operators.
The new text is:
Add support function capability to improve optimizer estimates
for functions (Tom Lane)
This allows extensions to create planner support functions that
can provide function-specific selectivity, cost, and row-count
estimates that can depend on the function arguments. Also, improve
in-core estimates for <function>generate_series()</function>,
<function>unnest()</function>, and functions that return boolean
values.
Notice that there are some improvments in in-core functions. Should this
still be moved to the source code section?
> > and perhaps you could opine on whether we ought to include
>
> >> <listitem>
> >> <!--
> >> Author: Tom Lane <tgl@sss.pgh.pa.us>
> >> 2019-02-11 [1d92a0c9f] Redesign the partition dependency mechanism.
> >> -->
> >>
> >> <para>
> >> Improve handling of partition dependency (Tom Lane)
> >> </para>
> >>
> >> <para>
> >> This prevents the creation of inconsistent partition hierarchies
> >> in rare cases.
> >> </para>
> >> </listitem>
>
> It's probably worth mentioning, but I'd say something like
>
> Fix bugs that could cause ALTER TABLE DETACH PARTITION
> to not drop objects that should be dropped, such as
> automatically-created child indexes.
>
> The rest of it is not terribly interesting from a user's standpoint,
> I think.
Done.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
On Mon, May 20, 2019 at 05:48:50PM -0700, Peter Geoghegan wrote: > On Mon, May 20, 2019 at 3:17 PM Andres Freund <andres@anarazel.de> wrote: > > <!-- > > Author: Alexander Korotkov <akorotkov@postgresql.org> > > 2018-07-28 [d2086b08b] Reduce path length for locking leaf B-tree pages during > > Author: Peter Geoghegan <pg@bowt.ie> > > 2019-03-25 [f21668f32] Add "split after new tuple" nbtree optimization. > > --> > > > > <para> > > Improve speed of btree index insertions (Peter Geoghegan, > > Alexander Korotkov) > > </para> > > My concern here (which I believe Alexander shares) is that it doesn't > make sense to group these two items together. They're two totally > unrelated pieces of work. Alexander's work does more or less help with > lock contention with writes, whereas the feature that that was merged > with is about preventing index bloat, which is mostly helpful for > reads (it helps writes to the extent that writes are also reads). > > The release notes go on to say that this item "gives better > performance for UPDATEs and DELETEs on indexes with many duplicates", > which is wrong. That is something that should have been listed below, > under the "duplicate index entries in heap-storage order" item. OK, I understand how the lock stuff improves things, but I have forgotten how indexes are made smaller. Is it because of better page split logic? > > Author: Peter Geoghegan <pg@bowt.ie> > > 2019-03-20 [dd299df81] Make heap TID a tiebreaker nbtree index column. > > Author: Peter Geoghegan <pg@bowt.ie> > > 2019-03-20 [fab250243] Consider secondary factors during nbtree splits. > > --> > > > > <para> > > Have new btree indexes sort duplicate index entries in heap-storage > > order (Peter Geoghegan, Heikki Linnakangas) > > </para> > > > I'm not sure that the grouping here is quite right. And the second entry > > probably should have some explanation about the benefits? > > It could stand to say something about the benefits. As I said, there > is already a little bit about the benefits, but that ended up being > tied to the "Improve speed of btree index insertions" item. Moving > that snippet to the correct item would be a good start. As I remember the benefit currently is that you can find update and deleted rows faster, right? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Bruce Momjian <bruce@momjian.us> writes:
> On Mon, May 20, 2019 at 08:48:15PM -0400, Tom Lane wrote:
>> Yes, this should be in "source code". I think it should be merged
>> with a391ff3c and 74dfe58a into something like
>>
>> Allow extensions to create planner support functions that
>> can provide function-specific selectivity, cost, and
>> row-count estimates that can depend on the function arguments.
>> Support functions can also transform WHERE clauses involving
>> an extension's functions and operators into indexable clauses
>> in ways that the core code cannot for lack of detailed semantic
>> knowledge of those functions/operators.
> The new text is:
> Add support function capability to improve optimizer estimates
> for functions (Tom Lane)
> This allows extensions to create planner support functions that
> can provide function-specific selectivity, cost, and row-count
> estimates that can depend on the function arguments. Also, improve
> in-core estimates for <function>generate_series()</function>,
> <function>unnest()</function>, and functions that return boolean
> values.
Uh ... you completely lost the business about custom indexable clauses.
I agree with Andres that that's the most important aspect of this.
> Notice that there are some improvments in in-core functions. Should this
> still be moved to the source code section?
I doubt that that's worth mentioning at all. It certainly isn't a
reason not to move this to the source-code section, because that's
where we generally put things that are of interest for improving
extensions, which is what this mainly is.
regards, tom lane
On Tue, May 21, 2019 at 1:51 PM Bruce Momjian <bruce@momjian.us> wrote:
> > My concern here (which I believe Alexander shares) is that it doesn't
> > make sense to group these two items together. They're two totally
> > unrelated pieces of work. Alexander's work does more or less help with
> > lock contention with writes, whereas the feature that that was merged
> > with is about preventing index bloat, which is mostly helpful for
> > reads (it helps writes to the extent that writes are also reads).
> >
> > The release notes go on to say that this item "gives better
> > performance for UPDATEs and DELETEs on indexes with many duplicates",
> > which is wrong. That is something that should have been listed below,
> > under the "duplicate index entries in heap-storage order" item.
>
> OK, I understand how the lock stuff improves things, but I have
> forgotten how indexes are made smaller. Is it because of better page
> split logic?
That is clearly the main reason, though suffix truncation (which
represents that trailing/suffix columns in index tuples from branch
pages have "negative infinity" sentinel values) also contributes to
making indexes smaller.
The page split stuff was mostly added by commit fab250243 ("Consider
secondary factors during nbtree splits"), but commit f21668f32 ("Add
"split after new tuple" nbtree optimization") added to that in a way
that really helped the TPC-C indexes. The TPC-C indexes are about 40%
smaller now.
> > > Author: Peter Geoghegan <pg@bowt.ie>
> > > 2019-03-20 [dd299df81] Make heap TID a tiebreaker nbtree index column.
> As I remember the benefit currently is that you can find update and
> deleted rows faster, right?
Yes, that's true when writing to the index. But more importantly, it
really helps VACUUM when there are lots of duplicates, which is fairly
common in the real world (imagine an index where 20% of the rows are
NULL, for example). In effect, there are no duplicates anymore,
because all index tuples are unique internally.
Indexes with lots of duplicates group older rows together, and new
rows together, because treating heap TID as a tiebreaker naturally has
that effect. VACUUM will generally dirty far fewer pages, because bulk
deletions tend to be correlated with heap TID. And, VACUUM has a much
better chance of deleting entire leaf pages, because dead tuples end
up getting grouped together.
--
Peter Geoghegan
On 5/12/19 5:33 AM, Bruce Momjian wrote: > I have posted a draft copy of the PG 12 release notes here: > > http://momjian.us/pgsql_docs/release-12.html > > They are committed to git. It includes links to the main docs, where > appropriate. Our official developer docs will rebuild in a few hours. In section "Authentication": https://www.postgresql.org/docs/devel/release-12.html#id-1.11.6.5.5.3.7 the last two items are performance improvements not related to authentication; presumably the VACUUM item would be better off in the "Utility Commands" section and the TRUNCATE item in "General Performance"? In section "Source code": https://www.postgresql.org/docs/devel/release-12.html#id-1.11.6.5.5.12 the item "Add CREATE ACCESS METHOD command" doesn't seem related to the source code itself, though I'm not sure where it should go. Regards Ian Barwick -- Ian Barwick https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Tue, May 21, 2019 at 3:49 AM Peter Geoghegan <pg@bowt.ie> wrote: > On Mon, May 20, 2019 at 3:17 PM Andres Freund <andres@anarazel.de> wrote: > > <!-- > > Author: Alexander Korotkov <akorotkov@postgresql.org> > > 2018-07-28 [d2086b08b] Reduce path length for locking leaf B-tree pages during > > Author: Peter Geoghegan <pg@bowt.ie> > > 2019-03-25 [f21668f32] Add "split after new tuple" nbtree optimization. > > --> > > > > <para> > > Improve speed of btree index insertions (Peter Geoghegan, > > Alexander Korotkov) > > </para> > > My concern here (which I believe Alexander shares) is that it doesn't > make sense to group these two items together. They're two totally > unrelated pieces of work. Alexander's work does more or less help with > lock contention with writes, whereas the feature that that was merged > with is about preventing index bloat, which is mostly helpful for > reads (it helps writes to the extent that writes are also reads). > > The release notes go on to say that this item "gives better > performance for UPDATEs and DELETEs on indexes with many duplicates", > which is wrong. That is something that should have been listed below, > under the "duplicate index entries in heap-storage order" item. +1 ------ Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
On Wed, May 22, 2019 at 09:19:53AM +0900, Ian Barwick wrote: > the last two items are performance improvements not related to authentication; > presumably the VACUUM item would be better off in the "Utility Commands" > section and the TRUNCATE item in "General Performance"? I agree with removing them from authentication, but these are not performance-related items. Instead I think that "Utility commands" is a place where they can live better. I am wondering if we should insist on the DOS attacks on a server, as non-authorized users are basically able to block any tables, and authorization is only a part of it, one of the worst parts actually... Anyway, I think that "This prevents unauthorized locking delays." does not provide enough details. What about reusing the first paragraph of the commits? Here is an idea: "A caller of TRUNCATE/VACUUM/ANALYZE could previously queue for an access exclusive lock on a relation it may not have permission to truncate/vacuum/analyze, potentially interfering with users authorized to work on it. This could prevent users from accessing some relations they have access to, in some cases preventing authentication if a critical catalog relation was blocked." -- Michael
Attachment
On 5/22/19 4:26 PM, Michael Paquier wrote: > On Wed, May 22, 2019 at 09:19:53AM +0900, Ian Barwick wrote: >> the last two items are performance improvements not related to authentication; >> presumably the VACUUM item would be better off in the "Utility Commands" >> section and the TRUNCATE item in "General Performance"? > > I agree with removing them from authentication, but these are not > performance-related items. Instead I think that "Utility commands" is > a place where they can live better. > > I am wondering if we should insist on the DOS attacks on a server, as > non-authorized users are basically able to block any tables, and > authorization is only a part of it, one of the worst parts > actually... Anyway, I think that "This prevents unauthorized locking > delays." does not provide enough details. What about reusing the > first paragraph of the commits? Here is an idea: > "A caller of TRUNCATE/VACUUM/ANALYZE could previously queue for an > access exclusive lock on a relation it may not have permission to > truncate/vacuum/analyze, potentially interfering with users authorized > to work on it. This could prevent users from accessing some relations > they have access to, in some cases preventing authentication if a > critical catalog relation was blocked." Ah, if that's the intent behind/use for those changes (I haven't looked at them in any detail, was just scanning the release notes) then it certainly needs some explanation along those lines. Regards Ian Barwick -- Ian Barwick https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Tue, May 21, 2019 at 05:00:31PM -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > On Mon, May 20, 2019 at 08:48:15PM -0400, Tom Lane wrote:
> >> Yes, this should be in "source code". I think it should be merged
> >> with a391ff3c and 74dfe58a into something like
> >>
> >> Allow extensions to create planner support functions that
> >> can provide function-specific selectivity, cost, and
> >> row-count estimates that can depend on the function arguments.
> >> Support functions can also transform WHERE clauses involving
> >> an extension's functions and operators into indexable clauses
> >> in ways that the core code cannot for lack of detailed semantic
> >> knowledge of those functions/operators.
>
> > The new text is:
>
> > Add support function capability to improve optimizer estimates
> > for functions (Tom Lane)
>
> > This allows extensions to create planner support functions that
> > can provide function-specific selectivity, cost, and row-count
> > estimates that can depend on the function arguments. Also, improve
> > in-core estimates for <function>generate_series()</function>,
> > <function>unnest()</function>, and functions that return boolean
> > values.
>
> Uh ... you completely lost the business about custom indexable clauses.
> I agree with Andres that that's the most important aspect of this.
Oh, I see what you mean now. I have updated the docs and moved the item
to Source Code:
<para>
Add support function capability to improve optimizer estimates,
inlining, and indexing for functions (Tom Lane)
</para>
<para>
This allows extensions to create planner support functions that
can provide function-specific selectivity, cost, and row-count
estimates that can depend on the function's arguments. Support
functions can also supply simplified representations and index
conditions, greatly expanding optimization possibilities.
</para>
> > Notice that there are some improvments in in-core functions. Should this
> > still be moved to the source code section?
>
> I doubt that that's worth mentioning at all. It certainly isn't a
> reason not to move this to the source-code section, because that's
> where we generally put things that are of interest for improving
> extensions, which is what this mainly is.
In-core function mention removed.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
Hi,
<listitem>
<!--
Author: Michael Paquier <michael@paquier.xyz>
2019-02-08 [3677a0b26] Add pg_partition_root to display top-most parent of a pa
-->
<para>
Add function <link
linkend="functions-info-partition"><function>pg_partition_root()</function></link>
to return top-most parent of a partition tree (Michaël Paquier)
</para>
</listitem>
<listitem>
<!--
Author: Alvaro Herrera <alvherre@alvh.no-ip.org>
2019-03-04 [b96f6b194] pg_partition_ancestors
-->
<para>
Add function <link
linkend="functions-info-partition"><function>pg_partition_ancestors()</function></link>
to report all ancestors of a partition (Álvaro Herrera)
</para>
</listitem>
<listitem>
<!--
Author: Michael Paquier <michael@paquier.xyz>
2018-10-30 [d5eec4eef] Add pg_partition_tree to display information about parti
-->
<para>
Add function <link
linkend="functions-info-partition"><function>pg_partition_tree()</function></link>
to display information about partitions (Amit Langote)
</para>
</listitem>
Can we combine these three?
Greetings,
Andres Freund
On 2019/05/29 0:58, Andres Freund wrote: > Hi, > > <listitem> > <!-- > Author: Michael Paquier <michael@paquier.xyz> > 2019-02-08 [3677a0b26] Add pg_partition_root to display top-most parent of a pa > --> > > <para> > Add function <link > linkend="functions-info-partition"><function>pg_partition_root()</function></link> > to return top-most parent of a partition tree (Michaël Paquier) > </para> > </listitem> > > <listitem> > <!-- > Author: Alvaro Herrera <alvherre@alvh.no-ip.org> > 2019-03-04 [b96f6b194] pg_partition_ancestors > --> > > <para> > Add function <link > linkend="functions-info-partition"><function>pg_partition_ancestors()</function></link> > to report all ancestors of a partition (Álvaro Herrera) > </para> > </listitem> > > <listitem> > <!-- > Author: Michael Paquier <michael@paquier.xyz> > 2018-10-30 [d5eec4eef] Add pg_partition_tree to display information about parti > --> > > <para> > Add function <link > linkend="functions-info-partition"><function>pg_partition_tree()</function></link> > to display information about partitions (Amit Langote) > </para> > </listitem> > > Can we combine these three? +1 Thanks, Amit
On 5/11/19 10:33 PM, Bruce Momjian wrote: > I have posted a draft copy of the PG 12 release notes here: > > http://momjian.us/pgsql_docs/release-12.html > > They are committed to git. It includes links to the main docs, where > appropriate. Our official developer docs will rebuild in a few hours. > Hello, By looking to a user request to add greek in our FTS [1], I suggest to mention which languages has been added in fd582317e. Patch attached. I hesitate to also mention these changes? > These all work in UTF8, and the indonesian and irish ones also work in LATIN1. > The non-UTF8 version of the hungarian stemmer now works in LATIN2 not LATIN1. 1: https://www.postgresql.org/message-id/trinity-f09793a1-8c13-4b56-94fe-10779e96c87e-1559896268438%403c-app-mailcom-bs16 Cheers, -- Adrien
Attachment
On Tue, May 21, 2019 at 02:22:53PM -0700, Peter Geoghegan wrote:
> On Tue, May 21, 2019 at 1:51 PM Bruce Momjian <bruce@momjian.us> wrote:
> > > My concern here (which I believe Alexander shares) is that it doesn't
> > > make sense to group these two items together. They're two totally
> > > unrelated pieces of work. Alexander's work does more or less help with
> > > lock contention with writes, whereas the feature that that was merged
> > > with is about preventing index bloat, which is mostly helpful for
> > > reads (it helps writes to the extent that writes are also reads).
> > >
> > > The release notes go on to say that this item "gives better
> > > performance for UPDATEs and DELETEs on indexes with many duplicates",
> > > which is wrong. That is something that should have been listed below,
> > > under the "duplicate index entries in heap-storage order" item.
> >
> > OK, I understand how the lock stuff improves things, but I have
> > forgotten how indexes are made smaller. Is it because of better page
> > split logic?
>
> That is clearly the main reason, though suffix truncation (which
> represents that trailing/suffix columns in index tuples from branch
> pages have "negative infinity" sentinel values) also contributes to
> making indexes smaller.
>
> The page split stuff was mostly added by commit fab250243 ("Consider
> secondary factors during nbtree splits"), but commit f21668f32 ("Add
> "split after new tuple" nbtree optimization") added to that in a way
> that really helped the TPC-C indexes. The TPC-C indexes are about 40%
> smaller now.
First, my apologies in getting to this so late. Peter Geoghegan
supplied me with slides and a video to study, and I now understand how
complex the btree improvements are. Here is a video of Peter's
presentation at PGCon:
https://www.youtube.com/watch?v=p5RaATILoiE
What I would like to do is to type them out here, and if I got it right,
I can then add these details to the release notes.
The over-arching improvement to btree in PG 12 is the ordering of index
entries by tid so all entries are unique. As Peter has stated, many
optimizations come out of that:
1. Since all index tuples are ordered, you can move from one leaf page
to the next without keeping a lock on the internal page that references
it, increasing concurrency.
2. When inserting a duplicate value in the index, we used to try a few
pages to see if there was space, then "get tired" and just split a page
containing duplicates. Now that there are no duplicates (because
duplicate key fields are sorted by tid) the system knows exactly what
page the index tuple belongs on, and inserts or splits the page as
necessary.
3. Pivot tuples are used on internal pages and as min/max indicators on
leaf pages. These pivot tuples are now trimmed if their trailing key
fields are not significant. For example, if an index is
field1/field2/field3, and the page contains values for which field1==5
and none that field1==6, there is no need to include field2 and field3
in the pivot tuple --- it can just list the pivot as field1==5,
field2=infinity. This is called suffix truncation.
Page splits used to just split the page in half, which minimizes the
number of page splits, but sometimes causes lots of wasted space. The
new code tries to split to reduce the length of pivot tuples, which ends
up being more efficient in space savings because the pivot tuples are
shorter, and the leaf pages end up being more tightly packed. This is
particularly true for ever-increasing keys because we often end up
creating a new empty page, rather than splitting an existing page.
4. Vacuum's removal of index tuples in indexes with many duplicates is
faster since it can more quickly find desired tuples.
Did I cover everything?
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
On Tue, May 21, 2019 at 12:57:56PM -0700, Andres Freund wrote: > Hi, > > On 2019-05-21 15:47:34 -0400, Bruce Momjian wrote: > > On Mon, May 20, 2019 at 03:17:19PM -0700, Andres Freund wrote: > > > Hi, > > > > > > Note that I've added a few questions to individuals involved with > > > specific points. If you're in the To: list, please search for your name. > > > > > > > > > On 2019-05-11 16:33:24 -0400, Bruce Momjian wrote: > > > > I have posted a draft copy of the PG 12 release notes here: > > > > > > > > http://momjian.us/pgsql_docs/release-12.html > > > > They are committed to git. > > > > > > Thanks! > > > > > > <title>Migration to Version 12</title> > > > > > > There's a number of features in the compat section that are more general > > > improvements with a side of incompatibility. Won't it be confusing to > > > e.g. have have the ryu floating point conversion speedups in the compat > > > section, but not in the "General Performance" section? > > > > Yes, it can be. What I did with the btree item was to split out the max > > length change with the larger changes. We can do the same for other > > items. As you rightly stated, it is for cases where the incompatibility > > is minor compared to the change. Do you have a list of the ones that > > need this treatment? > > I was concretely thinking of: > - floating point output changes, which are primarily about performance If we split out the compatibility change, we don't have much left but "performance", and that doesn't seem long enough for an entry. > - recovery.conf changes where I'd merge: > - Do not allow multiple different recovery_target specificatios > - Allow some recovery parameters to be changed with reload > - Cause recovery to advance to the latest timeline by default > - Add an explicit value of current for guc-recovery-target-time > into on entry on the feature side. > > After having to move recovery settings to a different file, disallowing > multiple targets isn't really a separate config break imo. And all the > other changes are also fallout from the recovery.conf GUCification. Even though we moved the recovery.conf values into postgresql.conf, I think people will assume they just behave the same and copy them into the new file. If their behavior changes, they need to know that explicitly. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Wed, May 22, 2019 at 04:50:14PM +0900, Ian Barwick wrote: > On 5/22/19 4:26 PM, Michael Paquier wrote: > > On Wed, May 22, 2019 at 09:19:53AM +0900, Ian Barwick wrote: > > > the last two items are performance improvements not related to authentication; > > > presumably the VACUUM item would be better off in the "Utility Commands" > > > section and the TRUNCATE item in "General Performance"? > > > > I agree with removing them from authentication, but these are not > > performance-related items. Instead I think that "Utility commands" is > > a place where they can live better. > > > > I am wondering if we should insist on the DOS attacks on a server, as > > non-authorized users are basically able to block any tables, and > > authorization is only a part of it, one of the worst parts > > actually... Anyway, I think that "This prevents unauthorized locking > > delays." does not provide enough details. What about reusing the > > first paragraph of the commits? Here is an idea: > > "A caller of TRUNCATE/VACUUM/ANALYZE could previously queue for an > > access exclusive lock on a relation it may not have permission to > > truncate/vacuum/analyze, potentially interfering with users authorized > > to work on it. This could prevent users from accessing some relations > > they have access to, in some cases preventing authentication if a > > critical catalog relation was blocked." > > Ah, if that's the intent behind/use for those changes (I haven't looked at them > in any detail, was just scanning the release notes) then it certainly needs some > explanation along those lines. Since we did not backpatch this fix, I am hesitant to spell out exactly how to exploit this DOS attack. Yes, people can read it in the email archives, and commit messages, but I don't see the value in spelling it out the release notes too. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, May 28, 2019 at 08:58:23AM -0700, Andres Freund wrote: > <!-- > Author: Michael Paquier <michael@paquier.xyz> > 2019-02-08 [3677a0b26] Add pg_partition_root to display top-most parent of a pa > --> > > <para> > Add function <link > linkend="functions-info-partition"><function>pg_partition_root()</function></link> > to return top-most parent of a partition tree (Michaël Paquier) > </para> > </listitem> > > <listitem> > <!-- > Author: Alvaro Herrera <alvherre@alvh.no-ip.org> > 2019-03-04 [b96f6b194] pg_partition_ancestors > --> > > <para> > Add function <link > linkend="functions-info-partition"><function>pg_partition_ancestors()</function></link> > to report all ancestors of a partition (Álvaro Herrera) > </para> > </listitem> > > <listitem> > <!-- > Author: Michael Paquier <michael@paquier.xyz> > 2018-10-30 [d5eec4eef] Add pg_partition_tree to display information about parti > --> > > <para> > Add function <link > linkend="functions-info-partition"><function>pg_partition_tree()</function></link> > to display information about partitions (Amit Langote) > </para> > </listitem> > > Can we combine these three? Good idea, done. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Fri, Jun 7, 2019 at 12:04:33PM +0200, Adrien Nayrat wrote: > On 5/11/19 10:33 PM, Bruce Momjian wrote: > > I have posted a draft copy of the PG 12 release notes here: > > > > http://momjian.us/pgsql_docs/release-12.html > > > > They are committed to git. It includes links to the main docs, where > > appropriate. Our official developer docs will rebuild in a few hours. > > > > Hello, > > By looking to a user request to add greek in our FTS [1], I suggest to mention > which languages has been added in fd582317e. > > Patch attached. > > I hesitate to also mention these changes? > > > These all work in UTF8, and the indonesian and irish ones also work in LATIN1. > > > The non-UTF8 version of the hungarian stemmer now works in LATIN2 not LATIN1. > > > 1: > https://www.postgresql.org/message-id/trinity-f09793a1-8c13-4b56-94fe-10779e96c87e-1559896268438%403c-app-mailcom-bs16 Good idea, done. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Wed, Jun 12, 2019 at 1:16 PM Bruce Momjian <bruce@momjian.us> wrote: > First, my apologies in getting to this so late. Peter Geoghegan > supplied me with slides and a video to study, and I now understand how > complex the btree improvements are. Here is a video of Peter's > presentation at PGCon: > > https://www.youtube.com/watch?v=p5RaATILoiE Thank you for going to the trouble of researching the B-Tree stuff in detail! I wouldn't ask that of anybody in the position of writing release notes, so it's appreciated. It is awkward to take the work that I've done and make it into multiple bullet points; I have a hard time with that myself. > The over-arching improvement to btree in PG 12 is the ordering of index > entries by tid so all entries are unique. Right. Everything good happens as a direct or indirect result of the TID-as-column thing. That is really the kernel of the whole thing, because it means that the implementation now *fully* follows the Lehman and Yao design. > 1. Since all index tuples are ordered, you can move from one leaf page > to the next without keeping a lock on the internal page that references > it, increasing concurrency. I'm not sure what you mean here. We've never had to keep a lock on an internal page while holding a lock on a leaf page -- such "lock coupling" was necessary in earlier B-Tree designs, but Lehman and Yao's algorithm avoids that. Of course, that isn't new. I think that you're talking about the way that we now check the high key during index scans, and find that we don't have to move to the next leaf page, per this commit: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=29b64d1de7c77ffb5cb10696693e6ed8a6fc481c All of the suffix truncation stuff is good because it makes separator keys in internal pages smaller, but it's also good because the separator keys are simultaneously more "discriminating". We tend to get a nice "clean break" between leaf pages, so checking the high key before moving right to find additional matches (once we've already returned some tuples to the executor) is surprisingly effective. It would have been possible to add this optimization even without the suffix truncation stuff, but truncation makes it effective. If you had to cut one thing from this list, then I would suggest that it be this item. It's nice, but it's also very obvious, which makes it hard to explain. I mean, why should there be any ambiguity at all? Unless we have to return *hundreds* of items to the index scan, then a simple "select * from foo where bar = ?" style query should only have to visit one leaf page, even when the constant happens to be on the boundary of a leaf page (maybe a concurrent page split could make this happen, but that should be rare). > 2. When inserting a duplicate value in the index, we used to try a few > pages to see if there was space, then "get tired" and just split a page > containing duplicates. Now that there are no duplicates (because > duplicate key fields are sorted by tid) the system knows exactly what > page the index tuple belongs on, and inserts or splits the page as > necessary. Right -- inserts will descend straight to the correct leaf page, and the "getting tired" dance isn't used anymore. This makes insertions faster, but more importantly is a better high level strategy for storing lots of duplicates. We'll dirty far fewer pages, because insertions automatically end up inserting around the same place as we inserted to a moment ago. Insertions of duplicates behave like serial/auto-incrementing insertions, which was already fast/well-optimized in various ways. It's easy to measure this by looking at index bloat when inserting lots of duplicates -- I came up with the 16% figure in the talk based on a simple insert-only test. > 3. Pivot tuples are used on internal pages and as min/max indicators on > leaf pages. These pivot tuples are now trimmed if their trailing key > fields are not significant. For example, if an index is > field1/field2/field3, and the page contains values for which field1==5 > and none that field1==6, there is no need to include field2 and field3 > in the pivot tuple --- it can just list the pivot as field1==5, > field2=infinity. This is called suffix truncation. Right -- that's exactly how it works. Users may find that indexes with lots of extra columns at the end won't get so bloated in Postgres 12. Indexing many columns is typically seen when index-only scans are important. Of course, you could have made those indexes INCLUDE indexes on v11, which is actually a closely related idea, but that makes it impossible to use the trailing/suffix columns in an ORDER BY. And, you have to know about INCLUDE indexes, and remember to use them. (This must be why Oracle can get away with not having INCLUDE indexes.) > Page splits used to just split the page in half, which minimizes the > number of page splits, but sometimes causes lots of wasted space. The > new code tries to split to reduce the length of pivot tuples, which ends > up being more efficient in space savings because the pivot tuples are > shorter, and the leaf pages end up being more tightly packed. This is > particularly true for ever-increasing keys because we often end up > creating a new empty page, rather than splitting an existing page. Right. We need to be somewhat cleverer about precisely where we split leaf pages in order to make suffix truncation work well. But, it turns out that there is another case where being *very* clever about leaf page split points matters a lot, which is targeted by the "split after new tuple" optimization. The "split after new tuple" optimization was just a bonus for me. > 4. Vacuum's removal of index tuples in indexes with many duplicates is > faster since it can more quickly find desired tuples. This may be true, but the interesting way that the TID-as-column thing helps VACUUM is related to locality (spatial and temporal locality). VACUUM can delete/remove whole pages at a time, though only when they're completely empty (just one remaining tuple blocks page deletion. because VACUUM cannot merge non-empty pages together). This is much more likely to occur when we group like with like. Heap TID is often correlated with primary key value, or with a timestamp, so we can easily imagine VACUUM deleting more pages filled with duplicates because they're grouped together in a way that's logical (instead of more-or-less random, which is what "getting tired" left us with). Even without page deletion occurring, we can reuse ranges in the index when there are lots of duplicates. We delete some rows in a table, and VACUUM ultimately removes the rows from both table and indexes -- including some interesting index that stores lots of duplicates, like an index on a enum field. VACUUM makes it possible to recycle the heap TIDs/space *everywhere*, not just in the table, as before. In the fairly likely event of a future insert that recycles a heap TID also having the same value for the index with duplicates (say its an enum value), the dup index tuple can go in exactly the same place as the earlier, now-deleted dup tuple. The "recycling" works at the index level, too. In practice this kind of locality is rather common. It's especially likely with non-HOT updates where the duplicate value isn't changed, though simple inserts and deletes can see the same benefit. Obviously you'll need to boil all that down -- good luck! Thanks -- Peter Geoghegan
On Wed, Jun 12, 2019 at 03:06:34PM -0700, Peter Geoghegan wrote: > On Wed, Jun 12, 2019 at 1:16 PM Bruce Momjian <bruce@momjian.us> wrote: > > First, my apologies in getting to this so late. Peter Geoghegan > > supplied me with slides and a video to study, and I now understand how > > complex the btree improvements are. Here is a video of Peter's > > presentation at PGCon: > > > > https://www.youtube.com/watch?v=p5RaATILoiE > > Thank you for going to the trouble of researching the B-Tree stuff in > detail! I wouldn't ask that of anybody in the position of writing > release notes, so it's appreciated. It is awkward to take the work > that I've done and make it into multiple bullet points; I have a hard > time with that myself. I had become so confused by this item that I needed a few weeks to settle on what was actually going on. > > The over-arching improvement to btree in PG 12 is the ordering of index > > entries by tid so all entries are unique. > > Right. Everything good happens as a direct or indirect result of the > TID-as-column thing. That is really the kernel of the whole thing, > because it means that the implementation now *fully* follows the > Lehman and Yao design. Right. > > 1. Since all index tuples are ordered, you can move from one leaf page > > to the next without keeping a lock on the internal page that references > > it, increasing concurrency. > > I'm not sure what you mean here. We've never had to keep a lock on an > internal page while holding a lock on a leaf page -- such "lock > coupling" was necessary in earlier B-Tree designs, but Lehman and > Yao's algorithm avoids that. Of course, that isn't new. > > I think that you're talking about the way that we now check the high > key during index scans, and find that we don't have to move to the > next leaf page, per this commit: > > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=29b64d1de7c77ffb5cb10696693e6ed8a6fc481c I was wrong. I was thinking of this commit: commit d2086b08b0 Author: Alexander Korotkov <akorotkov@postgresql.org> Date: Sat Jul 28 00:31:40 2018 +0300 Reduce path length for locking leaf B-tree pages during insertion In our B-tree implementation appropriate leaf page for new tuple insertion is acquired using _bt_search() function. This function always returns leaf page locked in shared mode. In order to obtain exclusive lock, caller have to relock the page. This commit makes _bt_search() function lock leaf page immediately in exclusive mode when needed. That removes unnecessary relock and, in turn reduces lock contention for B-tree leaf pages. Our experiments on multi-core systems showed acceleration up to 4.5 times in corner case. but got it confused by an optimization you mentioned in the video where you were talking about the need to perhaps recheck the internal page when moving right. We certainly don't keep the internal page locked. > All of the suffix truncation stuff is good because it makes separator > keys in internal pages smaller, but it's also good because the > separator keys are simultaneously more "discriminating". We tend to > get a nice "clean break" between leaf pages, so checking the high key > before moving right to find additional matches (once we've already > returned some tuples to the executor) is surprisingly effective. It > would have been possible to add this optimization even without the > suffix truncation stuff, but truncation makes it effective. > > If you had to cut one thing from this list, then I would suggest that > it be this item. It's nice, but it's also very obvious, which makes it > hard to explain. I mean, why should there be any ambiguity at all? > Unless we have to return *hundreds* of items to the index scan, then a > simple "select * from foo where bar = ?" style query should only have > to visit one leaf page, even when the constant happens to be on the > boundary of a leaf page (maybe a concurrent page split could make this > happen, but that should be rare). Right. The commit mentioned a 4.5x speedup in a rare benchmark, so I added it lower on the list. > > 2. When inserting a duplicate value in the index, we used to try a few > > pages to see if there was space, then "get tired" and just split a page > > containing duplicates. Now that there are no duplicates (because > > duplicate key fields are sorted by tid) the system knows exactly what > > page the index tuple belongs on, and inserts or splits the page as > > necessary. > > Right -- inserts will descend straight to the correct leaf page, and > the "getting tired" dance isn't used anymore. This makes insertions > faster, but more importantly is a better high level strategy for > storing lots of duplicates. We'll dirty far fewer pages, because > insertions automatically end up inserting around the same place as we > inserted to a moment ago. Insertions of duplicates behave like > serial/auto-incrementing insertions, which was already > fast/well-optimized in various ways. Yes, locality. > It's easy to measure this by looking at index bloat when inserting > lots of duplicates -- I came up with the 16% figure in the talk based > on a simple insert-only test. > > > 3. Pivot tuples are used on internal pages and as min/max indicators on > > leaf pages. These pivot tuples are now trimmed if their trailing key > > fields are not significant. For example, if an index is > > field1/field2/field3, and the page contains values for which field1==5 > > and none that field1==6, there is no need to include field2 and field3 > > in the pivot tuple --- it can just list the pivot as field1==5, > > field2=infinity. This is called suffix truncation. > > Right -- that's exactly how it works. Users may find that indexes with > lots of extra columns at the end won't get so bloated in Postgres 12. > Indexing many columns is typically seen when index-only scans are > important. Of course, you could have made those indexes INCLUDE > indexes on v11, which is actually a closely related idea, but that > makes it impossible to use the trailing/suffix columns in an ORDER BY. > And, you have to know about INCLUDE indexes, and remember to use them. > > (This must be why Oracle can get away with not having INCLUDE indexes.) > > > Page splits used to just split the page in half, which minimizes the > > number of page splits, but sometimes causes lots of wasted space. The > > new code tries to split to reduce the length of pivot tuples, which ends > > up being more efficient in space savings because the pivot tuples are > > shorter, and the leaf pages end up being more tightly packed. This is > > particularly true for ever-increasing keys because we often end up > > creating a new empty page, rather than splitting an existing page. > > Right. We need to be somewhat cleverer about precisely where we split > leaf pages in order to make suffix truncation work well. But, it turns > out that there is another case where being *very* clever about leaf > page split points matters a lot, which is targeted by the "split after > new tuple" optimization. The "split after new tuple" optimization was > just a bonus for me. > > > 4. Vacuum's removal of index tuples in indexes with many duplicates is > > faster since it can more quickly find desired tuples. > > This may be true, but the interesting way that the TID-as-column thing > helps VACUUM is related to locality (spatial and temporal locality). > VACUUM can delete/remove whole pages at a time, though only when > they're completely empty (just one remaining tuple blocks page > deletion. because VACUUM cannot merge non-empty pages together). This > is much more likely to occur when we group like with like. Heap TID is > often correlated with primary key value, or with a timestamp, so we > can easily imagine VACUUM deleting more pages filled with duplicates > because they're grouped together in a way that's logical (instead of > more-or-less random, which is what "getting tired" left us with). > > Even without page deletion occurring, we can reuse ranges in the index > when there are lots of duplicates. We delete some rows in a table, and > VACUUM ultimately removes the rows from both table and indexes -- > including some interesting index that stores lots of duplicates, like > an index on a enum field. VACUUM makes it possible to recycle the heap > TIDs/space *everywhere*, not just in the table, as before. In the > fairly likely event of a future insert that recycles a heap TID also > having the same value for the index with duplicates (say its an enum > value), the dup index tuple can go in exactly the same place as the > earlier, now-deleted dup tuple. The "recycling" works at the index > level, too. In practice this kind of locality is rather common. It's > especially likely with non-HOT updates where the duplicate value isn't > changed, though simple inserts and deletes can see the same benefit. Yes, I can see index space reuse as more closely matching the heap. > Obviously you'll need to boil all that down -- good luck! Attached is an updated patch. I might have missed something, but I think it might be close. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Attachment
On Wed, Jun 12, 2019 at 5:22 PM Bruce Momjian <bruce@momjian.us> wrote: > I had become so confused by this item that I needed a few weeks to > settle on what was actually going on. I put a lot of time into my pgCon talk, especially on the diagrams. Seems like that paid off. Even Heikki was confused by my explanations at one point. I should go add a similar diagram to our documentation, under "Chapter 63. B-Tree Indexes", because diagrams are the only sensible way to explain the concepts. > I was wrong. I was thinking of this commit: > > commit d2086b08b0 > Author: Alexander Korotkov <akorotkov@postgresql.org> > Date: Sat Jul 28 00:31:40 2018 +0300 > > Reduce path length for locking leaf B-tree pages during insertion > > If you had to cut one thing from this list, then I would suggest that > > it be this item. It's nice, but it's also very obvious, which makes it > > hard to explain. > Right. The commit mentioned a 4.5x speedup in a rare benchmark, so I > added it lower on the list. My remark about cutting an item referred to a lesser item that I worked on (the 'Add nbtree high key "continuescan" optimization' commit), not Alexander independent B-Tree work. I think that Alexander's optimization is also quite effective. Though FWIW the 4.5x improvement concerned a case involving lots of duplicates...cases with a lot of duplicates will be far far better in Postgres 12. (I never tested my patch without Alexander's commit, since it went in early in the v12 cycle.) > Yes, locality. "Locality" is one of my favorite words. > Attached is an updated patch. I might have missed something, but I > think it might be close. This looks great. I do have a few things: * I would put "Improve performance and space utilization of btree indexes with many duplicates" first (before "Allow multi-column btree indexes to be smaller"). I think that this is far more common than we tend to assume, and is also where the biggest benefits are. * The wording of the "many duplicates" item itself is almost perfect, though the "...and inefficiency when VACUUM needed to find a row for removal" part seems a bit off -- this is really about the effectiveness of VACUUM, not the speed at which the VACUUM completes (it's a bit faster, but that's not that important). Perhaps that part should read: "...and often failed to efficiently recycle space made available by VACUUM". Something like that. * The "Allow multi-column btree indexes to be smaller" item is about both suffix truncation, and about the "Split after new tuple" optimization. I think that that makes it more complicated than it needs to be. While the improvements that we saw with TPC-C on account of the "Split after new tuple" optimization were nice, I doubt that users will be looking out for it. I would be okay if you dropped any mention of the "Split after new tuple" optimization, in the interest of making the description more useful to users. We can just lose that. * Once you simplify the item by making it all about suffix truncation, it would make sense to change the single line summary to "Reduce the number of branch blocks needed for multi-column indexes". Then go on to talk about how we now only store those columns that are necessary to guide index scans in tuples stored in branch pages (we tend to call branch pages internal pages, but branch pages seems friendlier to me). Note that the user docs of other database systems reference these details, even in their introductory material on how B-Tree indexes work. The term "suffix truncation" isn't something users have heard of, and we shouldn't use it here, but the *idea* of suffix truncation is very well established. As I mentioned, it matters for things like covering indexes (indexes that are designed to be used by index-only scans, which are not necessarily INCLUDE indexes). Thanks! -- Peter Geoghegan
On Wed, Jun 12, 2019 at 06:34:27PM -0700, Peter Geoghegan wrote: > > I was wrong. I was thinking of this commit: > > > > commit d2086b08b0 > > Author: Alexander Korotkov <akorotkov@postgresql.org> > > Date: Sat Jul 28 00:31:40 2018 +0300 > > > > Reduce path length for locking leaf B-tree pages during insertion > > > > If you had to cut one thing from this list, then I would suggest that > > > it be this item. It's nice, but it's also very obvious, which makes it > > > hard to explain. > > > Right. The commit mentioned a 4.5x speedup in a rare benchmark, so I > > added it lower on the list. > > My remark about cutting an item referred to a lesser item that I > worked on (the 'Add nbtree high key "continuescan" optimization' > commit), not Alexander independent B-Tree work. I think that > Alexander's optimization is also quite effective. Though FWIW the 4.5x > improvement concerned a case involving lots of duplicates...cases with > a lot of duplicates will be far far better in Postgres 12. (I never > tested my patch without Alexander's commit, since it went in early in > the v12 cycle.) OK, good to know. > > Attached is an updated patch. I might have missed something, but I > > think it might be close. > > This looks great. I do have a few things: > > * I would put "Improve performance and space utilization of btree > indexes with many duplicates" first (before "Allow multi-column btree > indexes to be smaller"). I think that this is far more common than we > tend to assume, and is also where the biggest benefits are. OK, done, I was wondering about that. > * The wording of the "many duplicates" item itself is almost perfect, > though the "...and inefficiency when VACUUM needed to find a row for > removal" part seems a bit off -- this is really about the > effectiveness of VACUUM, not the speed at which the VACUUM completes > (it's a bit faster, but that's not that important). Perhaps that part > should read: "...and often failed to efficiently recycle space made > available by VACUUM". Something like that. Ah, I see what you mean --- recycle entire pages. I have updated the patch. > * The "Allow multi-column btree indexes to be smaller" item is about > both suffix truncation, and about the "Split after new tuple" > optimization. I think that that makes it more complicated than it > needs to be. While the improvements that we saw with TPC-C on account > of the "Split after new tuple" optimization were nice, I doubt that > users will be looking out for it. I would be okay if you dropped any > mention of the "Split after new tuple" optimization, in the interest > of making the description more useful to users. We can just lose that. OK, done. > * Once you simplify the item by making it all about suffix truncation, > it would make sense to change the single line summary to "Reduce the > number of branch blocks needed for multi-column indexes". Then go on > to talk about how we now only store those columns that are necessary > to guide index scans in tuples stored in branch pages (we tend to call > branch pages internal pages, but branch pages seems friendlier to me). > Note that the user docs of other database systems reference these > details, even in their introductory material on how B-Tree indexes > work. The term "suffix truncation" isn't something users have heard > of, and we shouldn't use it here, but the *idea* of suffix truncation > is very well established. As I mentioned, it matters for things like > covering indexes (indexes that are designed to be used by index-only > scans, which are not necessarily INCLUDE indexes). OK, I mentioned something about increased locality now. Patch attached. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Attachment
On Wed, Jun 12, 2019 at 7:29 PM Bruce Momjian <bruce@momjian.us> wrote: > OK, I mentioned something about increased locality now. Patch attached. Looks good -- locality is a good catch-all term. Thanks! -- Peter Geoghegan
On Wed, Jun 12, 2019 at 07:42:31PM -0700, Peter Geoghegan wrote: > On Wed, Jun 12, 2019 at 7:29 PM Bruce Momjian <bruce@momjian.us> wrote: > > OK, I mentioned something about increased locality now. Patch attached. > > Looks good -- locality is a good catch-all term. Great, patch applied. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Wed, Jun 12, 2019 at 05:25:37PM -0400, Bruce Momjian wrote: > Since we did not backpatch this fix, I am hesitant to spell out exactly > how to exploit this DOS attack. Yes, people can read it in the email > archives, and commit messages, but I don't see the value in spelling it > out the release notes too. We could go for a more general version of that, for the reason that it involves all relations, like: "A caller of TRUNCATE or VACUUM could previously queue for an access exclusive lock on a relation it may not have permission to truncate or vacuum, leading to relations to be blocked from being accessed." -- Michael
Attachment
On Thu, Jun 13, 2019 at 03:33:48PM +0900, Michael Paquier wrote: > On Wed, Jun 12, 2019 at 05:25:37PM -0400, Bruce Momjian wrote: > > Since we did not backpatch this fix, I am hesitant to spell out exactly > > how to exploit this DOS attack. Yes, people can read it in the email > > archives, and commit messages, but I don't see the value in spelling it > > out the release notes too. > > We could go for a more general version of that, for the reason that it > involves all relations, like: > "A caller of TRUNCATE or VACUUM could previously queue for an access > exclusive lock on a relation it may not have permission to truncate or > vacuum, leading to relations to be blocked from being accessed." Uh, that still seems to suggest an attack and I am not sure that information is very useful to users. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Thu, Jun 13, 2019 at 09:11:08AM -0400, Bruce Momjian wrote:
> On Thu, Jun 13, 2019 at 03:33:48PM +0900, Michael Paquier wrote:
> > On Wed, Jun 12, 2019 at 05:25:37PM -0400, Bruce Momjian wrote:
> > > Since we did not backpatch this fix, I am hesitant to spell out exactly
> > > how to exploit this DOS attack. Yes, people can read it in the email
> > > archives, and commit messages, but I don't see the value in spelling it
> > > out the release notes too.
> >
> > We could go for a more general version of that, for the reason that it
> > involves all relations, like:
> > "A caller of TRUNCATE or VACUUM could previously queue for an access
> > exclusive lock on a relation it may not have permission to truncate or
> > vacuum, leading to relations to be blocked from being accessed."
>
> Uh, that still seems to suggest an attack and I am not sure that
> information is very useful to users.
I went with this wording:
This prevents unauthorized locking, which could interfere with
user queries.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
On Thu, Jun 13, 2019 at 09:12:58AM -0400, Bruce Momjian wrote: > I went with this wording: > > This prevents unauthorized locking, which could interfere with > user queries. Okay, fine for me. Thanks for updating the notes. -- Michael
Attachment
I noticed a small typo in the release notes in the list of languages with new stemmers (see attached) -- John Naylor https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachment
On Sat, 2019-05-11 at 16:33 -0400, Bruce Momjian wrote: > I have posted a draft copy of the PG 12 release notes here: I wonder if commits 0ba06e0bf and 40cfe8606 are worth mentioning in the release notes. They make "pg_test_fsync" work correctly on Windows for the first time. Yours, Laurenz Albe
Attachment
On Mon, Jul 15, 2019 at 10:18:07PM +0700, John Naylor wrote: > I noticed a small typo in the release notes in the list of languages > with new stemmers (see attached) Sorry, fixed, thanks. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Wed, Jun 12, 2019 at 7:48 PM Bruce Momjian <bruce@momjian.us> wrote: > Great, patch applied. I think that it would make sense to have a v12 release note item for amcheck's new "rootdescend" verification option: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=c1afd175b5b2e5c44f6da34988342e00ecdfb518 It is a user facing feature, which increments the amcheck extension version number. Thanks -- Peter Geoghegan
On Mon, Jul 15, 2019 at 08:51:34PM +0200, Laurenz Albe wrote: > I wonder if commits 0ba06e0bf and 40cfe8606 are worth mentioning > in the release notes. They make "pg_test_fsync" work correctly > on Windows for the first time. I don't know about this point specifically. Improving support for pg_test_fsync on Windows is just a side effect of the first commit which benefits all frontend tools (the second commit is an embarrassing bug fix for the first one). And at the same time we don't really add in the release notes low-level improvements like these ones. -- Michael
Attachment
On Mon, Jul 15, 2019 at 08:51:34PM +0200, Laurenz Albe wrote: > On Sat, 2019-05-11 at 16:33 -0400, Bruce Momjian wrote: > > I have posted a draft copy of the PG 12 release notes here: > > I wonder if commits 0ba06e0bf and 40cfe8606 are worth mentioning > in the release notes. They make "pg_test_fsync" work correctly > on Windows for the first time. Oh, I missed adding that. I applied the attached patch, with updated wording, and moved it to the Server Applications section. Thanks. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Attachment
On Tue, Jul 16, 2019 at 03:26:31PM +0900, Michael Paquier wrote: > On Mon, Jul 15, 2019 at 08:51:34PM +0200, Laurenz Albe wrote: > > I wonder if commits 0ba06e0bf and 40cfe8606 are worth mentioning > > in the release notes. They make "pg_test_fsync" work correctly > > on Windows for the first time. > > I don't know about this point specifically. Improving support for > pg_test_fsync on Windows is just a side effect of the first commit > which benefits all frontend tools (the second commit is an > embarrassing bug fix for the first one). And at the same time we > don't really add in the release notes low-level improvements like > these ones. Well, if we were reporting incorrect results before, that seems like a fix, with updated wording, of course, to mention just the fix. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Mon, Jul 15, 2019 at 06:21:59PM -0700, Peter Geoghegan wrote: > On Wed, Jun 12, 2019 at 7:48 PM Bruce Momjian <bruce@momjian.us> wrote: > > Great, patch applied. > > I think that it would make sense to have a v12 release note item for > amcheck's new "rootdescend" verification option: > > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=c1afd175b5b2e5c44f6da34988342e00ecdfb518 > > It is a user facing feature, which increments the amcheck extension > version number. Attached patch applied, thanks. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Attachment
On Thu, Jul 25, 2019 at 6:37 PM Bruce Momjian <bruce@momjian.us> wrote: > Attached patch applied, thanks. Thanks Bruce, -- Peter Geoghegan
On 5/11/19 4:33 PM, Bruce Momjian wrote: > I have posted a draft copy of the PG 12 release notes here: > > http://momjian.us/pgsql_docs/release-12.html > > They are committed to git. It includes links to the main docs, where > appropriate. Our official developer docs will rebuild in a few hours. > Thanks again for compiling and writing up the release notes. I know it is no small effort. Attached is a patch proposing items for the major items section. This is working off of the ongoing draft of the press release[1]. Feedback welcome. With respect to the linking, I tried I to give a bunch of jumping off points for users to explore the features, but visually tried to ensure the readability was consistent. I also attached a patch addressing the "MENTION ITS AFFECT ON ORDERING?" in which I choose to answer it "yes" and added a comment :) Thanks, Jonathan [1] https://www.postgresql.org/message-id/c56eeb88-4a8c-2c6c-b5f1-9d46872c247c%40postgresql.org
Attachment
> 2 сент. 2019 г., в 22:02, Jonathan S. Katz <jkatz@postgresql.org> написал(а): > > > Attached is a patch proposing items for the major items section. This is > working off of the ongoing draft of the press release[1]. Feedback > welcome. With respect to the linking, I tried I to give a bunch of > jumping off points for users to explore the features, but visually tried > to ensure the readability was consistent. + <para> + Reduction of <acronym>WAL</acronym> overhead of + <link linkend="gist">GiST</link>, <link linkend="gin">GIN</link>, and + <link linkend="spgist">SP-GiST</link> indexes and added support + for covering indexes via the <link linkend="sql-createindex"> + <literal>INCLUDE</literal></link> clause for + <link linkend="spgist">SP-GiST</link> indexes + </para> Maybe I'm missing something, but covering indexes are supported in GiST, not in SP-GiST. Best regards, Andrey Borodin.
On 9/2/19 1:37 PM, Andrey Borodin wrote: > > >> 2 сент. 2019 г., в 22:02, Jonathan S. Katz <jkatz@postgresql.org> написал(а): >> >> >> Attached is a patch proposing items for the major items section. This is >> working off of the ongoing draft of the press release[1]. Feedback >> welcome. With respect to the linking, I tried I to give a bunch of >> jumping off points for users to explore the features, but visually tried >> to ensure the readability was consistent. > > + <para> > + Reduction of <acronym>WAL</acronym> overhead of > + <link linkend="gist">GiST</link>, <link linkend="gin">GIN</link>, and > + <link linkend="spgist">SP-GiST</link> indexes and added support > + for covering indexes via the <link linkend="sql-createindex"> > + <literal>INCLUDE</literal></link> clause for > + <link linkend="spgist">SP-GiST</link> indexes > + </para> > > Maybe I'm missing something, but covering indexes are supported in GiST, not in SP-GiST. Nope, you're absolutely correct: that was a typo as a result of copying/pasting formatting. Attached is a revision that correctly specifies covering indexes for GiST. Thanks! Jonathan
Attachment
On 5/12/19 11:42 PM, Bruce Momjian wrote: > On Sun, May 12, 2019 at 10:49:07AM -0400, Jonathan Katz wrote: >> Hi Bruce, >> >> On 5/11/19 4:33 PM, Bruce Momjian wrote: >>> I have posted a draft copy of the PG 12 release notes here: >>> >>> http://momjian.us/pgsql_docs/release-12.html >>> >>> They are committed to git. It includes links to the main docs, where >>> appropriate. Our official developer docs will rebuild in a few hours. >> >> Thank you for working on this, I know it's a gargantuan task. >> >> I have a small modification for a section entitled "Source Code" which >> is a repeat of the previous section. Based on the bullet points in that >> part, I thought "Documentation" might be a more appropriate name; please >> see attached. > > Yes, I saw that myself and just updated it. Thanks. Great! There is a placeholder for PG_COLORS: <para> This is enabled with by setting environment variable <envar>PG_COLORS</envar>. EXAMPLE? </para> I've attached the following that provides an example of how to use this environmental variable. Thanks! Jonathan