Thread: [HACKERS] PG 10 release notes
I have committed the first draft of the Postgres 10 release notes. They are current as of two days ago, and I will keep them current. Please give me any feedback you have. The only unusual thing is that this release has ~180 items while most recent release have had ~220. The pattern I see that there are more large features in this release than previous ones. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On 04/25/2017 03:31 AM, Bruce Momjian wrote: > I have committed the first draft of the Postgres 10 release notes. They > are current as of two days ago, and I will keep them current. Please > give me any feedback you have. This item is incorrectly attributed to me. I was only the reviewer, Peter is the author. +<listitem> +<!-- +Author: Peter Eisentraut <peter_e@gmx.net> +2016-12-20 [1753b1b02] Add pg_sequence system catalog +--> +<para> +Create a <link linkend="catalog-pg-sequence"><structname>pg_sequence</></> system catalog to store sequence metadata (Andreas +Karlsson) +</para> Andreas
On Tue, Apr 25, 2017 at 03:45:52AM +0200, Andreas Karlsson wrote: > On 04/25/2017 03:31 AM, Bruce Momjian wrote: > >I have committed the first draft of the Postgres 10 release notes. They > >are current as of two days ago, and I will keep them current. Please > >give me any feedback you have. > > This item is incorrectly attributed to me. I was only the reviewer, Peter is > the author. > > +<listitem> > +<!-- > +Author: Peter Eisentraut <peter_e@gmx.net> > +2016-12-20 [1753b1b02] Add pg_sequence system catalog > +--> > +<para> > +Create a <link linkend="catalog-pg-sequence"><structname>pg_sequence</></> > system catalog to store sequence metadata (Andreas > +Karlsson) > +</para> OK, fixed. I also moved some "incompatibility" items into the right section. FYI, you can see the most recent doc build here: http://momjian.us/pgsql_docs/release-10.html -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Hello, Bruce > From: pgsql-hackers-owner@postgresql.org > [mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Bruce Momjian > I have committed the first draft of the Postgres 10 release notes. They > are current as of two days ago, and I will keep them current. Please give > me any feedback you have. Thanks for a nice release note. Below is what I noticed: (1) Support parallel bitmap heap scans (Dilip Kum) This item appears twice. (2) E.1.3.1.4. Optimizer Remove SCO and Unixware ports (Tom Lane) The section doesn't seem appropriate. Is OS-specific code related to the optimizer? (3) Remove documented restriction about using large shared buffers on Windows (Tsunakawa, Takayuki) Please change the name to "Takayuki Tsunakawa". (4) Have to_timestamp() and to_date() check check input values for validity (Artur Zakirov) "check" is repeated. (5) Use POSIX semaphores rather than SysV semaphores on Linux and FreeBSD (Tom Lane) This avoids some limits on SysV semaphores usage. These two lines are put in two different items. Regards Takayuki Tsunakawa
All fixed, thanks. --------------------------------------------------------------------------- On Tue, Apr 25, 2017 at 02:40:23AM +0000, Tsunakawa, Takayuki wrote: > Hello, Bruce > > > From: pgsql-hackers-owner@postgresql.org > > [mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Bruce Momjian > > I have committed the first draft of the Postgres 10 release notes. They > > are current as of two days ago, and I will keep them current. Please give > > me any feedback you have. > > > Thanks for a nice release note. Below is what I noticed: > > > (1) > Support parallel bitmap heap scans (Dilip Kum) > > This item appears twice. > > > (2) > E.1.3.1.4. Optimizer > Remove SCO and Unixware ports (Tom Lane) > > The section doesn't seem appropriate. Is OS-specific code related to the optimizer? > > > > (3) > Remove documented restriction about using large shared buffers on Windows (Tsunakawa, Takayuki) > > Please change the name to "Takayuki Tsunakawa". > > > (4) > Have to_timestamp() and to_date() check check input values for validity (Artur Zakirov) > > "check" is repeated. > > > (5) > Use POSIX semaphores rather than SysV semaphores on Linux and FreeBSD (Tom Lane) > This avoids some limits on SysV semaphores usage. > > These two lines are put in two different items. > > > Regards > Takayuki Tsunakawa > -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 7:01 AM, Bruce Momjian <bruce@momjian.us> wrote: > I have committed the first draft of the Postgres 10 release notes. They > are current as of two days ago, and I will keep them current. Please > give me any feedback you have. > Some of the items which I feel could be added: 5e6d8d2bbbcace304450b309e79366c0da4063e4 Allow parallel workers to execute subplans. 61c2e1a95f94bb904953a6281ce17a18ac38ee6d Improve access to parallel query from procedural languages. In Parallel Queries section, we can add above two items as they increase the usage of the parallel query in many cases. ea69a0dead5128c421140dc53fac165ba4af8520 Expand hash indexes more gradually. I think the above commit needs a separate mention, as this is a really huge step forward to control the size of hash indexes. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
On Tue, Apr 25, 2017 at 08:30:50AM +0530, Amit Kapila wrote: > On Tue, Apr 25, 2017 at 7:01 AM, Bruce Momjian <bruce@momjian.us> wrote: > > I have committed the first draft of the Postgres 10 release notes. They > > are current as of two days ago, and I will keep them current. Please > > give me any feedback you have. > > > > Some of the items which I feel could be added: > > 5e6d8d2bbbcace304450b309e79366c0da4063e4 > Allow parallel workers to execute subplans. Uh, can you show me the commit on that and give some text ideas? > 61c2e1a95f94bb904953a6281ce17a18ac38ee6d > Improve access to parallel query from procedural languages. I think I have that: Increase parallel query usage in procedural language functions (RobertHaas) > In Parallel Queries section, we can add above two items as they > increase the usage of the parallel query in many cases. > > ea69a0dead5128c421140dc53fac165ba4af8520 > Expand hash indexes more gradually. That is in this item: Improve hash bucket split performance by reducing locking requirements(Amit Kapila, Mithun Cy)Also cache hash index meta-informationfor faster lookups. Additionalhash performance improvements have also been made. pg_upgrade'd hashindexesfrom previous major Postgres versions must be rebuilt. Can you suggest additional wording? I did merge many of the hash items into this so it would be understandable. You can see the commits in the SGML source. > I think the above commit needs a separate mention, as this is a really > huge step forward to control the size of hash indexes. Yes, it is unfotunate that the item is in the incompatibility item. I wonder if I should split out the need to rebuild the hash indexes and keep it there and move this item into the "Index" section. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 8:30 AM, Amit Kapila <amit.kapila16@gmail.com> wrote: > On Tue, Apr 25, 2017 at 7:01 AM, Bruce Momjian <bruce@momjian.us> wrote: >> I have committed the first draft of the Postgres 10 release notes. They >> are current as of two days ago, and I will keep them current. Please >> give me any feedback you have. >> > > Some of the items which I feel could be added: > > 5e6d8d2bbbcace304450b309e79366c0da4063e4 > Allow parallel workers to execute subplans. > > 61c2e1a95f94bb904953a6281ce17a18ac38ee6d > Improve access to parallel query from procedural languages. > > In Parallel Queries section, we can add above two items as they > increase the usage of the parallel query in many cases. > I see the second one in release notes, but maybe we should credit Rafia Sabih as well for that feature as she was the original author of the patch and has helped in fixing the bugs that occurred due to that feature. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
On Tue, Apr 25, 2017 at 08:36:38AM +0530, Amit Kapila wrote: > On Tue, Apr 25, 2017 at 8:30 AM, Amit Kapila <amit.kapila16@gmail.com> wrote: > > On Tue, Apr 25, 2017 at 7:01 AM, Bruce Momjian <bruce@momjian.us> wrote: > >> I have committed the first draft of the Postgres 10 release notes. They > >> are current as of two days ago, and I will keep them current. Please > >> give me any feedback you have. > >> > > > > Some of the items which I feel could be added: > > > > 5e6d8d2bbbcace304450b309e79366c0da4063e4 > > Allow parallel workers to execute subplans. > > > > 61c2e1a95f94bb904953a6281ce17a18ac38ee6d > > Improve access to parallel query from procedural languages. > > > > In Parallel Queries section, we can add above two items as they > > increase the usage of the parallel query in many cases. > > > > I see the second one in release notes, but maybe we should credit > Rafia Sabih as well for that feature as she was the original author of > the patch and has helped in fixing the bugs that occurred due to that > feature. Added. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Mon, Apr 24, 2017 at 11:05:41PM -0400, Bruce Momjian wrote: > > I think the above commit needs a separate mention, as this is a really > > huge step forward to control the size of hash indexes. > > Yes, it is unfotunate that the item is in the incompatibility item. I > wonder if I should split out the need to rebuild the hash indexes and > keep it there and move this item into the "Index" section. Done, items split. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 8:35 AM, Bruce Momjian <bruce@momjian.us> wrote: > On Tue, Apr 25, 2017 at 08:30:50AM +0530, Amit Kapila wrote: >> On Tue, Apr 25, 2017 at 7:01 AM, Bruce Momjian <bruce@momjian.us> wrote: >> > I have committed the first draft of the Postgres 10 release notes. They >> > are current as of two days ago, and I will keep them current. Please >> > give me any feedback you have. >> > >> >> Some of the items which I feel could be added: >> >> 5e6d8d2bbbcace304450b309e79366c0da4063e4 >> Allow parallel workers to execute subplans. > > Uh, can you show me the commit on that and give some text ideas? > I have already mentioned the commit id (5e6d8d2b). Text can be "Allow queries containing subplans to execute in parallel". We should also mention in some way that this applies only when the query contains uncorrelated subplan. >> 61c2e1a95f94bb904953a6281ce17a18ac38ee6d >> Improve access to parallel query from procedural languages. > > I think I have that: > > Increase parallel query usage in procedural language functions (Robert > Haas) > >> In Parallel Queries section, we can add above two items as they >> increase the usage of the parallel query in many cases. >> >> ea69a0dead5128c421140dc53fac165ba4af8520 >> Expand hash indexes more gradually. > > That is in this item: > > Improve hash bucket split performance by reducing locking requirements > (Amit Kapila, Mithun Cy) > > Also cache hash index meta-information for faster lookups. Additional > hash performance improvements have also been made. pg_upgrade'd hash > indexes from previous major Postgres versions must be rebuilt. > > Can you suggest additional wording? > Allow hash indexes to expand slowly This will help in controlling the size of hash indexes after the split. > I did merge many of the hash items > into this so it would be understandable. You can see the commits in the > SGML source. > >> I think the above commit needs a separate mention, as this is a really >> huge step forward to control the size of hash indexes. > > Yes, it is unfotunate that the item is in the incompatibility item. I > wonder if I should split out the need to rebuild the hash indexes and > keep it there and move this item into the "Index" section. > That sounds sensible. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
On 2017-04-24 21:31:44 -0400, Bruce Momjian wrote: > I have committed the first draft of the Postgres 10 release notes. They > are current as of two days ago, and I will keep them current. Please > give me any feedback you have. > > The only unusual thing is that this release has ~180 items while most > recent release have had ~220. The pattern I see that there are more > large features in this release than previous ones. I think that might also be because you skipped a few things that should get their own entries. I've not yet made a pass through your draft (and won't for some days), but a quick search shows the draft to e.g miss: b30d3ea824c5ccba43d3e942704f20686e7dbab8 - Add a macro templatized hashtable. 75ae538bc3168bf44475240d4e0487ee2f3bb376 - Use more efficient hashtable for tidbitmap.c to speed up bitmap scans. 5dfc198146b49ce7ecc8a1fc9d5e171fb75f6ba5 - Use more efficient hashtable for execGrouping.c to speed up hash aggregation. fc4b3dea2950e4f6081f1ed2380f82c9efd672e0 - User narrower representative tuples in the hash-agg hashtable. 8ed3f11bb045ad7a3607690be668dbd5b3cc31d7 - Perform one only projection to compute agg arguments. (not that this needs to five entries) - Andres
On Mon, Apr 24, 2017 at 08:36:00PM -0700, Andres Freund wrote: > On 2017-04-24 21:31:44 -0400, Bruce Momjian wrote: > > I have committed the first draft of the Postgres 10 release notes. They > > are current as of two days ago, and I will keep them current. Please > > give me any feedback you have. > > > > The only unusual thing is that this release has ~180 items while most > > recent release have had ~220. The pattern I see that there are more > > large features in this release than previous ones. > > I think that might also be because you skipped a few things that should > get their own entries. I've not yet made a pass through your draft (and > won't for some days), but a quick search shows the draft to e.g miss: > b30d3ea824c5ccba43d3e942704f20686e7dbab8 - Add a macro templatized hashtable. > 75ae538bc3168bf44475240d4e0487ee2f3bb376 - Use more efficient hashtable for tidbitmap.c to speed up bitmap scans. > 5dfc198146b49ce7ecc8a1fc9d5e171fb75f6ba5 - Use more efficient hashtable for execGrouping.c to speed up hash aggregation. > fc4b3dea2950e4f6081f1ed2380f82c9efd672e0 - User narrower representative tuples in the hash-agg hashtable. > 8ed3f11bb045ad7a3607690be668dbd5b3cc31d7 - Perform one only projection to compute agg arguments. > (not that this needs to five entries) I remember seeing those and those are normally details I do not put in the release notes as there isn't a clear user experience change except "Postgres is faster". Yeah, a bummer, and I can change my filter, but it would require discussion. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On 2017-04-24 23:37:42 -0400, Bruce Momjian wrote: > On Mon, Apr 24, 2017 at 08:36:00PM -0700, Andres Freund wrote: > > On 2017-04-24 21:31:44 -0400, Bruce Momjian wrote: > > > I have committed the first draft of the Postgres 10 release notes. They > > > are current as of two days ago, and I will keep them current. Please > > > give me any feedback you have. > > > > > > The only unusual thing is that this release has ~180 items while most > > > recent release have had ~220. The pattern I see that there are more > > > large features in this release than previous ones. > > > > I think that might also be because you skipped a few things that should > > get their own entries. I've not yet made a pass through your draft (and > > won't for some days), but a quick search shows the draft to e.g miss: > > b30d3ea824c5ccba43d3e942704f20686e7dbab8 - Add a macro templatized hashtable. > > 75ae538bc3168bf44475240d4e0487ee2f3bb376 - Use more efficient hashtable for tidbitmap.c to speed up bitmap scans. > > 5dfc198146b49ce7ecc8a1fc9d5e171fb75f6ba5 - Use more efficient hashtable for execGrouping.c to speed up hash aggregation. > > fc4b3dea2950e4f6081f1ed2380f82c9efd672e0 - User narrower representative tuples in the hash-agg hashtable. > > 8ed3f11bb045ad7a3607690be668dbd5b3cc31d7 - Perform one only projection to compute agg arguments. > > (not that this needs to five entries) > > I remember seeing those and those are normally details I do not put in > the release notes as there isn't a clear user experience change except > "Postgres is faster". Yeah, a bummer, and I can change my filter, but > it would require discussion. I think "postgres is faster" is one of the bigger user demands, so I don't think that policy makes much sense. A large number of the changes over the next few releases will focus solely on that. Nor do I think past release notes particularly filtered such changes out. - Andres
Andres Freund <andres@anarazel.de> writes:
> On 2017-04-24 23:37:42 -0400, Bruce Momjian wrote:
>> I remember seeing those and those are normally details I do not put in
>> the release notes as there isn't a clear user experience change except
>> "Postgres is faster". Yeah, a bummer, and I can change my filter, but
>> it would require discussion.
> I think "postgres is faster" is one of the bigger user demands, so I
> don't think that policy makes much sense. A large number of the changes
> over the next few releases will focus solely on that. Nor do I think
> past release notes particularly filtered such changes out.
I think it has been pretty common to accumulate a lot of such changes
into generic entries like, say, "speedups for hash joins". More detail
than that simply isn't useful to end users; and as a rule, our release
notes are too long anyway.
regards, tom lane
On 2017-04-24 23:45:06 -0400, Tom Lane wrote: > Andres Freund <andres@anarazel.de> writes: > > On 2017-04-24 23:37:42 -0400, Bruce Momjian wrote: > >> I remember seeing those and those are normally details I do not put in > >> the release notes as there isn't a clear user experience change except > >> "Postgres is faster". Yeah, a bummer, and I can change my filter, but > >> it would require discussion. > > > I think "postgres is faster" is one of the bigger user demands, so I > > don't think that policy makes much sense. A large number of the changes > > over the next few releases will focus solely on that. Nor do I think > > past release notes particularly filtered such changes out. > > I think it has been pretty common to accumulate a lot of such changes > into generic entries like, say, "speedups for hash joins". More detail > than that simply isn't useful to end users; and as a rule, our release > notes are too long anyway. Oh, I completely agree with accumulating related changes, and that code-level details aren't useful. I think we skipped them entirely here. And I just listed my own changes because I could find them quickly, but they're not alone, e.g: 090010f2ec9b1f9ac1124dc628b89586f911b641 - Improve performance of find_tabstat_entry()/get_tabstat_entry() which makes it realistic to have sessions touching many relations, which previously was O(#relations^2), and which caused repeated complaints over the years, and allows for different usecases. - Andres
Hi, I wonder if there's a reasonable way that allows to add links to the more crucial commits for changelog entries. The source e.g. has <listitem> <!-- Author: Robert Haas <rhaas@postgresql.org> 2017-03-08 [98e6e8904] tidbitmap: Support shared iteration. Author: Robert Haas <rhaas@postgresql.org> 2017-03-08 [f35742ccb] Support parallel bitmap heap scans. --> <para> Support parallel bitmap heap scans (Dilip Kumar) </para> <para> This allows a single index scan to dispatch parallel workers to process different areas of the heap. </para> </listitem> for an item, and it'd be pretty cool if we could have a link to those two commits from the entry. It'd need be pretty unobstrusive to avoid making things hard to read, but it'd obviate some of the need to list details, and it gives curious people changes to see what actually changed. - Andres
..On 25 April 2017 at 13:31, Bruce Momjian <bruce@momjian.us> wrote: > The only unusual thing is that this release has ~180 items while most > recent release have had ~220. The pattern I see that there are more > large features in this release than previous ones. Thanks for drafting this up. I understand that it may have been filtered out, but I'd say that 7e534adcdc70 is worth a mention. Users creating BRIN indexes previously would have had to know beforehand that the table would be sufficiently correlated on the indexed columns for the index to be worthwhile, whereas now there's a lesser need for the user to know this beforehand. Also: <para> New commands are <command><link linkend="SQL-CREATESTATISTICS">CREATE</></>, <command><link linkend="SQL-ALTERSTATISTICS">ALTER</></>, and <command><link linkend="SQL-DROPSTATISTICS">DROP STATISTICS</></>. This is helpful in estimating query memory usage and ... HOW IS RATIO USED? </para> HOW IS RATIO USED? .... There are two types of new stats; ndistinct, which can improve row estimations in the query planner for estimating the number of distinct value groups over multiple columns. functionaldeps, which provides the planner with a better understanding of functional depdenences between columns on which the statistics are defined. The planner takes the functional dependency degree into account when multiplying selectivity estimations for multiple columns. Unsure how best to trim that down to something short enough for the release notes. -- David Rowley http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
On Tue, Apr 25, 2017 at 9:15 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Andres Freund <andres@anarazel.de> writes: >> On 2017-04-24 23:37:42 -0400, Bruce Momjian wrote: >>> I remember seeing those and those are normally details I do not put in >>> the release notes as there isn't a clear user experience change except >>> "Postgres is faster". Yeah, a bummer, and I can change my filter, but >>> it would require discussion. > >> I think "postgres is faster" is one of the bigger user demands, so I >> don't think that policy makes much sense. A large number of the changes >> over the next few releases will focus solely on that. Nor do I think >> past release notes particularly filtered such changes out. > > I think it has been pretty common to accumulate a lot of such changes > into generic entries like, say, "speedups for hash joins". More detail > than that simply isn't useful to end users; and as a rule, our release > notes are too long anyway. > > regards, tom lane > > Just wondering if the mention of commit 0414b26bac09379a4cbf1fbd847d1cee2293c5e4 is missed? Not sure if this requires a separate entry or could be merged with -- Support parallel btree index scans. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/
On 2017/04/25 10:31, Bruce Momjian wrote: > I have committed the first draft of the Postgres 10 release notes. They > are current as of two days ago, and I will keep them current. Please > give me any feedback you have. > > The only unusual thing is that this release has ~180 items while most > recent release have had ~220. The pattern I see that there are more > large features in this release than previous ones. Thanks for this. Wondering if the following really belongs under postgres_fdw improvements: +<listitem> +<!-- +Author: Robert Haas <rhaas@postgresql.org> +2017-03-16 [b30fb56b0] postgres_fdw: Push down <literal>FULL JOIN</>s with restriction clau +--> +<para> +Improve optimization of <literal>FULL JOIN</> queries containing subqueries in the +<literal>FROM</> clause (Etsuro Fujita) +</para> +</listitem> Maybe, nearby the following: + <sect3> + <title>Additional Modules</title> ... +<listitem> +<!-- +Author: Robert Haas <rhaas@postgresql.org> +2016-10-21 [7012b132d] postgres_fdw: Push down aggregates to remote servers. +--> +<para> +Push aggregates to foreign data wrapper servers, where possible (Jeevan +Chalke, Ashutosh Bapat) +</para> Thanks, Amit
On 25/04/17 03:31, Bruce Momjian wrote: > I have committed the first draft of the Postgres 10 release notes. They > are current as of two days ago, and I will keep them current. Please > give me any feedback you have. > Cool! > The only unusual thing is that this release has ~180 items while most > recent release have had ~220. The pattern I see that there are more > large features in this release than previous ones. > Well I think for example > Optimizer > > Add the ability to compute a correlation ratio and the number of distinct values on several columns (Tomas Vondra,David Rowley) > could be easily 2 or 3 items (explicitly defining additional statistics, multicolumn ndistinct and functional dependencies). I also wonder if ability to run SQL queries on walsender connected to a database is worth mentioning (replication=database kind of connection). Or the ability of logical decoding to follow timeline switches. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Commit f9b1a0dd403ec0931213c66d5f979a3d3e8e7e30 mentions "Ashutosh Bapat" as author, which is not reflected in the release notes -- Allow explicit control over EXPLAIN's display of planning and execution time (Stephen Frost) By default planning and execution is display by EXPLAIN ANALYZE and not display in other cases. The new EXPLAIN option SUMMARY allows explicit control of this. -- On Tue, Apr 25, 2017 at 7:24 AM, Bruce Momjian <bruce@momjian.us> wrote: > On Tue, Apr 25, 2017 at 03:45:52AM +0200, Andreas Karlsson wrote: >> On 04/25/2017 03:31 AM, Bruce Momjian wrote: >> >I have committed the first draft of the Postgres 10 release notes. They >> >are current as of two days ago, and I will keep them current. Please >> >give me any feedback you have. >> >> This item is incorrectly attributed to me. I was only the reviewer, Peter is >> the author. >> >> +<listitem> >> +<!-- >> +Author: Peter Eisentraut <peter_e@gmx.net> >> +2016-12-20 [1753b1b02] Add pg_sequence system catalog >> +--> >> +<para> >> +Create a <link linkend="catalog-pg-sequence"><structname>pg_sequence</></> >> system catalog to store sequence metadata (Andreas >> +Karlsson) >> +</para> > > OK, fixed. I also moved some "incompatibility" items into the right > section. FYI, you can see the most recent doc build here: > > http://momjian.us/pgsql_docs/release-10.html > > -- > Bruce Momjian <bruce@momjian.us> http://momjian.us > EnterpriseDB http://enterprisedb.com > > + As you are, so once was I. As I am, so you will be. + > + Ancient Roman grave inscription + > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company
On Tue, Apr 25, 2017 at 10:54 AM, Bruce Momjian <bruce@momjian.us> wrote: > On Tue, Apr 25, 2017 at 03:45:52AM +0200, Andreas Karlsson wrote: >> On 04/25/2017 03:31 AM, Bruce Momjian wrote: >> >I have committed the first draft of the Postgres 10 release notes. They >> >are current as of two days ago, and I will keep them current. Please >> >give me any feedback you have. >> >> This item is incorrectly attributed to me. I was only the reviewer, Peter is >> the author. >> >> +<listitem> >> +<!-- >> +Author: Peter Eisentraut <peter_e@gmx.net> >> +2016-12-20 [1753b1b02] Add pg_sequence system catalog >> +--> >> +<para> >> +Create a <link linkend="catalog-pg-sequence"><structname>pg_sequence</></> >> system catalog to store sequence metadata (Andreas >> +Karlsson) >> +</para> > > OK, fixed. I also moved some "incompatibility" items into the right > section. FYI, you can see the most recent doc build here: > > http://momjian.us/pgsql_docs/release-10.html Here is some feedback based on a first read. +<para> +Issue fsync on the output files generated by <application>pg_dump</> and <application>pg_dumpall</>(Michael Paquier) +</para> Missing a space here between pg_dumpall and my name. <para> This is accessed via <function>ts_headline()</> and to_tsvector. RIGHT SECTION? </para> </listitem> This should use a <function> markup for both names. <para> Add <link linkend="auth-pg-hba-conf"><literal>SCRAM-SHA-256</></> support for password negotiation and storage (Michael Paquier, Heikki Linnakangas) </para> <para> This proves better security than the existing 'md5' negotiation and storage method. </para> This is quite vague... <listitem> <!-- Author: Robert Haas <rhaas@postgresql.org> 2016-11-21 [a734fd5d1] autovacuum: Drop orphan temp tables more quickly but wit Author: Tom Lane <tgl@sss.pgh.pa.us> 2016-11-27 [dafa0848d] Code review for early drop of orphaned temp relations in --> <para> Remove orphaned temporary tables more aggressively (Robert Haas, Tom Lane) </para> <para> Previously such tables were removed only when necessary. SECTION? </para> </listitem> Well, I wrote the first patch that has been committed, until Tom showed up and rewrote everything :) perhaps this could be in a section dedicated to autovacuum? <listitem> <!-- Author: Robert Haas <rhaas@postgresql.org> 2017-02-08 [a507b8690] Add WAL consistency checking facility. Author: Robert Haas <rhaas@postgresql.org> 2017-03-14 [bb4a39637] hash: Support WAL consistency checking. --> <para> Add <acronym>GUC</> <xref linkend="guc-wal-consistency-checking"> to add details to <acronym>WAL</> that can be sanity-checked on the standby (Kuntal Ghosh, Michael Paquier, Robert Haas) </para> I would be fine if my name is removed here. Kuntal has done all the work, I just reviewed his code. <para> Previously only specification of the stop name, time, and xid were supported. </para> </listitem> Missing timeline and 'immediate' here. <listitem> <!-- Author: Simon Riggs <simon@2ndQuadrant.com> 2017-04-04 [728bd991c] Speedup 2PC recovery by skipping two phase state files i --> <para> Speed up two-phase commit recovery performance (Simon Riggs) </para> </listitem> Simon is the committer here, authors are marked as Stas Kelvich, Nikhil Sontakke and Michael Paquier. <listitem> <!-- Author: Tom Lane <tgl@sss.pgh.pa.us> 2016-08-15 [ca9112a42] Stamp HEAD as 10devel. --> <para> New major version numbering (Peter Eisentraut, Tom Lane) </para> <para> Major versions will now increase just the first number, and minor releases will increase just the second number. A third number will no longer be used in Postgres version numbers. </para> </listitem> Perhaps this should be higher in the list? -- Michael
Hello Bruce,
On Tue, Apr 25, 2017 at 3:31 AM, Bruce Momjian <bruce@momjian.us> wrote:
On Tue, Apr 25, 2017 at 3:31 AM, Bruce Momjian <bruce@momjian.us> wrote:
I have committed the first draft of the Postgres 10 release notes. They
are current as of two days ago, and I will keep them current. Please
give me any feedback you have.
I just noticed:
Thanks,
Félix
On Tue, Apr 25, 2017 at 09:00:45AM +0530, Amit Kapila wrote: > On Tue, Apr 25, 2017 at 8:35 AM, Bruce Momjian <bruce@momjian.us> wrote: > > On Tue, Apr 25, 2017 at 08:30:50AM +0530, Amit Kapila wrote: > >> On Tue, Apr 25, 2017 at 7:01 AM, Bruce Momjian <bruce@momjian.us> wrote: > >> > I have committed the first draft of the Postgres 10 release notes. They > >> > are current as of two days ago, and I will keep them current. Please > >> > give me any feedback you have. > >> > > >> > >> Some of the items which I feel could be added: > >> > >> 5e6d8d2bbbcace304450b309e79366c0da4063e4 > >> Allow parallel workers to execute subplans. > > > > Uh, can you show me the commit on that and give some text ideas? > > > > I have already mentioned the commit id (5e6d8d2b). Text can be "Allow > queries containing subplans to execute in parallel". We should also > mention in some way that this applies only when the query contains > uncorrelated subplan. Sorry but I don't know what that means, and if I don't know, others might not either. > >> 61c2e1a95f94bb904953a6281ce17a18ac38ee6d > >> Improve access to parallel query from procedural languages. > > > > I think I have that: > > > > Increase parallel query usage in procedural language functions (Robert > > Haas) > > > >> In Parallel Queries section, we can add above two items as they > >> increase the usage of the parallel query in many cases. > >> > >> ea69a0dead5128c421140dc53fac165ba4af8520 > >> Expand hash indexes more gradually. > > > > That is in this item: > > > > Improve hash bucket split performance by reducing locking requirements > > (Amit Kapila, Mithun Cy) > > > > Also cache hash index meta-information for faster lookups. Additional > > hash performance improvements have also been made. pg_upgrade'd hash > > indexes from previous major Postgres versions must be rebuilt. > > > > Can you suggest additional wording? > > > > Allow hash indexes to expand slowly > > This will help in controlling the size of hash indexes after the split. OK, I split out the "growth" item into a separate item and moved the hash items into a separate section now that there are enough of them to warrant it. > > I did merge many of the hash items > > into this so it would be understandable. You can see the commits in the > > SGML source. > > > >> I think the above commit needs a separate mention, as this is a really > >> huge step forward to control the size of hash indexes. > > > > Yes, it is unfotunate that the item is in the incompatibility item. I > > wonder if I should split out the need to rebuild the hash indexes and > > keep it there and move this item into the "Index" section. > > > > That sounds sensible. Yes, already done. Again, the most current doc build is here: http://momjian.us/pgsql_docs/release-10.html -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Mon, Apr 24, 2017 at 08:39:59PM -0700, Andres Freund wrote: > On 2017-04-24 23:37:42 -0400, Bruce Momjian wrote: > > I remember seeing those and those are normally details I do not put in > > the release notes as there isn't a clear user experience change except > > "Postgres is faster". Yeah, a bummer, and I can change my filter, but > > it would require discussion. > > I think "postgres is faster" is one of the bigger user demands, so I > don't think that policy makes much sense. A large number of the changes > over the next few releases will focus solely on that. Nor do I think > past release notes particularly filtered such changes out. Can you supply an example of this? Ideally it would be from pre-9.6 because Tom did 9.6, but I think he used the same filtering I use. I seek to be consistent in my filtering. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Mon, Apr 24, 2017 at 08:52:05PM -0700, Andres Freund wrote: > On 2017-04-24 23:45:06 -0400, Tom Lane wrote: > > Andres Freund <andres@anarazel.de> writes: > > > On 2017-04-24 23:37:42 -0400, Bruce Momjian wrote: > > >> I remember seeing those and those are normally details I do not put in > > >> the release notes as there isn't a clear user experience change except > > >> "Postgres is faster". Yeah, a bummer, and I can change my filter, but > > >> it would require discussion. > > > > > I think "postgres is faster" is one of the bigger user demands, so I > > > don't think that policy makes much sense. A large number of the changes > > > over the next few releases will focus solely on that. Nor do I think > > > past release notes particularly filtered such changes out. > > > > I think it has been pretty common to accumulate a lot of such changes > > into generic entries like, say, "speedups for hash joins". More detail > > than that simply isn't useful to end users; and as a rule, our release > > notes are too long anyway. > > Oh, I completely agree with accumulating related changes, and that > code-level details aren't useful. I think we skipped them entirely > here. And I just listed my own changes because I could find them > quickly, but they're not alone, e.g: > 090010f2ec9b1f9ac1124dc628b89586f911b641 - Improve performance of find_tabstat_entry()/get_tabstat_entry() > which makes it realistic to have sessions touching many relations, which > previously was O(#relations^2), and which caused repeated complaints > over the years, and allows for different usecases. Looking at this commit it appears to improve pg_stat statistics handling. I don't see how that improves performance except to improve statistics aggregation, which happens in the statistics process. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 09:38:55AM +0530, Rafia Sabih wrote: > On Tue, Apr 25, 2017 at 9:15 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > > Andres Freund <andres@anarazel.de> writes: > >> On 2017-04-24 23:37:42 -0400, Bruce Momjian wrote: > >>> I remember seeing those and those are normally details I do not put in > >>> the release notes as there isn't a clear user experience change except > >>> "Postgres is faster". Yeah, a bummer, and I can change my filter, but > >>> it would require discussion. > > > >> I think "postgres is faster" is one of the bigger user demands, so I > >> don't think that policy makes much sense. A large number of the changes > >> over the next few releases will focus solely on that. Nor do I think > >> past release notes particularly filtered such changes out. > > > > I think it has been pretty common to accumulate a lot of such changes > > into generic entries like, say, "speedups for hash joins". More detail > > than that simply isn't useful to end users; and as a rule, our release > > notes are too long anyway. > > > > regards, tom lane > > > > > Just wondering if the mention of commit > 0414b26bac09379a4cbf1fbd847d1cee2293c5e4 is missed? Not sure if this > requires a separate entry or could be merged with -- Support parallel > btree index scans. This item was merged into the general parallel index scan item: Author: Robert Haas <rhaas@postgresql.org>2017-02-15 [569174f1b] btree: Support parallel index scans.Author: Robert Haas<rhaas@postgresql.org>2017-02-15 [5262f7a4f] Add optimizer and executor support for parallel index scAuthor: Robert Haas<rhaas@postgresql.org>2017-02-19 [0414b26ba] Add optimizer and executor support for parallel index-onSupport parallelbtree index scans (Rahila Syed, Amit Kapila, Robert -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Mon, Apr 24, 2017 at 08:55:33PM -0700, Andres Freund wrote: > Hi, > > I wonder if there's a reasonable way that allows to add links to the > more crucial commits for changelog entries. The source e.g. has > > <listitem> > <!-- > Author: Robert Haas <rhaas@postgresql.org> > 2017-03-08 [98e6e8904] tidbitmap: Support shared iteration. > Author: Robert Haas <rhaas@postgresql.org> > 2017-03-08 [f35742ccb] Support parallel bitmap heap scans. > --> > <para> > Support parallel bitmap heap scans (Dilip Kumar) > </para> > > <para> > This allows a single index scan to dispatch parallel workers to process > different areas of the heap. > </para> > </listitem> > > for an item, and it'd be pretty cool if we could have a link to those > two commits from the entry. It'd need be pretty unobstrusive to avoid > making things hard to read, but it'd obviate some of the need to list > details, and it gives curious people changes to see what actually > changed. If the SGML comments were passed into the HTML, I could have used JavaScript to pull out the git hashes and point them to our gitweb mirror. Unfortunately, they are not, so we would have to find a way to pass those comments into the HTML. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 04:03:53PM +1200, David Rowley wrote: > ..On 25 April 2017 at 13:31, Bruce Momjian <bruce@momjian.us> wrote: > > The only unusual thing is that this release has ~180 items while most > > recent release have had ~220. The pattern I see that there are more > > large features in this release than previous ones. > > Thanks for drafting this up. > > I understand that it may have been filtered out, but I'd say that > 7e534adcdc70 is worth a mention. > > Users creating BRIN indexes previously would have had to know > beforehand that the table would be sufficiently correlated on the > indexed columns for the index to be worthwhile, whereas now there's a > lesser need for the user to know this beforehand. I can't see how this can be added to an existing BRIN entry, so it would have to be new. The text would be: Improve accuracy in determining if a BRIN index scan is beneficial though this not something I would normally mention becuause most users don't understand the optimizer choices and just assume it works. > Also: > > <para> > New commands are <command><link linkend="SQL-CREATESTATISTICS">CREATE</></>, > <command><link linkend="SQL-ALTERSTATISTICS">ALTER</></>, and > <command><link linkend="SQL-DROPSTATISTICS">DROP STATISTICS</></>. > This is helpful in > estimating query memory usage and ... HOW IS RATIO USED? > </para> > > HOW IS RATIO USED? .... There are two types of new stats; > > ndistinct, which can improve row estimations in the query planner for > estimating the number of distinct value groups over multiple columns. Yes, I understood that one. > functionaldeps, which provides the planner with a better understanding > of functional depdenences between columns on which the statistics are > defined. The planner takes the functional dependency degree into > account when multiplying selectivity estimations for multiple columns. Ah, so it is used when combining estimates --- makes sense. New text: This is helpful in estimating query memory usage and when combiningthe statistics from individual columns. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Mon, Apr 24, 2017 at 08:55:33PM -0700, Andres Freund wrote:
>> I wonder if there's a reasonable way that allows to add links to the
>> more crucial commits for changelog entries.
I don't see any way you could do that that wouldn't be extremely
bulky and distracting. And, frankly, it would be of interest to
something well under one percent of the release-note audience.
We as hackers are not the target audience.
I think the reason for including the commit pointers in the SGML
comments was already so that interested and knowledgeable people
could look up the associated commits if they wanted. I don't see
a need to do more than that.
regards, tom lane
On Tue, Apr 25, 2017 at 01:10:35PM +0900, Amit Langote wrote: > On 2017/04/25 10:31, Bruce Momjian wrote: > > I have committed the first draft of the Postgres 10 release notes. They > > are current as of two days ago, and I will keep them current. Please > > give me any feedback you have. > > > > The only unusual thing is that this release has ~180 items while most > > recent release have had ~220. The pattern I see that there are more > > large features in this release than previous ones. > > Thanks for this. > > Wondering if the following really belongs under postgres_fdw improvements: > > +<listitem> > +<!-- > +Author: Robert Haas <rhaas@postgresql.org> > +2017-03-16 [b30fb56b0] postgres_fdw: Push down <literal>FULL JOIN</>s > with restriction clau > +--> > +<para> > +Improve optimization of <literal>FULL JOIN</> queries containing > subqueries in the > +<literal>FROM</> clause (Etsuro Fujita) > +</para> > +</listitem> > > Maybe, nearby the following: > > + <sect3> > + <title>Additional Modules</title> > ... > +<listitem> > +<!-- > +Author: Robert Haas <rhaas@postgresql.org> > +2016-10-21 [7012b132d] postgres_fdw: Push down aggregates to remote servers. > +--> > +<para> > +Push aggregates to foreign data wrapper servers, where possible (Jeevan > +Chalke, Ashutosh Bapat) > +</para> Yes, you are right. Text udpated and moved. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr Jelinek wrote: > On 25/04/17 03:31, Bruce Momjian wrote: > > I have committed the first draft of the Postgres 10 release notes. They > > are current as of two days ago, and I will keep them current. Please > > give me any feedback you have. > > > > Cool! > > > The only unusual thing is that this release has ~180 items while most > > recent release have had ~220. The pattern I see that there are more > > large features in this release than previous ones. > > > > Well I think for example > > > Optimizer > > > > Add the ability to compute a correlation ratio and the number of distinct values on several columns (Tomas Vondra,David Rowley) > > > > could be easily 2 or 3 items (explicitly defining additional statistics, > multicolumn ndistinct and functional dependencies). Yes, we have more wholistic improvements that are made of many complex parts, e.g. logical replication, partitioning. > I also wonder if ability to run SQL queries on walsender connected to a > database is worth mentioning (replication=database kind of connection). Uh, why would that be important to users? > Or the ability of logical decoding to follow timeline switches. I didn't think logical decoding was really more than a proof-of-concept until now. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 11:09:15AM +0530, Ashutosh Bapat wrote: > Commit f9b1a0dd403ec0931213c66d5f979a3d3e8e7e30 mentions "Ashutosh > Bapat" as author, which is not reflected in the release notes > > -- > Allow explicit control over EXPLAIN's display of planning and > execution time (Stephen Frost) > > By default planning and execution is display by EXPLAIN ANALYZE and > not display in other cases. The new EXPLAIN option SUMMARY allows > explicit control of this. Oh, I missed that 'Author' line, fixed. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 03:15:45PM +0200, Félix GERZAGUET wrote: > Hello Bruce, > > On Tue, Apr 25, 2017 at 3:31 AM, Bruce Momjian <bruce@momjian.us> wrote: > > I have committed the first draft of the Postgres 10 release notes. They > are current as of two days ago, and I will keep them current. Please > give me any feedback you have. > > > I just noticed: > > "Allow pg_buffercache to run without fewer locks (Ivan Kartyshov)" > > I think it should be > > "Allow pg_buffercache to run with fewer locks (Ivan Kartyshov)" Thanks, fixed. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 02:39:40PM +0900, Michael Paquier wrote: > > OK, fixed. I also moved some "incompatibility" items into the right > > section. FYI, you can see the most recent doc build here: > > > > http://momjian.us/pgsql_docs/release-10.html > > Here is some feedback based on a first read. > > +<para> > +Issue fsync on the output files generated by <application>pg_dump</> > and <application>pg_dumpall</>(Michael Paquier) > +</para> > Missing a space here between pg_dumpall and my name. > > <para> > This is accessed via <function>ts_headline()</> and to_tsvector. RIGHT SECTION? > </para> > </listitem> > This should use a <function> markup for both names. Fixed. > <para> > Add <link linkend="auth-pg-hba-conf"><literal>SCRAM-SHA-256</></> > support for password negotiation and storage (Michael > Paquier, Heikki Linnakangas) > </para> > <para> > This proves better security than the existing 'md5' negotiation and > storage method. > </para> > This is quite vague... Can you give me better text? I can't think of any. > <listitem> > <!-- > Author: Robert Haas <rhaas@postgresql.org> > 2016-11-21 [a734fd5d1] autovacuum: Drop orphan temp tables more quickly but wit > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2016-11-27 [dafa0848d] Code review for early drop of orphaned temp relations in > --> > <para> > Remove orphaned temporary tables more aggressively (Robert Haas, Tom Lane) > </para> > <para> > Previously such tables were removed only when necessary. SECTION? > </para> > </listitem> > Well, I wrote the first patch that has been committed, until Tom > showed up and rewrote everything :) > perhaps this could be in a section dedicated to autovacuum? Well, I need 3+ items to make a new section and I don't think we have that. > <listitem> > <!-- > Author: Robert Haas <rhaas@postgresql.org> > 2017-02-08 [a507b8690] Add WAL consistency checking facility. > Author: Robert Haas <rhaas@postgresql.org> > 2017-03-14 [bb4a39637] hash: Support WAL consistency checking. > --> > <para> > Add <acronym>GUC</> <xref linkend="guc-wal-consistency-checking"> to > add details to <acronym>WAL</> that can be > sanity-checked on the standby (Kuntal Ghosh, Michael Paquier, Robert > Haas) > </para> > I would be fine if my name is removed here. Kuntal has done all the > work, I just reviewed his code. OK, done. > <para> > Previously only specification of the stop name, time, and xid were > supported. > </para> > </listitem> > Missing timeline and 'immediate' here. Yes, added. > <listitem> > <!-- > Author: Simon Riggs <simon@2ndQuadrant.com> > 2017-04-04 [728bd991c] Speedup 2PC recovery by skipping two phase state files i > --> > <para> > Speed up two-phase commit recovery performance (Simon Riggs) > </para> > </listitem> > Simon is the committer here, authors are marked as Stas Kelvich, > Nikhil Sontakke and Michael Paquier. Oh, I made a mistake there, fixed. > <listitem> > <!-- > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2016-08-15 [ca9112a42] Stamp HEAD as 10devel. > --> > <para> > New major version numbering (Peter Eisentraut, Tom Lane) > </para> > <para> > Major versions will now increase just the first number, and minor > releases will increase just the second number. A third number will no > longer be used in Postgres version numbers. > </para> > </listitem> > Perhaps this should be higher in the list? OK, I moved it to the top of incompatibilities. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 11:20:46AM -0400, Bruce Momjian wrote: > Oh, I made a mistake there, fixed. > > > <listitem> > > <!-- > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > 2016-08-15 [ca9112a42] Stamp HEAD as 10devel. > > --> > > <para> > > New major version numbering (Peter Eisentraut, Tom Lane) > > </para> > > <para> > > Major versions will now increase just the first number, and minor > > releases will increase just the second number. A third number will no > > longer be used in Postgres version numbers. > > </para> > > </listitem> > > Perhaps this should be higher in the list? > > OK, I moved it to the top of incompatibilities. Sorry, I meant to the top of the "Source Code" section. It used to be the second item in that list. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 12:45 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Andres Freund <andres@anarazel.de> writes: >> On 2017-04-24 23:37:42 -0400, Bruce Momjian wrote: >>> I remember seeing those and those are normally details I do not put in >>> the release notes as there isn't a clear user experience change except >>> "Postgres is faster". Yeah, a bummer, and I can change my filter, but >>> it would require discussion. > >> I think "postgres is faster" is one of the bigger user demands, so I >> don't think that policy makes much sense. A large number of the changes >> over the next few releases will focus solely on that. Nor do I think >> past release notes particularly filtered such changes out. > > I think it has been pretty common to accumulate a lot of such changes > into generic entries like, say, "speedups for hash joins". More detail > than that simply isn't useful to end users; and as a rule, our release > notes are too long anyway. In that spirit, the truncation speedups it seems are missing: Might be summarized simply as: Vacuum truncation has been sped up for rotating media, sometimes considerably (up to five times in certain configurations). Full commit, for reference: commit 7e26e02eec90370dd222f35f00042f8188488ac4 Author: Alvaro Herrera <alvherre@alvh.no-ip.org> Date: Mon Jan 23 12:55:18 2017 -0300 Prefetch blocks during lazy vacuum's truncation scan Vacuum truncation scan can be sped up on rotating media by prefetching blocks in forward direction. That makes theblocks already present in memory by the time they are needed, while also letting OS read-ahead kick in. The truncate scan has been measured to be five times faster than without this patch (that was on a slow disk, but itshouldn't hurt on fast disks.) Author: Álvaro Herrera, loosely based on a submission by Claudio Freire Discussion: https://postgr.es/m/CAGTBQpa6NFGO_6g_y_7zQx8L9GcHDSQKYdo1tGuh791z6PYgEg@mail.gmail.com
On 2017-04-25 10:10:07 -0400, Bruce Momjian wrote: > On Mon, Apr 24, 2017 at 08:52:05PM -0700, Andres Freund wrote: > > On 2017-04-24 23:45:06 -0400, Tom Lane wrote: > > Oh, I completely agree with accumulating related changes, and that > > code-level details aren't useful. I think we skipped them entirely > > here. And I just listed my own changes because I could find them > > quickly, but they're not alone, e.g: > > 090010f2ec9b1f9ac1124dc628b89586f911b641 - Improve performance of find_tabstat_entry()/get_tabstat_entry() > > which makes it realistic to have sessions touching many relations, which > > previously was O(#relations^2), and which caused repeated complaints > > over the years, and allows for different usecases. > > Looking at this commit it appears to improve pg_stat statistics handling. > I don't see how that improves performance except to improve statistics > aggregation, which happens in the statistics process. Previously when creating a new relation, we had to walk a list of all open relations (i.e. O(N) work for each relation), now it's O(1) for each relation. And that happens in the backends, not in the statistics collector. It's pretty easy to see the effect. Write a plpgsql function that creates, say, 100k tables. Run it in 9.6 and v10. CREATE OR REPLACE FUNCTION create_tables(p_ntables int) RETURNS void LANGUAGE plpgsql AS $$DECLARE i int;BEGIN FOR i IN 1.. p_ntables LOOP EXECUTE 'CREATE TABLE tbl_'||i::text||'();';END LOOP;END;$$ Measuring the time for the SELECT in BEGIN;SELECT create_tables(10000);ROLLBACK; Recreating the server inbetween each run I get: version #nrels time 9.6 10 7ms 9.6 100 23ms 9.6 1000 159ms 9.6 10000 1465ms 9.6 100000 32367ms 9.6 200000 144026ms at this point, you can see, we've squarely left O(N) country, and entered the vast O(N^2) waste. 10 10 9ms 10 100 22ms 10 1000 162ms 10 10000 1497ms 10 100000 17260ms 10 200000 39275ms Here we roughly stay in O(N). (there's some other suboptimal behaviour in smgrclose(); but that's for another day) - Andres
Moin, On Mon, April 24, 2017 9:31 pm, Bruce Momjian wrote: > I have committed the first draft of the Postgres 10 release notes. They > are current as of two days ago, and I will keep them current. Please > give me any feedback you have. Thank you! Here is one thing I noticed: "By default planning and execution is display by EXPLAIN ANALYZE and not display in other cases. The new EXPLAIN option SUMMARY allows explicit control of this." The grammar sounds a bit off, I'd suggest: "By default planning and execution time are both displayed by EXPLAIN ANALYZE and not displayed in other cases. The new EXPLAIN option SUMMARY allows explicit control of this feature." Regards, Tels
On Tue, Apr 25, 2017 at 01:37:13PM -0300, Claudio Freire wrote: > > I think it has been pretty common to accumulate a lot of such changes > > into generic entries like, say, "speedups for hash joins". More detail > > than that simply isn't useful to end users; and as a rule, our release > > notes are too long anyway. > > In that spirit, the truncation speedups it seems are missing: > > Might be summarized simply as: > > Vacuum truncation has been sped up for rotating media, sometimes > considerably (up to five times in certain configurations). > > Full commit, for reference: > > commit 7e26e02eec90370dd222f35f00042f8188488ac4 > Author: Alvaro Herrera <alvherre@alvh.no-ip.org> > Date: Mon Jan 23 12:55:18 2017 -0300 > > Prefetch blocks during lazy vacuum's truncation scan > > Vacuum truncation scan can be sped up on rotating media by prefetching > blocks in forward direction. That makes the blocks already present in > memory by the time they are needed, while also letting OS read-ahead > kick in. > > The truncate scan has been measured to be five times faster than without > this patch (that was on a slow disk, but it shouldn't hurt on fast > disks.) > > Author: Álvaro Herrera, loosely based on a submission by Claudio Freire > Discussion: > https://postgr.es/m/CAGTBQpa6NFGO_6g_y_7zQx8L9GcHDSQKYdo1tGuh791z6PYgEg@mail.gmail.com I don't think this warrants inclusion in the release notes for reasons already discussed. The vacuum truncation operation is a rare one and an implementation detail. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 01:06:05PM -0400, Tels wrote:
> Moin,
>
> On Mon, April 24, 2017 9:31 pm, Bruce Momjian wrote:
> > I have committed the first draft of the Postgres 10 release notes. They
> > are current as of two days ago, and I will keep them current. Please
> > give me any feedback you have.
>
> Thank you! Here is one thing I noticed:
>
> "By default planning and execution is display by EXPLAIN ANALYZE and not
> display in other cases. The new EXPLAIN option SUMMARY allows explicit
> control of this."
>
> The grammar sounds a bit off, I'd suggest:
>
> "By default planning and execution time are both displayed by EXPLAIN
> ANALYZE and not displayed in other cases. The new EXPLAIN option SUMMARY
> allows explicit control of this feature."
I think all that was missing was "time":
By default planning and execution time is display by <command>EXPLAIN ANALYZE</> and not display in other
cases. The new <command>EXPLAIN</> option <literal>SUMMARY</> allows explicit control of this.
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 10:00:52AM -0700, Andres Freund wrote:
> at this point, you can see, we've squarely left O(N) country, and
> entered the vast O(N^2) waste.
>
> 10 10 9ms
> 10 100 22ms
> 10 1000 162ms
> 10 10000 1497ms
> 10 100000 17260ms
> 10 200000 39275ms
>
> Here we roughly stay in O(N).
>
> (there's some other suboptimal behaviour in smgrclose(); but that's for
> another day)
OK, I got it now. :-) Here is the new item:
Improve table creation speed in sessions that reference many relations (Aleksander Alekseev)
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 2:11 PM, Bruce Momjian <bruce@momjian.us> wrote: > On Tue, Apr 25, 2017 at 01:37:13PM -0300, Claudio Freire wrote: >> > I think it has been pretty common to accumulate a lot of such changes >> > into generic entries like, say, "speedups for hash joins". More detail >> > than that simply isn't useful to end users; and as a rule, our release >> > notes are too long anyway. >> >> In that spirit, the truncation speedups it seems are missing: >> >> Might be summarized simply as: >> >> Vacuum truncation has been sped up for rotating media, sometimes >> considerably (up to five times in certain configurations). >> >> Full commit, for reference: >> >> commit 7e26e02eec90370dd222f35f00042f8188488ac4 >> Author: Alvaro Herrera <alvherre@alvh.no-ip.org> >> Date: Mon Jan 23 12:55:18 2017 -0300 >> >> Prefetch blocks during lazy vacuum's truncation scan >> >> Vacuum truncation scan can be sped up on rotating media by prefetching >> blocks in forward direction. That makes the blocks already present in >> memory by the time they are needed, while also letting OS read-ahead >> kick in. >> >> The truncate scan has been measured to be five times faster than without >> this patch (that was on a slow disk, but it shouldn't hurt on fast >> disks.) >> >> Author: Álvaro Herrera, loosely based on a submission by Claudio Freire >> Discussion: >> https://postgr.es/m/CAGTBQpa6NFGO_6g_y_7zQx8L9GcHDSQKYdo1tGuh791z6PYgEg@mail.gmail.com > > I don't think this warrants inclusion in the release notes for reasons > already discussed. The vacuum truncation operation is a rare one and > an implementation detail. \_(0_0)_/ As you wish. Though if I wasn't already aware of it, I would like to know, because it's been a source of trouble in the past.
On 2017-04-25 13:11:32 -0400, Bruce Momjian wrote: > On Tue, Apr 25, 2017 at 01:37:13PM -0300, Claudio Freire wrote: > > The truncate scan has been measured to be five times faster than without > > this patch (that was on a slow disk, but it shouldn't hurt on fast > > disks.) > > > > Author: Álvaro Herrera, loosely based on a submission by Claudio Freire > > Discussion: > > https://postgr.es/m/CAGTBQpa6NFGO_6g_y_7zQx8L9GcHDSQKYdo1tGuh791z6PYgEg@mail.gmail.com > > I don't think this warrants inclusion in the release notes for reasons > already discussed. The vacuum truncation operation is a rare one and > an implementation detail. I think that's backwards. The truncate operation is quite delicate because it happens with AccessExclusiveLock held. This regularly does cause issues in production. When users look for things they possibly should update for, something like "performance improvents in final vacuum phase" + oneliner is going to be a lot more interesting than, say, "Add MONEY operators for multiplication and division with INT8 values". More and more users are going to be primarily interested in three classes of new things in postgres: 1) performance 2) replication 3) easier management. Arbitrarily excluding one of the major categories from release notes isn't a useful policy, especially if we continue to list new feature that'll effect no more than a handful of people. - Andres
On Tue, Apr 25, 2017 at 02:31:50PM -0300, Claudio Freire wrote: > >> Author: Álvaro Herrera, loosely based on a submission by Claudio Freire > >> Discussion: > >> https://postgr.es/m/CAGTBQpa6NFGO_6g_y_7zQx8L9GcHDSQKYdo1tGuh791z6PYgEg@mail.gmail.com > > > > I don't think this warrants inclusion in the release notes for reasons > > already discussed. The vacuum truncation operation is a rare one and > > an implementation detail. > > \_(0_0)_/ > > As you wish. > > Though if I wasn't already aware of it, I would like to know, because > it's been a source of trouble in the past. Understood, but the question is whether the release notes are the right place to educate users of something that will no longer be a problem. I am happy to adjust things to whatever the community wants, but, on the other hand I have a responsibility to be consistent what what they have wanted in the past. I am also open to reviewing how we filter things compared other projects. On a larger note, not being in the release notes doesn't mean it isn't important, but rather, that is isn't a change users need to know about. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Bruce Momjian <bruce@momjian.us> writes:
> On Tue, Apr 25, 2017 at 10:00:52AM -0700, Andres Freund wrote:
>> at this point, you can see, we've squarely left O(N) country, and
>> entered the vast O(N^2) waste.
> OK, I got it now. :-) Here is the new item:
> Improve table creation speed in sessions that reference many
> relations (Aleksander Alekseev)
That's not specifically about table creation. How about
Reduce statistics tracking overhead in sessions that referencemany thousands of relations (Aleksander Alekseev)
I think it is fair, based on Andres' example, to quantify this as not
being of much interest till you get to thousands of relations. But
it would matter for any action that creates a statistics report,
whether it be scans, inserts, updates, deletes ...
regards, tom lane
On Tue, Apr 25, 2017 at 10:37:48AM -0700, Andres Freund wrote:
> On 2017-04-25 13:11:32 -0400, Bruce Momjian wrote:
> > I don't think this warrants inclusion in the release notes for reasons
> > already discussed. The vacuum truncation operation is a rare one and
> > an implementation detail.
>
> I think that's backwards. The truncate operation is quite delicate
> because it happens with AccessExclusiveLock held. This regularly does
> cause issues in production. When users look for things they possibly
> should update for, something like "performance improvents in final
> vacuum phase" + oneliner is going to be a lot more interesting than,
> say, "Add MONEY operators for multiplication and division with INT8
> values".
>
> More and more users are going to be primarily interested in three
> classes of new things in postgres: 1) performance 2) replication 3)
> easier management. Arbitrarily excluding one of the major categories
> from release notes isn't a useful policy, especially if we continue to
> list new feature that'll effect no more than a handful of people.
I disputed that my logic is arbitrary. However, given your
explanation, I have added the item:
Improve speed of <command>VACUUM</>'s removal of trailing empty heap pages (Alvaro Herrera)
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 01:44:09PM -0400, Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > On Tue, Apr 25, 2017 at 10:00:52AM -0700, Andres Freund wrote: > >> at this point, you can see, we've squarely left O(N) country, and > >> entered the vast O(N^2) waste. > > > OK, I got it now. :-) Here is the new item: > > > Improve table creation speed in sessions that reference many > > relations (Aleksander Alekseev) > > That's not specifically about table creation. How about > > Reduce statistics tracking overhead in sessions that reference > many thousands of relations (Aleksander Alekseev) > > I think it is fair, based on Andres' example, to quantify this as not > being of much interest till you get to thousands of relations. But > it would matter for any action that creates a statistics report, > whether it be scans, inserts, updates, deletes ... OK, text updated. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Hi, On 2017-04-25 13:39:07 -0400, Bruce Momjian wrote: > Understood, but the question is whether the release notes are the right > place to educate users of something that will no longer be a problem. I think it's the *prime* place for it. It obviously doesn't matter if you're not affected by $performance_problem, but if you are affected, then it's quite likely going to be important. Deciding whether to migrate to a new version will often be a decision about a *lot* of work, so it'll not be made lightly, and without a motivator. If we go by that logic, why are we listing parallelism? Why are we listing new planner/executor features that lead to faster plans? The reason we do, is that they're addressing concerns that users had, which they need to know about. > I am happy to adjust things to whatever the community wants, but, on the > other hand I have a responsibility to be consistent what what they have > wanted in the past. Why? Greetings, Andres Freund
On 25/04/17 17:01, Bruce Momjian wrote: > >> I also wonder if ability to run SQL queries on walsender connected to a >> database is worth mentioning (replication=database kind of connection). > > Uh, why would that be important to users? Because every tool that uses logical decoding to capture changes does no need to open extra connection to fetch initial state. (and we document much less user visible changes than this in the release notes) > >> Or the ability of logical decoding to follow timeline switches. > > I didn't think logical decoding was really more than a proof-of-concept > until now. > Huh? -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Tue, Apr 25, 2017 at 2:45 PM, Bruce Momjian <bruce@momjian.us> wrote: > On Tue, Apr 25, 2017 at 10:37:48AM -0700, Andres Freund wrote: >> On 2017-04-25 13:11:32 -0400, Bruce Momjian wrote: >> > I don't think this warrants inclusion in the release notes for reasons >> > already discussed. The vacuum truncation operation is a rare one and >> > an implementation detail. >> >> I think that's backwards. The truncate operation is quite delicate >> because it happens with AccessExclusiveLock held. This regularly does >> cause issues in production. When users look for things they possibly >> should update for, something like "performance improvents in final >> vacuum phase" + oneliner is going to be a lot more interesting than, >> say, "Add MONEY operators for multiplication and division with INT8 >> values". >> >> More and more users are going to be primarily interested in three >> classes of new things in postgres: 1) performance 2) replication 3) >> easier management. Arbitrarily excluding one of the major categories >> from release notes isn't a useful policy, especially if we continue to >> list new feature that'll effect no more than a handful of people. > > I disputed that my logic is arbitrary. However, given your > explanation, I have added the item: > > Improve speed of <command>VACUUM</>'s removal of trailing empty > heap pages (Alvaro Herrera) That's enough for me, thanks.
On Tue, April 25, 2017 1:21 pm, Bruce Momjian wrote: > On Tue, Apr 25, 2017 at 01:06:05PM -0400, Tels wrote: >> Moin, >> >> On Mon, April 24, 2017 9:31 pm, Bruce Momjian wrote: >> > I have committed the first draft of the Postgres 10 release notes. >> They >> > are current as of two days ago, and I will keep them current. Please >> > give me any feedback you have. >> >> Thank you! Here is one thing I noticed: >> >> "By default planning and execution is display by EXPLAIN ANALYZE and not >> display in other cases. The new EXPLAIN option SUMMARY allows explicit >> control of this." >> >> The grammar sounds a bit off, I'd suggest: >> >> "By default planning and execution time are both displayed by EXPLAIN >> ANALYZE and not displayed in other cases. The new EXPLAIN option SUMMARY >> allows explicit control of this feature." > > I think all that was missing was "time": > > By default planning and execution time is display by > <command>EXPLAIN ANALYZE</> and not display in other cases. > The new <command>EXPLAIN</> option <literal>SUMMARY</> allows > explicit control of this. Hm, no, I disagree, the the grammar is not right: It should be: "X and Y _are_ display_ed_" instead of: "X and Y is display" likewise "and not display_ed_ in other cases." Best wishes, Tels
On Tue, Apr 25, 2017 at 08:12:05PM +0200, Petr Jelinek wrote: > On 25/04/17 17:01, Bruce Momjian wrote: > > > >> I also wonder if ability to run SQL queries on walsender connected to a > >> database is worth mentioning (replication=database kind of connection). > > > > Uh, why would that be important to users? > > Because every tool that uses logical decoding to capture changes does not > need to open extra connection to fetch initial state. > > (and we document much less user visible changes than this in the release > notes) Are you saying that 3rd party applications that use logical WAL records need to know about that? Can you provide some text? > >> Or the ability of logical decoding to follow timeline switches. > > > > I didn't think logical decoding was really more than a proof-of-concept > > until now. > > > > Huh? Uh, the only logical decoding code that I know we ship pre-PG 10 is contrib/test_decoding/. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 03:17:29PM -0400, Tels wrote:
> > I think all that was missing was "time":
> >
> > By default planning and execution time is display by
> > <command>EXPLAIN ANALYZE</> and not display in other cases.
> > The new <command>EXPLAIN</> option <literal>SUMMARY</> allows
> > explicit control of this.
>
> Hm, no, I disagree, the the grammar is not right:
>
> It should be: "X and Y _are_ display_ed_"
>
> instead of:
>
> "X and Y is display"
>
> likewise "and not display_ed_ in other cases."
OK, got it:
By default planning and execution time are display by <command>EXPLAIN ANALYZE</> and are not display in
othercases. The new <command>EXPLAIN</> option <literal>SUMMARY</> allows explicit control of this.
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
On Wed, Apr 26, 2017 at 12:20 AM, Bruce Momjian <bruce@momjian.us> wrote: > On Tue, Apr 25, 2017 at 02:39:40PM +0900, Michael Paquier wrote: >> <para> >> Add <link linkend="auth-pg-hba-conf"><literal>SCRAM-SHA-256</></> >> support for password negotiation and storage (Michael >> Paquier, Heikki Linnakangas) >> </para> >> <para> >> This proves better security than the existing 'md5' negotiation and >> storage method. >> </para> >> This is quite vague... > > Can you give me better text? I can't think of any. Sure, here is an idea: Add support for SASL authentication using protocol mechanism SCRAM-SHA-256 per RFC 5802 and 7677. (adding a reference to the RFCs with a link seems important to me). SCRAM-SHA-256 improves deficiencies of MD5 password hashing by preventing any kind of pass-the-hash vulnerabilities, where a user would be able to connect to a PostgreSQL instance by just knowing the hash of a password and not the password itself. -- Michael
Hello, Bruce I forgot to point out one thing. Allow libpq to connect to multiple specified host names (Robert Haas) libpq will connect with the first responsive host name. According to the following CF entry and my memory, https://commitfest.postgresql.org/12/879/ Authors mithun cy (mithun.cy) Reviewers Gerdan Santos (gerdan), Takayuki Tsunakawa (maumau)Remove from reviewers Committer Robert Haas (rhaas) The author should probably be "(Mithun Cy, Robert Haas, Victor Wagner)" Regards Takayuki Tsunakawa
On Wed, Apr 26, 2017 at 12:26:33AM +0000, Tsunakawa, Takayuki wrote: > Hello, Bruce > > I forgot to point out one thing. > > Allow libpq to connect to multiple specified host names (Robert Haas) > libpq will connect with the first responsive host name. > > According to the following CF entry and my memory, > > https://commitfest.postgresql.org/12/879/ > > Authors > mithun cy (mithun.cy) > > Reviewers > Gerdan Santos (gerdan), Takayuki Tsunakawa (maumau)Remove from reviewers > > Committer > Robert Haas (rhaas) > > The author should probably be "(Mithun Cy, Robert Haas, Victor Wagner)" I based the release note text on this commit: commit 274bb2b3857cc987cfa21d14775cae9b0dababa5Author: Robert Haas <rhaas@postgresql.org>Date: Thu Nov 3 09:25:20 2016-0400 libpq: Allow connection strings and URIs to specify multiple hosts. It's also possible to specify a separateport for each host. Previously, we'd loop over every address returned by looking up the host name; now, we'lltry every address for every host name. Patch by me. Victor Wagner wrote an earlier patch for this feature, whichI read, but I didn't use any of his code. Review by Mithun Cy. Was it wrong? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr Jelinek wrote: > I also wonder if ability to run SQL queries on walsender connected to a > database is worth mentioning (replication=database kind of connection). > > Or the ability of logical decoding to follow timeline switches. When you say "logical decoding", you don't mean contrib/test_decoding? Do you mean the WAL logical stream now has timeline information and 3rd party software should know about that? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
From: 'Bruce Momjian' [mailto:bruce@momjian.us] > > I forgot to point out one thing. > > > > Allow libpq to connect to multiple specified host names (Robert Haas) > > libpq will connect with the first responsive host name. > > > > According to the following CF entry and my memory, > > > > https://commitfest.postgresql.org/12/879/ > > > > Authors > > mithun cy (mithun.cy) > > > > Reviewers > > Gerdan Santos (gerdan), Takayuki Tsunakawa (maumau)Remove from > > reviewers > > > > Committer > > Robert Haas (rhaas) > > > > The author should probably be "(Mithun Cy, Robert Haas, Victor Wagner)" > > I based the release note text on this commit: > > commit 274bb2b3857cc987cfa21d14775cae9b0dababa5 > Author: Robert Haas <rhaas@postgresql.org> > Date: Thu Nov 3 09:25:20 2016 -0400 > > libpq: Allow connection strings and URIs to specify multiple > hosts. > > It's also possible to specify a separate port for each host. > > Previously, we'd loop over every address returned by looking > up the > host name; now, we'll try every address for every host name. > > Patch by me. Victor Wagner wrote an earlier patch for this > feature, > which I read, but I didn't use any of his code. Review by Mithun > Cy. > > Was it wrong? Oh, that commit. I got it. Regards Takayuki Tsunakawa
On 2017-04-25 21:19:41 -0400, Bruce Momjian wrote:
> On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr Jelinek wrote:
> > Or the ability of logical decoding to follow timeline switches.
>
> When you say "logical decoding", you don't mean contrib/test_decoding?
No. test_decoding is just an example output plugin ("formatting" the
changes), but there's in-production users of logical decoding out
there. So features adding to logical decoding in general, are worth to
be mention in general.
> Do you mean the WAL logical stream now has timeline information and 3rd
> party software should know about that?
It's less about knowing about it, and more about being empowering new usecases.
But, TBH, in this case I'm not sure what the point would be - given
there's no way to sanely create and use such slots, I don't think the
timeline following has advantages on its own, it's imo more of a
stepping stone.
Greetings,
Andres Freund
On Wed, Apr 26, 2017 at 09:02:51AM +0900, Michael Paquier wrote: > On Wed, Apr 26, 2017 at 12:20 AM, Bruce Momjian <bruce@momjian.us> wrote: > > On Tue, Apr 25, 2017 at 02:39:40PM +0900, Michael Paquier wrote: > >> <para> > >> Add <link linkend="auth-pg-hba-conf"><literal>SCRAM-SHA-256</></> > >> support for password negotiation and storage (Michael > >> Paquier, Heikki Linnakangas) > >> </para> > >> <para> > >> This proves better security than the existing 'md5' negotiation and > >> storage method. > >> </para> > >> This is quite vague... > > > > Can you give me better text? I can't think of any. > > Sure, here is an idea: > Add support for SASL authentication using protocol mechanism > SCRAM-SHA-256 per RFC 5802 and 7677. (adding a reference to the RFCs > with a link seems important to me). > > SCRAM-SHA-256 improves deficiencies of MD5 password hashing by > preventing any kind of pass-the-hash vulnerabilities, where a user > would be able to connect to a PostgreSQL instance by just knowing the > hash of a password and not the password itself. First, I don't think RFC references belong in the release notes, let alone RFC links. Second, there seems to be some confusion over what SCRAM-SHA-256 gives us over MD5. I think there are a few benefits: o packets cannot be replayed as easily, i.e. md5 replayed random salt packets with a 50% probability after 16k sessions o hard to re-use SCRAM-SHA-256 string if disclosed vs. simple for md5 o harder to brute-force trying all possible strings to find a matching hash So if you tell users that SCRAM-SHA-256 is better than MD5 only because of one of those, they will not realize that three benefits of changing to SCRAM-SHA-256. I might have even missed some benefits. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Wed, Apr 26, 2017 at 10:56 AM, Bruce Momjian <bruce@momjian.us> wrote: > First, I don't think RFC references belong in the release notes, let > alone RFC links. > > Second, there seems to be some confusion over what SCRAM-SHA-256 gives > us over MD5. I think there are a few benefits: > > o packets cannot be replayed as easily, i.e. md5 replayed random salt > packets with a 50% probability after 16k sessions > o hard to re-use SCRAM-SHA-256 string if disclosed vs. simple for md5 > o harder to brute-force trying all possible strings to find a matching > hash > > So if you tell users that SCRAM-SHA-256 is better than MD5 only because > of one of those, they will not realize that three benefits of changing > to SCRAM-SHA-256. I might have even missed some benefits. If the release notes keep a general tone, perhaps it would be better to mention as well that SCRAM is the recommended password-based authentication method moving forward? -- Michael
On Tue, Apr 25, 2017 at 06:40:08PM -0700, Andres Freund wrote:
> On 2017-04-25 21:19:41 -0400, Bruce Momjian wrote:
> > On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr Jelinek wrote:
> > > Or the ability of logical decoding to follow timeline switches.
> >
> > When you say "logical decoding", you don't mean contrib/test_decoding?
>
> No. test_decoding is just an example output plugin ("formatting" the
> changes), but there's in-production users of logical decoding out
> there. So features adding to logical decoding in general, are worth to
> be mention in general.
Yes, but I am still trying to find out what changed because we don't
ship much that does "decoding" except contrib/test_decoding. Are you
saying logical information is included in WAL?
> > Do you mean the WAL logical stream now has timeline information and 3rd
> > party software should know about that?
>
> It's less about knowing about it, and more about being empowering new usecases.
>
> But, TBH, in this case I'm not sure what the point would be - given
> there's no way to sanely create and use such slots, I don't think the
> timeline following has advantages on its own, it's imo more of a
> stepping stone.
OK, makes sense. Let me know if we should add:
> >> I also wonder if ability to run SQL queries on walsender connected to a
> >> database is worth mentioning (replication=database kind of connection).
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
On Wed, Apr 26, 2017 at 11:06:03AM +0900, Michael Paquier wrote: > On Wed, Apr 26, 2017 at 10:56 AM, Bruce Momjian <bruce@momjian.us> wrote: > > First, I don't think RFC references belong in the release notes, let > > alone RFC links. > > > > Second, there seems to be some confusion over what SCRAM-SHA-256 gives > > us over MD5. I think there are a few benefits: > > > > o packets cannot be replayed as easily, i.e. md5 replayed random salt > > packets with a 50% probability after 16k sessions > > o hard to re-use SCRAM-SHA-256 string if disclosed vs. simple for md5 > > o harder to brute-force trying all possible strings to find a matching > > hash > > > > So if you tell users that SCRAM-SHA-256 is better than MD5 only because > > of one of those, they will not realize that three benefits of changing > > to SCRAM-SHA-256. I might have even missed some benefits. > > If the release notes keep a general tone, perhaps it would be better > to mention as well that SCRAM is the recommended password-based > authentication method moving forward? Well, we could add "MD5 users are encouraged to switch to SCRAM-SHA-256". Now whether we want to list this as something on the SCRAM-SHA-256 description, or mention it as an incompatibility, or under Migration. I am not clear that MD5 is in such terrible shape that this is warranted. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On 2017-04-25 22:13:00 -0400, Bruce Momjian wrote:
> On Tue, Apr 25, 2017 at 06:40:08PM -0700, Andres Freund wrote:
> > On 2017-04-25 21:19:41 -0400, Bruce Momjian wrote:
> > > On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr Jelinek wrote:
> > > > Or the ability of logical decoding to follow timeline switches.
> > >
> > > When you say "logical decoding", you don't mean contrib/test_decoding?
> >
> > No. test_decoding is just an example output plugin ("formatting" the
> > changes), but there's in-production users of logical decoding out
> > there. So features adding to logical decoding in general, are worth to
> > be mention in general.
>
> Yes, but I am still trying to find out what changed because we don't
> ship much that does "decoding" except contrib/test_decoding. Are you
> saying logical information is included in WAL?
We ship an extension API [1] [2] that allows to use decoding. And that
the internals of that API now knows how to deal with timeline changes.
You could e.g. compare it with the FDW API now providing the ability to
do aggregate pushdown or such.
> > >> I also wonder if ability to run SQL queries on walsender connected to a
> > >> database is worth mentioning (replication=database kind of connection).
Yes, but we're still discussing the feature's exact look nearby.
Greetings,
Andres Freund
[1] https://www.postgresql.org/docs/devel/static/logicaldecoding.html
[2] https://www.postgresql.org/docs/devel/static/logicaldecoding-output-plugin.html
On Tue, Apr 25, 2017 at 07:17:20PM -0700, Andres Freund wrote:
> On 2017-04-25 22:13:00 -0400, Bruce Momjian wrote:
> > On Tue, Apr 25, 2017 at 06:40:08PM -0700, Andres Freund wrote:
> > > On 2017-04-25 21:19:41 -0400, Bruce Momjian wrote:
> > > > On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr Jelinek wrote:
> > > > > Or the ability of logical decoding to follow timeline switches.
> > > >
> > > > When you say "logical decoding", you don't mean contrib/test_decoding?
> > >
> > > No. test_decoding is just an example output plugin ("formatting" the
> > > changes), but there's in-production users of logical decoding out
> > > there. So features adding to logical decoding in general, are worth to
> > > be mention in general.
> >
> > Yes, but I am still trying to find out what changed because we don't
> > ship much that does "decoding" except contrib/test_decoding. Are you
> > saying logical information is included in WAL?
>
> We ship an extension API [1] [2] that allows to use decoding. And that
> the internals of that API now knows how to deal with timeline changes.
> You could e.g. compare it with the FDW API now providing the ability to
> do aggregate pushdown or such.
Yes, if we changed the developer-visible API we should mention that in
the release notes.
> > > >> I also wonder if ability to run SQL queries on walsender connected to a
> > > >> database is worth mentioning (replication=database kind of connection).
>
> Yes, but we're still discussing the feature's exact look nearby.
OK, please keep us up-to-date on what needs to be added.
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 7:45 PM, Bruce Momjian <bruce@momjian.us> wrote: > On Tue, Apr 25, 2017 at 09:38:55AM +0530, Rafia Sabih wrote: >> On Tue, Apr 25, 2017 at 9:15 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> > Andres Freund <andres@anarazel.de> writes: >> >> On 2017-04-24 23:37:42 -0400, Bruce Momjian wrote: >> >>> I remember seeing those and those are normally details I do not put in >> >>> the release notes as there isn't a clear user experience change except >> >>> "Postgres is faster". Yeah, a bummer, and I can change my filter, but >> >>> it would require discussion. >> > >> >> I think "postgres is faster" is one of the bigger user demands, so I >> >> don't think that policy makes much sense. A large number of the changes >> >> over the next few releases will focus solely on that. Nor do I think >> >> past release notes particularly filtered such changes out. >> > >> > I think it has been pretty common to accumulate a lot of such changes >> > into generic entries like, say, "speedups for hash joins". More detail >> > than that simply isn't useful to end users; and as a rule, our release >> > notes are too long anyway. >> > >> > regards, tom lane >> > >> > >> Just wondering if the mention of commit >> 0414b26bac09379a4cbf1fbd847d1cee2293c5e4 is missed? Not sure if this >> requires a separate entry or could be merged with -- Support parallel >> btree index scans. > > This item was merged into the general parallel index scan item: > Okay, but then shouldn't we add Rafia's name as well to that release notes entry as she is the author of one of the merged item (commit 0414b26bac09379a4cbf1fbd847d1cee2293c5e4) which has an independent value. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
On Tue, Apr 25, 2017 at 09:56:58PM -0400, Bruce Momjian wrote: > > SCRAM-SHA-256 improves deficiencies of MD5 password hashing by > > preventing any kind of pass-the-hash vulnerabilities, where a user > > would be able to connect to a PostgreSQL instance by just knowing the > > hash of a password and not the password itself. > > First, I don't think RFC references belong in the release notes, let > alone RFC links. > > Second, there seems to be some confusion over what SCRAM-SHA-256 gives > us over MD5. I think there are a few benefits: > > o packets cannot be replayed as easily, i.e. md5 replayed random salt > packets with a 50% probability after 16k sessions ^^^ Sorry, it is after 64k sessions. The rule of thumb is that if you choose from 2^x random values, you have a 50% chance of repeating one after 2^(x/2) numbers. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Wed, Apr 26, 2017 at 09:34:02AM +0530, Amit Kapila wrote: > >> Just wondering if the mention of commit > >> 0414b26bac09379a4cbf1fbd847d1cee2293c5e4 is missed? Not sure if this > >> requires a separate entry or could be merged with -- Support parallel > >> btree index scans. > > > > This item was merged into the general parallel index scan item: > > > > Okay, but then shouldn't we add Rafia's name as well to that release > notes entry as she is the author of one of the merged item (commit > 0414b26bac09379a4cbf1fbd847d1cee2293c5e4) which has an independent > value. Oh, yes, added. I think in a quick look I saw Rafia Sabih as the same as Rahila Syed and thought the name was already listed. Sorry to both of them. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tue, Apr 25, 2017 at 7:19 PM, Bruce Momjian <bruce@momjian.us> wrote:
> On Tue, Apr 25, 2017 at 09:00:45AM +0530, Amit Kapila wrote:
>> On Tue, Apr 25, 2017 at 8:35 AM, Bruce Momjian <bruce@momjian.us> wrote:
>> > On Tue, Apr 25, 2017 at 08:30:50AM +0530, Amit Kapila wrote:
>> >> On Tue, Apr 25, 2017 at 7:01 AM, Bruce Momjian <bruce@momjian.us> wrote:
>> >> > I have committed the first draft of the Postgres 10 release notes. They
>> >> > are current as of two days ago, and I will keep them current. Please
>> >> > give me any feedback you have.
>> >> >
>> >>
>> >> Some of the items which I feel could be added:
>> >>
>> >> 5e6d8d2bbbcace304450b309e79366c0da4063e4
>> >> Allow parallel workers to execute subplans.
>> >
>> > Uh, can you show me the commit on that and give some text ideas?
>> >
>>
>> I have already mentioned the commit id (5e6d8d2b). Text can be "Allow
>> queries containing subplans to execute in parallel". We should also
>> mention in some way that this applies only when the query contains
>> uncorrelated subplan.
>
> Sorry but I don't know what that means, and if I don't know, others
> might not either.
>
Let me try to explain by example:
Without this feature, the queries that refer subplans will be executed
serially like below:
regression=# explain (costs off) select count(*) from tenk1 where
(two, four) not in (select hundred, thousand from tenk2 where thousand
> 100); QUERY PLAN
------------------------------------------Aggregate -> Seq Scan on tenk1 Filter: (NOT (hashed SubPlan 1))
SubPlan 1 -> Seq Scan on tenk2 Filter: (thousand > 100)
(6 rows)
After this feature, the queries that refer subplans can use parallel
plans like below:
regression=# explain (costs off) select count(*) from tenk1 where
(two, four) not in (select hundred, thousand from tenk2 where
thousand > 100); QUERY PLAN
------------------------------------------------------Finalize Aggregate -> Gather Workers Planned: 2
-> Partial Aggregate -> Parallel Seq Scan on tenk1 Filter: (NOT (hashed SubPlan 1))
SubPlan 1 -> Seq Scan on tenk2 Filter: (thousand >
100)
(9 rows)
Now, it won't use parallelism if there is correlated subplan like below:
Seq Scan on t1 Filter: (SubPlan 1) SubPlan 1 -> Result One-Time Filter: (t1.k = 0)
-> Parallel Seq Scan on t2
In this plan difference is that SubPlan refers to outer relation t1.
Do the above examples helps in understanding the feature?
--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
On Wed, Apr 26, 2017 at 07:38:05PM +0530, Amit Kapila wrote: > >> I have already mentioned the commit id (5e6d8d2b). Text can be "Allow > >> queries containing subplans to execute in parallel". We should also > >> mention in some way that this applies only when the query contains > >> uncorrelated subplan. > > > > Sorry but I don't know what that means, and if I don't know, others > > might not either. > > > > regression=# explain (costs off) select count(*) from tenk1 where > (two, four) not in (select hundred, thousand from tenk2 where > thousand > 100); > QUERY PLAN > ------------------------------------------------------ > Finalize Aggregate > -> Gather > Workers Planned: 2 > -> Partial Aggregate > -> Parallel Seq Scan on tenk1 > Filter: (NOT (hashed SubPlan 1)) > SubPlan 1 > -> Seq Scan on tenk2 > Filter: (thousand > 100) > (9 rows) > Oh, so non-correlated subqueries can be run in parallel. Yes, that is something we should have in the release notes. How is this? Author: Robert Haas <rhaas@postgresql.org>2017-02-14 [5e6d8d2bb] Allow parallel workers to execute subplans.Allow non-correlatedsubqueries to be run in parallel (Amit Kapila) -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Wed, Apr 26, 2017 at 8:46 PM, Bruce Momjian <bruce@momjian.us> wrote: > On Wed, Apr 26, 2017 at 07:38:05PM +0530, Amit Kapila wrote: >> >> I have already mentioned the commit id (5e6d8d2b). Text can be "Allow >> >> queries containing subplans to execute in parallel". We should also >> >> mention in some way that this applies only when the query contains >> >> uncorrelated subplan. >> > >> > Sorry but I don't know what that means, and if I don't know, others >> > might not either. >> > >> >> regression=# explain (costs off) select count(*) from tenk1 where >> (two, four) not in (select hundred, thousand from tenk2 where >> thousand > 100); >> QUERY PLAN >> ------------------------------------------------------ >> Finalize Aggregate >> -> Gather >> Workers Planned: 2 >> -> Partial Aggregate >> -> Parallel Seq Scan on tenk1 >> Filter: (NOT (hashed SubPlan 1)) >> SubPlan 1 >> -> Seq Scan on tenk2 >> Filter: (thousand > 100) >> (9 rows) >> > > Oh, so non-correlated subqueries can be run in parallel. Yes, that is > something we should have in the release notes. How is this? > > Author: Robert Haas <rhaas@postgresql.org> > 2017-02-14 [5e6d8d2bb] Allow parallel workers to execute subplans. > > Allow non-correlated subqueries to be run in parallel (Amit Kapila) > Looks good to me. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
On Tue, Apr 25, 2017 at 8:59 AM, Bruce Momjian <bruce@momjian.us> wrote: > On Mon, Apr 24, 2017 at 11:05:41PM -0400, Bruce Momjian wrote: >> > I think the above commit needs a separate mention, as this is a really >> > huge step forward to control the size of hash indexes. >> >> Yes, it is unfotunate that the item is in the incompatibility item. I >> wonder if I should split out the need to rebuild the hash indexes and >> keep it there and move this item into the "Index" section. > > Done, items split. > <listitem> <!-- Author: Robert Haas <rhaas@postgresql.org> 2017-02-27 [b0f18cb77] hash: Refactor bucketsqueeze code. Author: Robert Haas <rhaas@postgresql.org> 2017-02-27 [30df93f69] hash: Refactor overflowpage allocation. Author: Robert Haas <rhaas@postgresql.org> 2017-04-03 [ea69a0dea] Expand hash indexesmore gradually. --> <para> Improve efficiency of hash index growth (Amit Kapila, Mithun Cy) </para></listitem> The first two commits b0f18cb77, 30df93f69 are done as preliminary work to "Add write-ahead logging support to hash indexes", so it seems inappropriate to add them here. We can add it along with below item: <!-- Author: Robert Haas <rhaas@postgresql.org> 2017-03-14 [c11453ce0] hash: Add write-ahead logging support. --> <para> Add write-ahead logging support to hash indexes (Amit Kapila) </para> -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
Hi, Bruce! 2017-04-25 6:31 GMT+05:00 Bruce Momjian <bruce@momjian.us>: > The only unusual thing is that this release has ~180 items while most > recent release have had ~220. The pattern I see that there are more > large features in this release than previous ones. I'm not sure, but, probably, it worth mentioning this [1,2] in the E.1.3.1.2. Indexes section. Better tuple management on GiST page allows faster inserts and updates. (In numbers: 15% for randomized data,3x for ordered data in specific cases) [1] https://commitfest.postgresql.org/10/661/ [2] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=b1328d78f88cdf4f7504004159e530b776f0de16 Best regards, Andrey Borodin.
Hello Bruce, > I have committed the first draft of the Postgres 10 release notes. They > are current as of two days ago, and I will keep them current. Please > give me any feedback you have. About: """ Fix psql \p to always print what would be executed by \g or \w (Daniel Vérité) Previously \p didn't properly print the reverted-to command after a buffer contents reset. CLARIFY? """ The fix is linked to the change introduced by Tom when refactoring/cleaning up in e984ef586 (\if) which change psql's \p behavior. This is not a user visible change version-to-version, it is more like a bug fix for a bug/behavioral change introduced within pg10 developement process itself. I'm not sure how this should appear in the release notes. Maybe not at all, associated to the feature in which the behavioral change was introduced... -- Fabien.
Fabien COELHO wrote: > Fix psql \p to always print what would be executed by \g or \w (Daniel > Vérité) > > Previously \p didn't properly print the reverted-to command after a > buffer contents reset. CLARIFY? > > The fix is linked to the change introduced by Tom when > refactoring/cleaning up in e984ef586 (\if) which change psql's \p > behavior. That's my recollection as well. The "Previously" does not refer to 9.6, but to that commit. > I'm not sure how this should appear in the release notes. Maybe not at > all, associated to the feature in which the behavioral change was > introduced... There is small change of behavior coming as a by-product of the introduction of /if.../endif blocks. When doing in 9.x: select '1st buffer' \g followed by \e and editing with select '2nd buffer' (without ending the query) and then back in psql doing '\r' and '\p', the result is select '2nd buffer' The same with v10 leads instead to select '1st buffer' I'm not sure whether it's above the level of detail worth being mentioned in the release notes. Best regards, -- Daniel Vérité PostgreSQL-powered mailer: http://www.manitou-mail.org Twitter: @DanielVerite
On Thu, Apr 27, 2017 at 08:00:28AM +0530, Amit Kapila wrote: > > Oh, so non-correlated subqueries can be run in parallel. Yes, that is > > something we should have in the release notes. How is this? > > > > Author: Robert Haas <rhaas@postgresql.org> > > 2017-02-14 [5e6d8d2bb] Allow parallel workers to execute subplans. > > > > Allow non-correlated subqueries to be run in parallel (Amit Kapila) > > > > Looks good to me. OK, added. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Thu, Apr 27, 2017 at 08:14:34AM +0530, Amit Kapila wrote: > <listitem> > <!-- > Author: Robert Haas <rhaas@postgresql.org> > 2017-02-27 [b0f18cb77] hash: Refactor bucket squeeze code. > Author: Robert Haas <rhaas@postgresql.org> > 2017-02-27 [30df93f69] hash: Refactor overflow page allocation. > Author: Robert Haas <rhaas@postgresql.org> > 2017-04-03 [ea69a0dea] Expand hash indexes more gradually. > --> > <para> > Improve efficiency of hash index growth (Amit Kapila, Mithun Cy) > </para> > </listitem> > > The first two commits b0f18cb77, 30df93f69 are done as preliminary > work to "Add write-ahead logging support to hash indexes", so it seems > inappropriate to add them here. We can add it along with below item: > > <!-- > Author: Robert Haas <rhaas@postgresql.org> > 2017-03-14 [c11453ce0] hash: Add write-ahead logging support. > --> > <para> > Add write-ahead logging support to hash indexes (Amit Kapila) > </para> OK, commits moved. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Thu, Apr 27, 2017 at 4:13 PM, Bruce Momjian <bruce@momjian.us> wrote:
On Thu, Apr 27, 2017 at 08:00:28AM +0530, Amit Kapila wrote:
> > Oh, so non-correlated subqueries can be run in parallel. Yes, that is
> > something we should have in the release notes. How is this?
> >
> > Author: Robert Haas <rhaas@postgresql.org>
> > 2017-02-14 [5e6d8d2bb] Allow parallel workers to execute subplans.
> >
> > Allow non-correlated subqueries to be run in parallel (Amit Kapila)
> >
>
> Looks good to me.
It's not clear from this item what the previous behavior was. How about adding something like "Correlated subqueries can not yet be parallelized."? (I'm guessing that's the change, anyway, please correct me if I'm mistaken.)
.m
Bruce Momjian <bruce@momjian.us> writes:
> I have committed the first draft of the Postgres 10 release notes. They
> are current as of two days ago, and I will keep them current. Please
> give me any feedback you have.
I noticed a few niggles with the links in "my" entires, patch attached.
Cheers,
ilmari
--
"A disappointingly low fraction of the human race is,
at any given time, on fire." - Stig Sandbeck Mathisen
From a32336f420a4fb25bdedeb0dc8ccf68503638e93 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Dagfinn=20Ilmari=20Manns=C3=A5ker?= <ilmari@ilmari.org>
Date: Thu, 27 Apr 2017 15:20:59 +0100
Subject: [PATCH] Fix some release note links
- Make max_pred_locks_per_page a link too
- The ALTER TYPE link was pointing to the ALTER TABLE page
---
doc/src/sgml/release-10.sgml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/release-10.sgml b/doc/src/sgml/release-10.sgml
index de2129758c..6f49618d07 100644
--- a/doc/src/sgml/release-10.sgml
+++ b/doc/src/sgml/release-10.sgml
@@ -665,7 +665,7 @@
<para>
The new settings are <xref
linkend="guc-max-pred-locks-per-relation"> and
- <varname>max_pred_locks_per_page</>.
+ <xref linkend="guc-max-pred-locks-per-page">.
</para>
</listitem>
@@ -1830,7 +1830,7 @@
</para>
<para>
- This uses the syntax <link linkend="SQL-ALTERTABLE"><command>ALTER
+ This uses the syntax <link linkend="SQL-ALTERTYPE"><command>ALTER
TYPE ... RENAME VALUE</></>.
</para>
</listitem>
--
2.11.0
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Apr 27, 2017 at 7:53 PM, Marko Tiikkaja <marko@joh.to> wrote: > On Thu, Apr 27, 2017 at 4:13 PM, Bruce Momjian <bruce@momjian.us> wrote: >> >> On Thu, Apr 27, 2017 at 08:00:28AM +0530, Amit Kapila wrote: >> > > Oh, so non-correlated subqueries can be run in parallel. Yes, that is >> > > something we should have in the release notes. How is this? >> > > >> > > Author: Robert Haas <rhaas@postgresql.org> >> > > 2017-02-14 [5e6d8d2bb] Allow parallel workers to execute >> > > subplans. >> > > >> > > Allow non-correlated subqueries to be run in parallel (Amit >> > > Kapila) >> > > >> > >> > Looks good to me. > > > It's not clear from this item what the previous behavior was. How about > adding something like "Correlated subqueries can not yet be parallelized."? > (I'm guessing that's the change, anyway, please correct me if I'm mistaken.) > Previous to this change queries containing a reference to subplans (correlated or un-correlated) doesn't get parallelised. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
On Tue, Apr 25, 2017 at 10:31 AM, Bruce Momjian <bruce@momjian.us> wrote: > I have committed the first draft of the Postgres 10 release notes. They > are current as of two days ago, and I will keep them current. Please > give me any feedback you have. > Related to a following item in release note, oldestxmin is also reported by VACUUM VERBOSE and autovacuum , which is introduced by commit 9eb344faf54a849898d9be012ddfa8204cfeb57c (author is Simon). * Have VACUUM VERBOSE report the number of skipped frozen pages (Masahiko Sawada) Could we add this item to the release note? Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
On Tue, Apr 25, 2017 at 11:01 AM, Bruce Momjian <bruce@momjian.us> wrote: >> Or the ability of logical decoding to follow timeline switches. > > I didn't think logical decoding was really more than a proof-of-concept > until now. /me searches for jaw on floor. It sounds like you don't understand how logical decoding works. There are plugins -- fairly widely used, I think -- like https://github.com/confluentinc/bottledwater-pg and https://github.com/eulerto/wal2json which use the in-core infrastructure to do very nifty things, much like there are foreign data wrappers other than postgres_fdw. Even test_decoding is (perhaps regrettably) being used to build production solutions. The point is that most of the logic is in core; test_decoding or bottlewater or wal2json are just small plugins that tap into that infrastructure. I would not in any way refer to logical decoding as being only a proof of concept, even before logical replication. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Tue, Apr 25, 2017 at 9:56 PM, Bruce Momjian <bruce@momjian.us> wrote: > First, I don't think RFC references belong in the release notes, let > alone RFC links. Why not? I think RFC references are a great thing to include in the release notes, and including links seems helpful, too. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 05/01/2017 05:40 AM, Robert Haas wrote: > On Tue, Apr 25, 2017 at 9:56 PM, Bruce Momjian <bruce@momjian.us> wrote: >> First, I don't think RFC references belong in the release notes, let >> alone RFC links. > > Why not? I think RFC references are a great thing to include in the > release notes, and including links seems helpful, too. I could see not including RFC references in a PR but release notes seem to be the perfect place for them. JD -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.
On Mon, Apr 24, 2017 at 11:37 PM, Bruce Momjian <bruce@momjian.us> wrote: >> I think that might also be because you skipped a few things that should >> get their own entries. I've not yet made a pass through your draft (and >> won't for some days), but a quick search shows the draft to e.g miss: >> b30d3ea824c5ccba43d3e942704f20686e7dbab8 - Add a macro templatized hashtable. >> 75ae538bc3168bf44475240d4e0487ee2f3bb376 - Use more efficient hashtable for tidbitmap.c to speed up bitmap scans. >> 5dfc198146b49ce7ecc8a1fc9d5e171fb75f6ba5 - Use more efficient hashtable for execGrouping.c to speed up hash aggregation. >> fc4b3dea2950e4f6081f1ed2380f82c9efd672e0 - User narrower representative tuples in the hash-agg hashtable. >> 8ed3f11bb045ad7a3607690be668dbd5b3cc31d7 - Perform one only projection to compute agg arguments. >> (not that this needs to five entries) > > I remember seeing those and those are normally details I do not put in > the release notes as there isn't a clear user experience change except > "Postgres is faster". Yeah, a bummer, and I can change my filter, but > it would require discussion. I'm pretty sure this is not the first year in which your policy of excluding certain performance-related items has met with opposition. I agree that there are some improvements that are sufficiently small and boring that they do not merit a mention, but (1) that's also true of non-performance items and (2) the fact that people keep complaining about performance items you excluded constitutes a discussion of changing your filter. My own opinion is that the filter should not be particularly different for performance items vs. non-performance. The question ought to be whether the change is significant enough that users are likely to care. If you've got several people saying "hey, you forgot NNNNNN in the release notes!" it is probably a good bet that the change is significant enough that users will care about it. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Claudio Freire wrote: > On Tue, Apr 25, 2017 at 2:45 PM, Bruce Momjian <bruce@momjian.us> wrote: > > However, given your explanation, I have added the item: > > > > Improve speed of <command>VACUUM</>'s removal of trailing empty > > heap pages (Alvaro Herrera) > > That's enough for me, thanks. Thanks! I amended the entry by cons'ing Claudio's name as author. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Bruce Momjian wrote: > On Tue, Apr 25, 2017 at 04:03:53PM +1200, David Rowley wrote: > > ..On 25 April 2017 at 13:31, Bruce Momjian <bruce@momjian.us> wrote: > > > The only unusual thing is that this release has ~180 items while most > > > recent release have had ~220. The pattern I see that there are more > > > large features in this release than previous ones. > > > > Thanks for drafting this up. > > > > I understand that it may have been filtered out, but I'd say that > > 7e534adcdc70 is worth a mention. > > > > Users creating BRIN indexes previously would have had to know > > beforehand that the table would be sufficiently correlated on the > > indexed columns for the index to be worthwhile, whereas now there's a > > lesser need for the user to know this beforehand. > > I can't see how this can be added to an existing BRIN entry, so it would > have to be new. The text would be: > > Improve accuracy in determining if a BRIN index scan is beneficial > > though this not something I would normally mention becuause most users > don't understand the optimizer choices and just assume it works. The problem is that previously it was possible for a BRIN index to be created, and cause some queries to choose it which were faster using some other index (a btree). The point is that with this change it becomes more credible to create BRIN indexes without fear that such queries are going to slow down. Your proposed text sounds good to me. Authors would be David Rowley and Emre Hasegeli. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Thanks for doing this, looks great. A few notes:
<listitem> <!-- Author: Alvaro Herrera <alvherre@alvh.no-ip.org> 2017-03-24 [7b504eb28] Implement
multivariaten-distinct coefficients Author: Simon Riggs <simon@2ndQuadrant.com> 2017-04-05 [2686ee1b7]
Collectand use multi-column dependency stats --> <para> Add the ability to compute a correlation ratio
andthe number of distinct values on several columns (Tomas Vondra, David Rowley) </para>
I think this should be worded in terms of "extended data statistics" or
such. I think your proposed wording is a bit obscure. How about
something like "Add extended statistics to improve query planning".
Also, I'd add myself as co-author, with Tomas' permission.
<listitem> <!-- Author: Alvaro Herrera <alvherre@alvh.no-ip.org> 2017-04-01 [7526e1022] BRIN
auto-summarization --> <para> Cause <acronym>BRIN</> index summarization to happen more
aggressively(Álvaro Herrera) </para>
<para> Specifically, summarize the previous page range when a new page range is created. </para>
</listitem>
I think it should be pointed out that this is optional and defaults to
off. Maybe start with "add option to ..." (I wonder if it's worth
creating a linkable area in the docs that explain this feature, so that
we could have a link in the relnotes entry).
<listitem> <!-- Author: Alvaro Herrera <alvherre@alvh.no-ip.org> 2017-04-01 [c655899ba] BRIN
de-summarization --> <para> Add function <function>brin_desummarize_range()</> to remove
<acronym>BRIN</>summarization of a specified range (Álvaro Herrera) </para>
<para> This allows future <acronym>BRIN</> index summarization to be more compact. CLARIFY
</para> </listitem>
The point of this change is that if data has changed (delete, update) in
a way that could use tighter min/max limits, it would be worthwhile to
remove the previous BRIN tuple so that a new one is created so that
future scans can skip scanning the range. Maybe word it something like
"This is useful if UPDATE and DELETE have changed the data to tighter
limits; a future BRIN index entry will be more accurate"?
<listitem> <!-- Author: Alvaro Herrera <alvherre@alvh.no-ip.org> 2017-03-08 [fcec6caaf] Support
XMLTABLEquery expression --> <para> Add support for converting <type>XML</>-formatted data into a row
set(Pavel Stehule, Álvaro Herrera) </para>
XMLTABLE is a sql-standard-specified construct, so we should mention it
by name in the leading paragraph more prominently ("add support for the
XMLTABLE standard feature" or something like that); also, I think it
should be in the Queries section, not Functions.
<para> Add <acronym>GUC</> <xref linkend="guc-max-parallel-workers"> to limit the number of worker
processesthat can be used for parallelism (Julien Rouhaud) </para>
<para> This can be set lower than <xref linkend="guc-max-worker-processes"> to reserve worker
processes for non-parallel purposes. </para>
Strange last phrase. I suggest "...to reserve worker processes for
purposes other than parallel queries". Also in the leading para, "for
parallelism" should probably be "for query parallelism".
I think commit e306df7f9 ("Full Text Search support for <type>JSON</>
and <type>JSONB</>") does definitely not belong to section Indexes; I
think Data Types is a better fit.
I think commits 67dc4ccbb and 1753b1b02 (pg_sequence and pg_sequences)
should be listed in the opposite order. Maybe merge them into one?
<para> This proves better security than the existing <literal>md5</> negotiation and storage method.
</para>
You probably mean "provides" not "proves".
<para> Allow <acronym>SSL</> configuration to be updated at <literal>SIGHUP</> (Andreas Karlsson, Tom
Lane) </para>
SIGHUP seems much too technical. How about "... to be updated during
configurating reload"
I think the entry for commits a734fd5d1 and dafa0848d does not belong in
the reliability section. In fact I wonder if it's necessary to list
this one at all.
<para> Increase the maximum configurable <acronym>WAL</> size to 1 gigabyte (Beena Emerson)
</para>
"WAL segment size" perhaps, and note that this is useful to reduce
frequency of archive_command?
<para> Also add <option>-nosync</> option to disable fsync. </para>
Misspelled --no-sync.
<para> Add log options for <application>pg_ctl</> wait (<option>--wait</>) and no-wait
(<option>--no-wait</>)(Vik Fearing) </para>
Mispelled "long options".
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Mon, May 1, 2017 at 7:02 AM, Robert Haas <robertmhaas@gmail.com> wrote: > On Tue, Apr 25, 2017 at 11:01 AM, Bruce Momjian <bruce@momjian.us> wrote: >> I didn't think logical decoding was really more than a proof-of-concept >> until now. > > /me searches for jaw on floor. > > I would not in any way refer to logical decoding as being only a proof > of concept, even before logical replication. That's fair, but I think I understand what Bruce was going for here. Data point: github third party modules are generally not approved for deployment in my organization so logical decoding from a production perspective does not exist (for me) until 10.0. Point being, an important threshold has been crossed. merlin
On 2017-04-25 15:29:01 -0400, Bruce Momjian wrote: > Uh, the only logical decoding code that I know we ship pre-PG 10 is > contrib/test_decoding/. That's completely wrong. src/backend/replication/logical/ is a a bit bigger than that... - Andres
On 2017-05-04 17:33:13 -0500, Merlin Moncure wrote: > On Mon, May 1, 2017 at 7:02 AM, Robert Haas <robertmhaas@gmail.com> wrote: > > On Tue, Apr 25, 2017 at 11:01 AM, Bruce Momjian <bruce@momjian.us> wrote: > >> I didn't think logical decoding was really more than a proof-of-concept > >> until now. > > > > /me searches for jaw on floor. Yea, this kind of argument is getting really old. People, including Robert and I in this thread, have spent tremendous amounts of work on it. Not fun to just get that discounted. > > I would not in any way refer to logical decoding as being only a proof > > of concept, even before logical replication. > > That's fair, but I think I understand what Bruce was going for here. > Data point: github third party modules are generally not approved for > deployment in my organization so logical decoding from a production > perspective does not exist (for me) until 10.0. Point being, an > important threshold has been crossed. By that argument the extension APIs, which after all are what external FDWs, external index types, postgis, and other extensions use, aren't a feature of postgres. Some sites not being able to use external extensions is *one* argument for building some things into core, but that doesn't mean the extension APIs don't exist or aren't features. - Andres
On Thu, Apr 27, 2017 at 10:21:44AM +0500, Andrew Borodin wrote: > Hi, Bruce! > > 2017-04-25 6:31 GMT+05:00 Bruce Momjian <bruce@momjian.us>: > > The only unusual thing is that this release has ~180 items while most > > recent release have had ~220. The pattern I see that there are more > > large features in this release than previous ones. > > I'm not sure, but, probably, it worth mentioning this [1,2] in the > E.1.3.1.2. Indexes section. > Better tuple management on GiST page allows faster inserts and > updates. (In numbers: 15% for randomized data,3x for ordered data in > specific cases) > > [1] https://commitfest.postgresql.org/10/661/ > [2] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=b1328d78f88cdf4f7504004159e530b776f0de16 OK, got it. Here is the new item: Author: Tom Lane <tgl@sss.pgh.pa.us> 2016-09-09 [b1328d78f] Invent PageIndexTupleOverwrite, and teach BRINand GiST Allow faster <acronym>GiST</> inserts and updates by reusing index space more efficiently (Andrey Borodin) -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Thu, Apr 27, 2017 at 10:43:34AM +0200, Fabien COELHO wrote: > > Hello Bruce, > > >I have committed the first draft of the Postgres 10 release notes. They > >are current as of two days ago, and I will keep them current. Please > >give me any feedback you have. > > About: > > """ > Fix psql \p to always print what would be executed by \g or \w (Daniel > Vérité) > > Previously \p didn't properly print the reverted-to command after a > buffer contents reset. CLARIFY? > """ > > The fix is linked to the change introduced by Tom when refactoring/cleaning > up in e984ef586 (\if) which change psql's \p behavior. > > This is not a user visible change version-to-version, it is more like a bug > fix for a bug/behavioral change introduced within pg10 developement process > itself. > > I'm not sure how this should appear in the release notes. Maybe not at all, > associated to the feature in which the behavioral change was introduced... Agreed, I removed the item and moved those commits to the \if item. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Thu, Apr 27, 2017 at 03:04:57PM +0200, Daniel Verite wrote: > Fabien COELHO wrote: > > > Fix psql \p to always print what would be executed by \g or \w (Daniel > > Vérité) > > > > Previously \p didn't properly print the reverted-to command after a > > buffer contents reset. CLARIFY? > > > > The fix is linked to the change introduced by Tom when > > refactoring/cleaning up in e984ef586 (\if) which change psql's \p > > behavior. > > That's my recollection as well. The "Previously" does not refer to 9.6, > but to that commit. Yes, adjusted. > > I'm not sure how this should appear in the release notes. Maybe not at > > all, associated to the feature in which the behavioral change was > > introduced... > > There is small change of behavior coming as a by-product of the > introduction of /if.../endif blocks. > > When doing in 9.x: > > select '1st buffer' \g > followed by \e > and editing with select '2nd buffer' (without ending the query) > and then back in psql doing '\r' and '\p', the result is > select '2nd buffer' > > The same with v10 leads instead to > select '1st buffer' > > I'm not sure whether it's above the level of detail worth being mentioned > in the release notes. Wow, I am not sure how to even explain that. ;-) Let's see if we get any user feedback on the change. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Thu, Apr 27, 2017 at 04:05:09PM +0100, Dagfinn Ilmari Mannsåker wrote: > Bruce Momjian <bruce@momjian.us> writes: > > > I have committed the first draft of the Postgres 10 release notes. They > > are current as of two days ago, and I will keep them current. Please > > give me any feedback you have. > > I noticed a few niggles with the links in "my" entires, patch attached. Thanks, applied. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Fri, Apr 28, 2017 at 01:12:34PM +0900, Masahiko Sawada wrote:
> On Tue, Apr 25, 2017 at 10:31 AM, Bruce Momjian <bruce@momjian.us> wrote:
> > I have committed the first draft of the Postgres 10 release notes. They
> > are current as of two days ago, and I will keep them current. Please
> > give me any feedback you have.
> >
>
> Related to a following item in release note, oldestxmin is also
> reported by VACUUM VERBOSE and autovacuum , which is introduced by
> commit 9eb344faf54a849898d9be012ddfa8204cfeb57c (author is Simon).
>
> * Have VACUUM VERBOSE report the number of skipped frozen pages
> (Masahiko Sawada)
>
> Could we add this item to the release note?
OK, adjusted text below and commit added above this item:
Have <link linkend="SQL-VACUUM"><command>VACUUM VERBOSE</></> report the number of skipped frozen pages and
oldestxmin (Masahiko Sawada)
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
On Mon, May 1, 2017 at 08:02:46AM -0400, Robert Haas wrote: > On Tue, Apr 25, 2017 at 11:01 AM, Bruce Momjian <bruce@momjian.us> wrote: > >> Or the ability of logical decoding to follow timeline switches. > > > > I didn't think logical decoding was really more than a proof-of-concept > > until now. > > /me searches for jaw on floor. > > It sounds like you don't understand how logical decoding works. There > are plugins -- fairly widely used, I think -- like > https://github.com/confluentinc/bottledwater-pg and > https://github.com/eulerto/wal2json which use the in-core > infrastructure to do very nifty things, much like there are foreign > data wrappers other than postgres_fdw. Even test_decoding is (perhaps > regrettably) being used to build production solutions. The point is > that most of the logic is in core; test_decoding or bottlewater or > wal2json are just small plugins that tap into that infrastructure. > > I would not in any way refer to logical decoding as being only a proof > of concept, even before logical replication. The community ships a reliable logical _encoding_, and a test logical _decoding_. This came up from discussion related to this item: the ability of logical decoding to follow timeline switches My point was that based on the text it is test_decoding that can do timeline switches, and is that significant enough to mention in the release notes? Now, if it is that logical "encoding" now allows external logical decoding modules to handle timeline switches, that is different, but no one has said that yet. You can have all the emotional reactions you want. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Mon, May 1, 2017 at 10:20:38AM -0400, Robert Haas wrote: > I'm pretty sure this is not the first year in which your policy of > excluding certain performance-related items has met with opposition. > I agree that there are some improvements that are sufficiently small > and boring that they do not merit a mention, but (1) that's also true > of non-performance items and (2) the fact that people keep complaining > about performance items you excluded constitutes a discussion of > changing your filter. > > My own opinion is that the filter should not be particularly different > for performance items vs. non-performance. The question ought to be > whether the change is significant enough that users are likely to > care. If you've got several people saying "hey, you forgot NNNNNN in > the release notes!" it is probably a good bet that the change is > significant enough that users will care about it. Yes, the "do people care" filter is what I usually use. When new functionality is added, we usually mention it because someone usually care, but for performance, the threshold is usually whether workloads, even rare ones, would have a visible performance change. It is difficult to determine this from the release notes, which is why I always need feedback on these items. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On 2017-05-04 19:56:21 -0400, Bruce Momjian wrote: > On Mon, May 1, 2017 at 08:02:46AM -0400, Robert Haas wrote: > > On Tue, Apr 25, 2017 at 11:01 AM, Bruce Momjian <bruce@momjian.us> wrote: > > >> Or the ability of logical decoding to follow timeline switches. > > > > > > I didn't think logical decoding was really more than a proof-of-concept > > > until now. > > > > /me searches for jaw on floor. > > > > It sounds like you don't understand how logical decoding works. There > > are plugins -- fairly widely used, I think -- like > > https://github.com/confluentinc/bottledwater-pg and > > https://github.com/eulerto/wal2json which use the in-core > > infrastructure to do very nifty things, much like there are foreign > > data wrappers other than postgres_fdw. Even test_decoding is (perhaps > > regrettably) being used to build production solutions. The point is > > that most of the logic is in core; test_decoding or bottlewater or > > wal2json are just small plugins that tap into that infrastructure. > > > > I would not in any way refer to logical decoding as being only a proof > > of concept, even before logical replication. > > The community ships a reliable logical _encoding_, and a test logical > _decoding_. Yes, so what? What you said is "I didn't think logical decoding was really more than a proof-of-concept until now", which is plainly wrong, given I know a significant number of users using it in production. Some of them are well known & large enterprises, and it's used to enable critical things. On Mon, May 1, 2017 at 08:02:46AM -0400, Robert Haas wrote: > Even test_decoding is (perhaps regrettably) being used to build production solutions. E.g. to power amazon's data migration service (yes, that scares me). > My point was that based on the text it is test_decoding that can do > timeline switches, and is that significant enough to mention in the > release notes? Now, if it is that logical "encoding" now allows > external logical decoding modules to handle timeline switches, that is > different, but no one has said that yet. The change has nothing to do with test_decoding. Petr: The timeline change itself does, for the moment, not seem particularly noteworthy to me - it's not really useful atm on its own? To me it's more of infrastructure to add "logical decoding on standby" next release? > You can have all the emotional reactions you want. Nice one.
On Thu, May 4, 2017 at 06:02:58PM -0300, Alvaro Herrera wrote:
> > I can't see how this can be added to an existing BRIN entry, so it would
> > have to be new. The text would be:
> >
> > Improve accuracy in determining if a BRIN index scan is beneficial
> >
> > though this not something I would normally mention becuause most users
> > don't understand the optimizer choices and just assume it works.
>
> The problem is that previously it was possible for a BRIN index to be
> created, and cause some queries to choose it which were faster using
> some other index (a btree). The point is that with this change it
> becomes more credible to create BRIN indexes without fear that such
> queries are going to slow down.
>
> Your proposed text sounds good to me. Authors would be David Rowley and
> Emre Hasegeli.
OK, added, thanks:
<listitem> <!--
+ Author: Alvaro Herrera <alvherre@alvh.no-ip.org>
+ 2017-04-06 [7e534adcd] Fix BRIN cost estimation
+ -->
+ <para>
+ Improve accuracy in determining if a <acronym>BRIN</> index scan
+ is beneficial (David Rowley, Emre Hasegeli)
+ </para>
+ </listitem>
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
On Thu, May 4, 2017 at 07:01:17PM -0300, Alvaro Herrera wrote:
> Thanks for doing this, looks great. A few notes:
>
> <listitem>
> <!--
> Author: Alvaro Herrera <alvherre@alvh.no-ip.org>
> 2017-03-24 [7b504eb28] Implement multivariate n-distinct coefficients
> Author: Simon Riggs <simon@2ndQuadrant.com>
> 2017-04-05 [2686ee1b7] Collect and use multi-column dependency stats
> -->
> <para>
> Add the ability to compute a correlation ratio and the number of
> distinct values on several columns (Tomas Vondra, David Rowley)
> </para>
>
> I think this should be worded in terms of "extended data statistics" or
> such. I think your proposed wording is a bit obscure. How about
> something like "Add extended statistics to improve query planning".
> Also, I'd add myself as co-author, with Tomas' permission.
I have adjusted the text to add your term, and added your name:
Add multi-column optimizer statistics to compute the correlation ratio and number of distinct values (Tomas
Vondra,David Rowley, Álvaro Herrera)
I think we have to mention the exact statistics collected because we
know that they are of limited usefulness (per Tomas) and full
multi-column statistics are needed and hopefully coming in PG 11. If we
don't mention the details I am afraid people will be disappointed with PG
10 and will not be excited when they are more powerful in PG 11. Any
better wording?
I will work on the other items you posted shortly.
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
On 2017-05-04 19:56:21 -0400, Bruce Momjian wrote: > On Mon, May 1, 2017 at 08:02:46AM -0400, Robert Haas wrote: > This came up from discussion related to this item: > > the ability of logical decoding to follow timeline switches > > My point was that based on the text it is test_decoding that can do > timeline switches, and is that significant enough to mention in the > release notes? FWIW, the relevant commit neither mentions nor modifies test_decoding (It does add tests that *use* test_decoding, but that's it). Nor did Petr's comment reference test_decoding. So I don't see how "based on the text it is test_decoding" follows. - Andres
On Thu, May 4, 2017 at 05:09:40PM -0700, Andres Freund wrote: > > > I would not in any way refer to logical decoding as being only a proof > > > of concept, even before logical replication. > > > > The community ships a reliable logical _encoding_, and a test logical > > _decoding_. > > Yes, so what? What you said is "I didn't think logical decoding was > really more than a proof-of-concept until now", which is plainly wrong, > given I know a significant number of users using it in production. Some > of them are well known & large enterprises, and it's used to enable > critical things. I am getting tired of saying this. When I am writing the release notes, I am trying to figure out how it affects our shipped code, and the only "decoding" I know of is test_decoding. My message was this: https://www.postgresql.org/message-id/a6d13cf7-fbf8-913c-2353-f149c6f95fdc%402ndquadrant.com> >> Or the ability of logicaldecoding to follow timeline switches.> > I didn't think logical decoding was really more than a proof-of-concept>until now. Now, if the "encoding" was changed so the external decoders can do more, or the API we supply to the decoders has new abilities, that is fine to mention, but someone needs to say that. That is an "encoding" change, right? I said nothing about external decoders, only the user-interface decoder we ship. I assume it is this commit: Author: Simon Riggs <simon@2ndQuadrant.com>2017-03-22 [1148e22a8] Teach xlogreader to follow timeline switches Teach xlogreaderto follow timeline switches Uses page-based mechanism to ensure we’re using the correct timeline. Tests areincluded to exercise the functionality using a cold disk-level copy of the master that's started up as a replica withslots intact, but the intended use of the functionality is with later features. Craig Ringer, reviewed by SimonRiggs and Andres Freund -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Thu, May 4, 2017 at 7:56 PM, Bruce Momjian <bruce@momjian.us> wrote: > The community ships a reliable logical _encoding_, and a test logical > _decoding_. As far as I am aware, logical encoding is a term you just made up, because it's not referenced anywhere in the commit log or the source tree, unlike logical decoding, which is referenced many times. [rhaas pgsql]$ git log --grep='logical decoding' --oneline | wc -l 59 [rhaas pgsql]$ git log --grep='logical encoding' --oneline | wc -l 0 [rhaas pgsql]$ git grep 'logical decoding' | wc -l 242 [rhaas pgsql]$ git grep 'logical encoding' | wc -l 0 > This came up from discussion related to this item: > > the ability of logical decoding to follow timeline switches > > My point was that based on the text it is test_decoding that can do > timeline switches, and is that significant enough to mention in the > release notes? Actually, that commit message says nothing about test_decoding. You seem to be confusing a logical decoding output plugin (of which test_decoding is an example) with the core logical decoding facilities (which are what got modified by this commit). As Andres said, that's like confusing postgres_fdw with the fact that PostgreSQL supports foreign data wrappers. > You can have all the emotional reactions you want. I'll try to avoid emotional reactions on the list, but I think you should try to understand how the features work if you're going to write the release notes. The email to which I responded was factually incorrect, and what you said in the email to which I'm replying now was, too. If you want to keep insisting otherwise, you have that right. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 2017-05-04 20:46:24 -0400, Bruce Momjian wrote: > On Thu, May 4, 2017 at 05:09:40PM -0700, Andres Freund wrote: > > > > I would not in any way refer to logical decoding as being only a proof > > > > of concept, even before logical replication. > > > > > > The community ships a reliable logical _encoding_, and a test logical > > > _decoding_. > > > > Yes, so what? What you said is "I didn't think logical decoding was > > really more than a proof-of-concept until now", which is plainly wrong, > > given I know a significant number of users using it in production. Some > > of them are well known & large enterprises, and it's used to enable > > critical things. > > I am getting tired of saying this. When I am writing the release notes, > I am trying to figure out how it affects our shipped code, and the only > "decoding" I know of is test_decoding. Then ask for input instead of saying uninformed stuff like "I didn't think logical decoding was really more than a proof-of-concept". That's what people are complaining about, not that you're not all-knowledgeable. I agree with you - and wrote that previously - that the specific change is not necessarily worthwhile to be mentioned on its own, I don't see how that makes your various statements in this subthread ok. My message was this: > https://www.postgresql.org/message-id/a6d13cf7-fbf8-913c-2353-f149c6f95fdc%402ndquadrant.com > > > >> Or the ability of logical decoding to follow timeline switches. > > > > I didn't think logical decoding was really more than a proof-of-concept > > until now. > > Now, if the "encoding" There's not really a "encoding" part of this. The decoding refers to decoding the WAL into something externally usable. > I assume it is this commit: That, and commit 24c5f1a103ce6656a5cb430d9a996c34e61ab2a5 Author: Alvaro Herrera <alvherre@alvh.no-ip.org> Date: 2016-03-30 20:07:05 -0300 Enable logical slots to follow timeline switches When decoding from a logical slot, it's necessary for xlog readingto be able to read xlog from historical (i.e. not current) timelines; otherwise, decoding fails after failover,because the archives are in the historical timeline. This is required to make "failover logical slots" possible;it currently has no other use, although theoretically it could be used by an extension that creates a slot ona standby and continues to replay from the slot when the standby is promoted. This commit includes a module in src/test/moduleswith functions to manipulate the slots (which is not otherwise possible in SQL code) in order to enabletesting, and a new test in src/test/recovery to ensure that the behavior is as expected. Author: Craig Ringer Reviewed-By: Oleksii Kliukin, Andres Freund, Petr Jelínek
On Thu, May 4, 2017 at 8:09 PM, Andres Freund <andres@anarazel.de> wrote: > On Mon, May 1, 2017 at 08:02:46AM -0400, Robert Haas wrote: >> Even test_decoding is (perhaps regrettably) being used to build production solutions. > > E.g. to power amazon's data migration service (yes, that scares me). If you recall, I did predict prior to commit that test_decoding would get put into production use regardless of the name. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 2017-05-04 21:08:41 -0400, Robert Haas wrote: > On Thu, May 4, 2017 at 8:09 PM, Andres Freund <andres@anarazel.de> wrote: > > On Mon, May 1, 2017 at 08:02:46AM -0400, Robert Haas wrote: > >> Even test_decoding is (perhaps regrettably) being used to build production solutions. > > > > E.g. to power amazon's data migration service (yes, that scares me). > > If you recall, I did predict prior to commit that test_decoding would > get put into production use regardless of the name. Yea. I think we really should add a generic json plugin into core, I've seen three (I think) distinct versions deployed into production now. One of them derived from Euler's https://github.com/eulerto/wal2json, two apparently developed largely from scratch (more efficient schema handling). - Andres
On Thu, May 4, 2017 at 8:46 PM, Bruce Momjian <bruce@momjian.us> wrote: > On Thu, May 4, 2017 at 05:09:40PM -0700, Andres Freund wrote: >> > > I would not in any way refer to logical decoding as being only a proof >> > > of concept, even before logical replication. >> > >> > The community ships a reliable logical _encoding_, and a test logical >> > _decoding_. >> >> Yes, so what? What you said is "I didn't think logical decoding was >> really more than a proof-of-concept until now", which is plainly wrong, >> given I know a significant number of users using it in production. Some >> of them are well known & large enterprises, and it's used to enable >> critical things. > > I am getting tired of saying this. When I am writing the release notes, > I am trying to figure out how it affects our shipped code, and the only > "decoding" I know of is test_decoding. If you run 'git show --stat b89e151054a05f0f6d356ca52e3b725dd0505e53', you will see that it includes a test_decoding module (which is a sample logical decoding output plugin) and a tremendous pile of changes to src/backend/replication/logical (which is the core logical decoding infrastructure). The latter is a larger volume of changes than the former. It would perhaps be fair to describe test_decoding as a proof-of-concept, but it is not fair or correct to describe the core infrastructure that way. Anyway, they're separate things. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Thu, May 4, 2017 at 6:08 PM, Robert Haas <robertmhaas@gmail.com> wrote: >> E.g. to power amazon's data migration service (yes, that scares me). > > If you recall, I did predict prior to commit that test_decoding would > get put into production use regardless of the name. I thought you were completely wrong when you said that. Lesson learned, I suppose. -- Peter Geoghegan VMware vCenter Server https://www.vmware.com/
On 2017-05-04 18:23:38 -0700, Peter Geoghegan wrote: > On Thu, May 4, 2017 at 6:08 PM, Robert Haas <robertmhaas@gmail.com> wrote: > >> E.g. to power amazon's data migration service (yes, that scares me). > > > > If you recall, I did predict prior to commit that test_decoding would > > get put into production use regardless of the name. > > I thought you were completely wrong when you said that. Lesson > learned, I suppose. At least I was a bit more optimistic that we'd get a decent json plugin into care soon-ish. While there was some initial work towards that, it unfortunately stalled at some point. Euler, are you perchance interested in working towards that? ;) - Andres
On 05/05/17 03:26, Andres Freund wrote: > On 2017-05-04 18:23:38 -0700, Peter Geoghegan wrote: >> On Thu, May 4, 2017 at 6:08 PM, Robert Haas <robertmhaas@gmail.com> wrote: >>>> E.g. to power amazon's data migration service (yes, that scares me). >>> >>> If you recall, I did predict prior to commit that test_decoding would >>> get put into production use regardless of the name. >> >> I thought you were completely wrong when you said that. Lesson >> learned, I suppose. > > At least I was a bit more optimistic that we'd get a decent json plugin > into care soon-ish. While there was some initial work towards that, it > unfortunately stalled at some point. > > Euler, are you perchance interested in working towards that? ;) > I am happy to review any work in this area BTW. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On 05/05/17 02:46, Bruce Momjian wrote: > On Thu, May 4, 2017 at 05:09:40PM -0700, Andres Freund wrote: >>>> I would not in any way refer to logical decoding as being only a proof >>>> of concept, even before logical replication. >>> >>> The community ships a reliable logical _encoding_, and a test logical >>> _decoding_. >> >> Yes, so what? What you said is "I didn't think logical decoding was >> really more than a proof-of-concept until now", which is plainly wrong, >> given I know a significant number of users using it in production. Some >> of them are well known & large enterprises, and it's used to enable >> critical things. > > I am getting tired of saying this. When I am writing the release notes, > I am trying to figure out how it affects our shipped code, and the only > "decoding" I know of is test_decoding. My message was this: I actually think the main misunderstanding here comes from the test_decoding name interpretation. The test_decoding does not decode anything and there are no external "decoders". The decoding happens in core, the decoding just provides C API for plugins to consume the decoded info (ie, heap tuples and similar). The test_decoding *tests* the decoding API and the external projects use the decoding API as well but they don't really decode anything, their role is of filtering and deciding the wire protocol mostly. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Fri, May 5, 2017 at 02:28:50PM +0200, Petr Jelinek wrote: > On 05/05/17 02:46, Bruce Momjian wrote: > > I am getting tired of saying this. When I am writing the release notes, > > I am trying to figure out how it affects our shipped code, and the only > > "decoding" I know of is test_decoding. My message was this: > > I actually think the main misunderstanding here comes from the > test_decoding name interpretation. The test_decoding does not decode > anything and there are no external "decoders". The decoding happens in > core, the decoding just provides C API for plugins to consume the > decoded info (ie, heap tuples and similar). The test_decoding *tests* > the decoding API and the external projects use the decoding API as well > but they don't really decode anything, their role is of filtering and > deciding the wire protocol mostly. Yes, I thought test_decoding did decoding and not just use a backend-internal decoder. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Thu, May 4, 2017 at 07:01:17PM -0300, Alvaro Herrera wrote:
> Thanks for doing this, looks great. A few notes:
>
> <listitem>
> <!--
> Author: Alvaro Herrera <alvherre@alvh.no-ip.org>
> 2017-03-24 [7b504eb28] Implement multivariate n-distinct coefficients
> Author: Simon Riggs <simon@2ndQuadrant.com>
> 2017-04-05 [2686ee1b7] Collect and use multi-column dependency stats
> -->
> <para>
> Add the ability to compute a correlation ratio and the number of
> distinct values on several columns (Tomas Vondra, David Rowley)
> </para>
>
> I think this should be worded in terms of "extended data statistics" or
> such. I think your proposed wording is a bit obscure. How about
> something like "Add extended statistics to improve query planning".
> Also, I'd add myself as co-author, with Tomas' permission.
Already done.
> <listitem>
> <!--
> Author: Alvaro Herrera <alvherre@alvh.no-ip.org>
> 2017-04-01 [7526e1022] BRIN auto-summarization
> -->
> <para>
> Cause <acronym>BRIN</> index summarization to happen more
> aggressively (Álvaro Herrera)
> </para>
>
> <para>
> Specifically, summarize the previous page range when a new page
> range is created.
> </para>
> </listitem>
>
> I think it should be pointed out that this is optional and defaults to
> off. Maybe start with "add option to ..." (I wonder if it's worth
> creating a linkable area in the docs that explain this feature, so that
> we could have a link in the relnotes entry).
Wow, looking at the patch, there is a lot more going on. I have updated
the docs but please let me know if there is more needed.
Also, can ALTER INDEX change the index summariziation? Should it?
Also, when you remove summarization, when is it re-added? Why would you
remove summarization rather than just re-add it. I must be missing
something.
> <listitem>
> <!--
> Author: Alvaro Herrera <alvherre@alvh.no-ip.org>
> 2017-04-01 [c655899ba] BRIN de-summarization
> -->
> <para>
> Add function <function>brin_desummarize_range()</> to remove
> <acronym>BRIN</> summarization of a specified range (Álvaro
> Herrera)
> </para>
>
> <para>
> This allows future <acronym>BRIN</> index summarization to be
> more compact. CLARIFY
> </para>
> </listitem>
>
> The point of this change is that if data has changed (delete, update) in
> a way that could use tighter min/max limits, it would be worthwhile to
> remove the previous BRIN tuple so that a new one is created so that
> future scans can skip scanning the range. Maybe word it something like
> "This is useful if UPDATE and DELETE have changed the data to tighter
> limits; a future BRIN index entry will be more accurate"?
OK, updated, but please let me know if more changes are needed.
> <listitem>
> <!--
> Author: Alvaro Herrera <alvherre@alvh.no-ip.org>
> 2017-03-08 [fcec6caaf] Support XMLTABLE query expression
> -->
> <para>
> Add support for converting <type>XML</>-formatted data into a row
> set (Pavel Stehule, Álvaro Herrera)
> </para>
> XMLTABLE is a sql-standard-specified construct, so we should mention it
> by name in the leading paragraph more prominently ("add support for the
> XMLTABLE standard feature" or something like that); also, I think it
Done.
> should be in the Queries section, not Functions.
Done.
> <para>
> Add <acronym>GUC</> <xref linkend="guc-max-parallel-workers">
> to limit the number of worker processes that can be used for
> parallelism (Julien Rouhaud)
> </para>
>
> <para>
> This can be set lower than <xref
> linkend="guc-max-worker-processes"> to reserve worker processes
> for non-parallel purposes.
> </para>
>
> Strange last phrase. I suggest "...to reserve worker processes for
> purposes other than parallel queries". Also in the leading para, "for
> parallelism" should probably be "for query parallelism".
Done.
> I think commit e306df7f9 ("Full Text Search support for <type>JSON</>
> and <type>JSONB</>") does definitely not belong to section Indexes; I
> think Data Types is a better fit.
Moved.
> I think commits 67dc4ccbb and 1753b1b02 (pg_sequence and pg_sequences)
> should be listed in the opposite order. Maybe merge them into one?
Merged.
> <para>
> This proves better security than the existing <literal>md5</>
> negotiation and storage method.
> </para>
> You probably mean "provides" not "proves".
Fixed.
> <para>
> Allow <acronym>SSL</> configuration to be updated at
> <literal>SIGHUP</> (Andreas Karlsson, Tom Lane)
> </para>
> SIGHUP seems much too technical. How about "... to be updated during
> configurating reload"
Yes, updated.
> I think the entry for commits a734fd5d1 and dafa0848d does not belong in
> the reliability section. In fact I wonder if it's necessary to list
> this one at all.
Yes, I removed it. We can always re-add it later.
> <para>
> Increase the maximum configurable <acronym>WAL</> size to 1
> gigabyte (Beena Emerson)
> </para>
> "WAL segment size" perhaps, and note that this is useful to reduce
> frequency of archive_command?
Yes, updated.
> <para>
> Also add <option>-nosync</> option to disable fsync.
> </para>
> Misspelled --no-sync.
Fixed.
> <para>
> Add log options for <application>pg_ctl</> wait (<option>--wait</>)
> and no-wait (<option>--no-wait</>) (Vik Fearing)
> </para>
> Mispelled "long options".
Fixed.
Applied patch attached.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachment
On Fri, May 5, 2017 at 8:38 AM, Bruce Momjian <bruce@momjian.us> wrote: > On Fri, Apr 28, 2017 at 01:12:34PM +0900, Masahiko Sawada wrote: >> On Tue, Apr 25, 2017 at 10:31 AM, Bruce Momjian <bruce@momjian.us> wrote: >> > I have committed the first draft of the Postgres 10 release notes. They >> > are current as of two days ago, and I will keep them current. Please >> > give me any feedback you have. >> > >> >> Related to a following item in release note, oldestxmin is also >> reported by VACUUM VERBOSE and autovacuum , which is introduced by >> commit 9eb344faf54a849898d9be012ddfa8204cfeb57c (author is Simon). >> >> * Have VACUUM VERBOSE report the number of skipped frozen pages >> (Masahiko Sawada) >> >> Could we add this item to the release note? > > OK, adjusted text below and commit added above this item: > > Have <link linkend="SQL-VACUUM"><command>VACUUM VERBOSE</></> report > the number of skipped frozen pages and oldest xmin (Masahiko Sawada) > Thank you! But actually main author of this item is Simon, I didn't involved in it. Maybe we can have it as a separate item. Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
On Mon, May 8, 2017 at 12:18:54PM +0900, Masahiko Sawada wrote: > On Fri, May 5, 2017 at 8:38 AM, Bruce Momjian <bruce@momjian.us> wrote: > > On Fri, Apr 28, 2017 at 01:12:34PM +0900, Masahiko Sawada wrote: > >> On Tue, Apr 25, 2017 at 10:31 AM, Bruce Momjian <bruce@momjian.us> wrote: > >> > I have committed the first draft of the Postgres 10 release notes. They > >> > are current as of two days ago, and I will keep them current. Please > >> > give me any feedback you have. > >> > > >> > >> Related to a following item in release note, oldestxmin is also > >> reported by VACUUM VERBOSE and autovacuum , which is introduced by > >> commit 9eb344faf54a849898d9be012ddfa8204cfeb57c (author is Simon). > >> > >> * Have VACUUM VERBOSE report the number of skipped frozen pages > >> (Masahiko Sawada) > >> > >> Could we add this item to the release note? > > > > OK, adjusted text below and commit added above this item: > > > > Have <link linkend="SQL-VACUUM"><command>VACUUM VERBOSE</></> report > > the number of skipped frozen pages and oldest xmin (Masahiko Sawada) > > > > Thank you! But actually main author of this item is Simon, I didn't > involved in it. Maybe we can have it as a separate item. I just added Simon's name to the same item. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Mon, May 8, 2017 at 10:50 PM, Bruce Momjian <bruce@momjian.us> wrote: > On Mon, May 8, 2017 at 12:18:54PM +0900, Masahiko Sawada wrote: >> On Fri, May 5, 2017 at 8:38 AM, Bruce Momjian <bruce@momjian.us> wrote: >> > On Fri, Apr 28, 2017 at 01:12:34PM +0900, Masahiko Sawada wrote: >> >> On Tue, Apr 25, 2017 at 10:31 AM, Bruce Momjian <bruce@momjian.us> wrote: >> >> > I have committed the first draft of the Postgres 10 release notes. They >> >> > are current as of two days ago, and I will keep them current. Please >> >> > give me any feedback you have. >> >> > >> >> >> >> Related to a following item in release note, oldestxmin is also >> >> reported by VACUUM VERBOSE and autovacuum , which is introduced by >> >> commit 9eb344faf54a849898d9be012ddfa8204cfeb57c (author is Simon). >> >> >> >> * Have VACUUM VERBOSE report the number of skipped frozen pages >> >> (Masahiko Sawada) >> >> >> >> Could we add this item to the release note? >> > >> > OK, adjusted text below and commit added above this item: >> > >> > Have <link linkend="SQL-VACUUM"><command>VACUUM VERBOSE</></> report >> > the number of skipped frozen pages and oldest xmin (Masahiko Sawada) >> > >> >> Thank you! But actually main author of this item is Simon, I didn't >> involved in it. Maybe we can have it as a separate item. > > I just added Simon's name to the same item. > Thank you! Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
On Mon, May 8, 2017 at 10:50 PM, Bruce Momjian <bruce@momjian.us> wrote: > [other stuff] Bruce, the release notes do not mention yet that support for cleartext passwords is removed. This has been done recently by Heikki in eb61136d. This has as consequence that CREATE ROLE PASSWORD UNENCRYPTED returns an error, and that password_encryption loses its value 'off' compared to last releases. -- Michael
Michael Paquier <michael.paquier@gmail.com> writes:
> Bruce, the release notes do not mention yet that support for cleartext
> passwords is removed. This has been done recently by Heikki in
> eb61136d. This has as consequence that CREATE ROLE PASSWORD
> UNENCRYPTED returns an error, and that password_encryption loses its
> value 'off' compared to last releases.
The release notes only say that they're current through 4-22. The
usual process is to update them in batches every so often. It'd be
great if Bruce has time to do another round before Monday, but the
current situation is not really broken.
regards, tom lane
On Thu, May 11, 2017 at 11:50:03PM -0400, Tom Lane wrote: > Michael Paquier <michael.paquier@gmail.com> writes: > > Bruce, the release notes do not mention yet that support for cleartext > > passwords is removed. This has been done recently by Heikki in > > eb61136d. This has as consequence that CREATE ROLE PASSWORD > > UNENCRYPTED returns an error, and that password_encryption loses its > > value 'off' compared to last releases. > > The release notes only say that they're current through 4-22. The > usual process is to update them in batches every so often. It'd be > great if Bruce has time to do another round before Monday, but the > current situation is not really broken. Done. Applied patch attached. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Attachment
2017-05-04 22:26 GMT-03:00 Andres Freund <andres@anarazel.de>:
At least I was a bit more optimistic that we'd get a decent json plugin
into care soon-ish. While there was some initial work towards that, it
unfortunately stalled at some point.
That was my initial idea but there wasn't some interest at that time. wal2json is used in some replication solutions but also for audit and change capture solution. I recently received some patches/suggestions from Daniele Varrazzo [1].
Euler, are you perchance interested in working towards that? ;)
Sure. I can submit it for the next CF.
[1] https://www.postgresql.org/message-id/CA%2Bmi_8bJ_uPr67j-6mbin537DVvfk%3DbOhmWneyBRfbZu89q0tw%40mail.gmail.com
--
Euler Taveira Timbira - http://www.timbira.com.br/
PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento
PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento
On 4/24/17 8:31 PM, Bruce Momjian wrote: > I have committed the first draft of the Postgres 10 release notes. They > are current as of two days ago, and I will keep them current. Please > give me any feedback you have. > > The only unusual thing is that this release has ~180 items while most > recent release have had ~220. The pattern I see that there are more > large features in this release than previous ones. Can you change the attribution on Allow PL/Tcl functions to return composite types and sets to Karl Lehenbauer? He actually wrote the original patch; I just helped to get it through the community (something that FlightAware paid for). I didn't realize at the time that you could change the listed Author in the commitfest. -- Jim C. Nasby, Data Architect jim@nasby.net 512.569.9461 (cell) http://jim.nasby.net
On Mon, May 15, 2017 at 12:45 PM, Bruce Momjian <bruce@momjian.us> wrote:
On Thu, May 11, 2017 at 11:50:03PM -0400, Tom Lane wrote:
> Michael Paquier <michael.paquier@gmail.com> writes:
> > Bruce, the release notes do not mention yet that support for cleartext
> > passwords is removed. This has been done recently by Heikki in
> > eb61136d. This has as consequence that CREATE ROLE PASSWORD
> > UNENCRYPTED returns an error, and that password_encryption loses its
> > value 'off' compared to last releases.
>
> The release notes only say that they're current through 4-22. The
> usual process is to update them in batches every so often. It'd be
> great if Bruce has time to do another round before Monday, but the
> current situation is not really broken.
Done. Applied patch attached.
The patch introduced one release note item twice in https://www.postgresql.org/docs/10/static/release-10.html :
Rename WAL-related functions and views to use
lsninstead oflocation(David Rowley)RenameWAL-related functions and views to use
lsninstead oflocation(David Rowley)
Perhaps just one entry is good.
Regards,
Neha
On Wed, Jun 7, 2017 at 03:18:49PM +1000, Neha Khatri wrote: > > On Mon, May 15, 2017 at 12:45 PM, Bruce Momjian <bruce@momjian.us> wrote: > > On Thu, May 11, 2017 at 11:50:03PM -0400, Tom Lane wrote: > > Michael Paquier <michael.paquier@gmail.com> writes: > > > Bruce, the release notes do not mention yet that support for cleartext > > > passwords is removed. This has been done recently by Heikki in > > > eb61136d. This has as consequence that CREATE ROLE PASSWORD > > > UNENCRYPTED returns an error, and that password_encryption loses its > > > value 'off' compared to last releases. > > > > The release notes only say that they're current through 4-22. The > > usual process is to update them in batches every so often. It'd be > > great if Bruce has time to do another round before Monday, but the > > current situation is not really broken. > > Done. Applied patch attached. > > > > The patch introduced one release note item twice in https://www.postgresql.org/ > docs/10/static/release-10.html : > > > • Rename WAL-related functions and views to use lsn instead of location > (David Rowley) > > • RenameWAL-related functions and views to use lsn instead of location (David > Rowley) > > Perhaps just one entry is good. Yes, this has been fixed already, see: https://www.postgresql.org/docs/devel/static/release-10.html -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Hello hackers, From: Peter Geoghegan <pg@bowt.ie> > Date: Wed, 5 Jul 2017 15:19:57 -0700 > Subject: Re: [BUGS] BUG #14722: Segfault in tuplesort_heap_siftup, 32 bit overflow > On pgsql-bugs@postgresql.org On 07/06/2017 12:19 AM, Peter Geoghegan wrote: > In Postgres 10, tuplesort external sort run merging became much faster > following commit 24598337c8d. It might be noticeable if such a machine > were using Postgres 10 [...] Should-we mention this improvement in release notes? Regards, -- Adrien NAYRAT http://dalibo.com - http://dalibo.org
On 07/13/2017 04:36 PM, Adrien Nayrat wrote: > Hello hackers, > > From: Peter Geoghegan <pg@bowt.ie> >> Date: Wed, 5 Jul 2017 15:19:57 -0700 >> Subject: Re: [BUGS] BUG #14722: Segfault in tuplesort_heap_siftup, 32 bit overflow >> On pgsql-bugs@postgresql.org > > On 07/06/2017 12:19 AM, Peter Geoghegan wrote: >> In Postgres 10, tuplesort external sort run merging became much faster >> following commit 24598337c8d. It might be noticeable if such a machine >> were using Postgres 10 [...] > > Should-we mention this improvement in release notes? > > Regards, > Patch attached. Magic sorting is not my cup of tea, so I only put "Improve external sort". Maybe we should add an explanation. Regards, -- Adrien NAYRAT http://dalibo.com - http://dalibo.org
Attachment
On Tue, Apr 25, 2017 at 1:31 PM, Bruce Momjian <bruce@momjian.us> wrote: > I have committed the first draft of the Postgres 10 release notes. They > are current as of two days ago, and I will keep them current. Please > give me any feedback you have. Hi Bruce, "Add AFTER trigger transition tables to record changed rows (Kevin Grittner)" Any chance I could ask for a secondary author credit here? While I started out as a reviewer and I understand that we don't list those, I finished up writing quite a lot of lines of committed code for this (see commits 1add0b15, 8c55244a, c46c0e52, 501ed02c, f32d57fd, 9e6104c6, 29fd3d9d, 304007d9, 5ebeb579) and was already listed as a co-author by Kevin in the original commits (59702716, 18ce3a4a). Of course I wish I'd identified and fixed all of those things *before* the original commits, but that's how it played out... -- Thomas Munro http://www.enterprisedb.com
Hi all, On Tue, Aug 1, 2017 at 5:53 AM, Thomas Munro <thomas.munro@enterprisedb.com> wrote: > On Tue, Apr 25, 2017 at 1:31 PM, Bruce Momjian <bruce@momjian.us> wrote: >> I have committed the first draft of the Postgres 10 release notes. They >> are current as of two days ago, and I will keep them current. Please >> give me any feedback you have. > > Hi Bruce, > > "Add AFTER trigger transition tables to record changed rows (Kevin Grittner)" > > Any chance I could ask for a secondary author credit here? While I > started out as a reviewer and I understand that we don't list those, I > finished up writing quite a lot of lines of committed code for this > (see commits 1add0b15, 8c55244a, c46c0e52, 501ed02c, f32d57fd, > 9e6104c6, 29fd3d9d, 304007d9, 5ebeb579) and was already listed as a > co-author by Kevin in the original commits (59702716, 18ce3a4a). Of > course I wish I'd identified and fixed all of those things *before* > the original commits, but that's how it played out... > It might be too late but I found that the following entry is categorized in Monitoring. But in PostgreSQL 9.6 release note, the feature related to default role is categorized in Permissions Management. I think the adding new default roles can be categorized in the same category to not confuse users and personally it's more suitable. "Add default monitoring roles (Dave Page)" Thought? Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
On 07/13/2017 04:36 PM, Adrien Nayrat wrote: > Hello hackers, > > From: Peter Geoghegan <pg@bowt.ie> >> Date: Wed, 5 Jul 2017 15:19:57 -0700 >> Subject: Re: [BUGS] BUG #14722: Segfault in tuplesort_heap_siftup, 32 bit overflow >> On pgsql-bugs@postgresql.org > > On 07/06/2017 12:19 AM, Peter Geoghegan wrote: >> In Postgres 10, tuplesort external sort run merging became much faster >> following commit 24598337c8d. It might be noticeable if such a machine >> were using Postgres 10 [...] > > Should-we mention this improvement in release notes? > > Regards, > Hello, After seeing theses slides (especially 52) : https://speakerdeck.com/peterg/sort-hash-pgconfus-2017 I noticed several commits which improves performance of hash tables. Commit's messages mentions performance improvements for bitmap scans, hash aggregation and (according to Peter Geoghegan's conference) hash join : commit b30d3ea824c5ccba43d3e942704f20686e7dbab8 Author: Andres Freund <andres@anarazel.de> Date: Fri Oct 14 16:05:30 2016 -0700 Add a macro templatized hashtable. [...] In queries where these use up a large fraction of the time, this has been measured to lead to performance improvementsof over 100%. commit 75ae538bc3168bf44475240d4e0487ee2f3bb376 Author: Andres Freund <andres@anarazel.de> Date: Fri Oct 14 16:05:30 2016 -0700 Use more efficient hashtable for tidbitmap.c to speed up bitmap scans. [...] For bitmap scan heavy queries speedups of over 100% have been measured. commit 5dfc198146b49ce7ecc8a1fc9d5e171fb75f6ba5 Author: Andres Freund <andres@anarazel.de> Date: Fri Oct 14 17:22:51 2016 -0700 Use more efficient hashtable for execGrouping.c to speed up hash aggregation. [...] Improvements of over 120% have been measured. Should we mention it ? Regards, -- Adrien NAYRAT http://dalibo.com - http://dalibo.org
On Fri, Jun 2, 2017 at 04:05:44PM -0500, Jim Nasby wrote: > On 4/24/17 8:31 PM, Bruce Momjian wrote: > >I have committed the first draft of the Postgres 10 release notes. They > >are current as of two days ago, and I will keep them current. Please > >give me any feedback you have. > > > >The only unusual thing is that this release has ~180 items while most > >recent release have had ~220. The pattern I see that there are more > >large features in this release than previous ones. > > Can you change the attribution on > > Allow PL/Tcl functions to return composite types and sets > > to Karl Lehenbauer? He actually wrote the original patch; I just helped to > get it through the community (something that FlightAware paid for). I didn't > realize at the time that you could change the listed Author in the > commitfest. Done and backpatched. Sorry for the delay. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Bruce Momjian <bruce@momjian.us> writes:
> On Fri, Jun 2, 2017 at 04:05:44PM -0500, Jim Nasby wrote:
>> Can you change the attribution on
>> Allow PL/Tcl functions to return composite types and sets
>> to Karl Lehenbauer?
> Done and backpatched. Sorry for the delay.
I don't see this pushed to the repo?
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Sep 11, 2017 at 06:30:58PM -0400, Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > On Fri, Jun 2, 2017 at 04:05:44PM -0500, Jim Nasby wrote: > >> Can you change the attribution on > >> Allow PL/Tcl functions to return composite types and sets > >> to Karl Lehenbauer? > > > Done and backpatched. Sorry for the delay. > > I don't see this pushed to the repo? Sorry, pushed now. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On Sat, Sep 9, 2017 at 12:39:43PM +0200, Adrien Nayrat wrote: > On 07/13/2017 04:36 PM, Adrien Nayrat wrote: > > Hello hackers, > > > > From: Peter Geoghegan <pg@bowt.ie> > >> Date: Wed, 5 Jul 2017 15:19:57 -0700 > >> Subject: Re: [BUGS] BUG #14722: Segfault in tuplesort_heap_siftup, 32 bit overflow > >> On pgsql-bugs@postgresql.org > > > > On 07/06/2017 12:19 AM, Peter Geoghegan wrote: > >> In Postgres 10, tuplesort external sort run merging became much faster > >> following commit 24598337c8d. It might be noticeable if such a machine > >> were using Postgres 10 [...] > > > > Should-we mention this improvement in release notes? > > > > Regards, > > > > Hello, > > After seeing theses slides (especially 52) : > https://speakerdeck.com/peterg/sort-hash-pgconfus-2017 ... > > > > Should we mention it ? I don't know, but I suggest you read this email thread from April to get an idea of how performance items are handled: https://www.postgresql.org/message-id/flat/20170425013144.GA7513%40momjian.us#20170425013144.GA7513@momjian.us -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Aug 1, 2017 at 08:53:51AM +1200, Thomas Munro wrote: > On Tue, Apr 25, 2017 at 1:31 PM, Bruce Momjian <bruce@momjian.us> wrote: > > I have committed the first draft of the Postgres 10 release notes. They > > are current as of two days ago, and I will keep them current. Please > > give me any feedback you have. > > Hi Bruce, > > "Add AFTER trigger transition tables to record changed rows (Kevin Grittner)" > > Any chance I could ask for a secondary author credit here? While I > started out as a reviewer and I understand that we don't list those, I > finished up writing quite a lot of lines of committed code for this > (see commits 1add0b15, 8c55244a, c46c0e52, 501ed02c, f32d57fd, > 9e6104c6, 29fd3d9d, 304007d9, 5ebeb579) and was already listed as a > co-author by Kevin in the original commits (59702716, 18ce3a4a). Of > course I wish I'd identified and fixed all of those things *before* > the original commits, but that's how it played out... Sure, done. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Sep 1, 2017 at 05:39:31PM +0900, Masahiko Sawada wrote: > Hi all, > > On Tue, Aug 1, 2017 at 5:53 AM, Thomas Munro > <thomas.munro@enterprisedb.com> wrote: > > On Tue, Apr 25, 2017 at 1:31 PM, Bruce Momjian <bruce@momjian.us> wrote: > >> I have committed the first draft of the Postgres 10 release notes. They > >> are current as of two days ago, and I will keep them current. Please > >> give me any feedback you have. > > > > Hi Bruce, > > > > "Add AFTER trigger transition tables to record changed rows (Kevin Grittner)" > > > > Any chance I could ask for a secondary author credit here? While I > > started out as a reviewer and I understand that we don't list those, I > > finished up writing quite a lot of lines of committed code for this > > (see commits 1add0b15, 8c55244a, c46c0e52, 501ed02c, f32d57fd, > > 9e6104c6, 29fd3d9d, 304007d9, 5ebeb579) and was already listed as a > > co-author by Kevin in the original commits (59702716, 18ce3a4a). Of > > course I wish I'd identified and fixed all of those things *before* > > the original commits, but that's how it played out... > > > > It might be too late but I found that the following entry is > categorized in Monitoring. But in PostgreSQL 9.6 release note, the > feature related to default role is categorized in Permissions > Management. I think the adding new default roles can be categorized in > the same category to not confuse users and personally it's more > suitable. > > "Add default monitoring roles (Dave Page)" We don't have a "Permissions Management" category in PG 10 and unless we have at least three items to add in there, I don't think it is worth adding it. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Sep 12, 2017 at 11:57 AM, Bruce Momjian <bruce@momjian.us> wrote: > On Tue, Aug 1, 2017 at 08:53:51AM +1200, Thomas Munro wrote: >> "Add AFTER trigger transition tables to record changed rows (Kevin Grittner)" >> >> Any chance I could ask for a secondary author credit here? > > Sure, done. Thanks. Can you take it out again? (Just kidding!) -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On 09/12/2017 01:50 AM, Bruce Momjian wrote: > I don't know, but I suggest you read this email thread from April to get > an idea of how performance items are handled: > > https://www.postgresql.org/message-id/flat/20170425013144.GA7513%40momjian.us#20170425013144.GA7513@momjian.us Oh, sorry, I missed your explanation : https://www.postgresql.org/message-id/20170425033742.GG7513%40momjian.us > I remember seeing those and those are normally details I do not put in > the release notes as there isn't a clear user experience change except > "Postgres is faster". Yeah, a bummer, and I can change my filter, but > it would require discussion. From an user experience, IMHO, it is a huge improvement. I thought it could be mentioned in "General Performance section". But I do not know your habits. If it was not mentioned in previous releases, I understand we do not mention in v10's releases notes. Regards, -- Adrien NAYRAT
It's embarrassing to ask about such a trivial thing, but I noticed the following line was missing in the latest release note,which was originally in Bruce's website: Remove documented restriction about using large shared buffers on Windows (Takayuki Tsunakawa) Is this intended? Regards Takayuki Tsunakawa -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Sep 14, 2017 at 03:13:50AM +0000, Tsunakawa, Takayuki wrote:
> It's embarrassing to ask about such a trivial thing, but I noticed
> the following line was missing in the latest release note, which was
> originally in Bruce's website:
>
> Remove documented restriction about using large shared buffers on
> Windows (Takayuki Tsunakawa)
>
> Is this intended?
I don't know. The original text was:
Remove documented restriction about using large shared buffers on <systemitem class="osname">Windows</>
(TakayukiTsunakawa)
and was removed in this commit:
commit 749eceff4a1f9740391b126c81af9fd4bf3b1eaaAuthor: Tom Lane <tgl@sss.pgh.pa.us>Date: Sun Jul 9 20:11:21 2017
-0400 Doc: desultory copy-editing for v10 release notes. Improve many item descriptions, improve markup, relocate
someitems that seemed to be in the wrong section.
I am sure Tom can explain his reasoning.
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
"'Bruce Momjian'" <bruce@momjian.us> writes:
> I am sure Tom can explain his reasoning.
We don't normally release-note documentation changes. If this
wasn't purely a documentation change, then I was probably in error
to decide it didn't need to be in the notes.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Sep 19, 2017 at 12:22:39PM -0400, Tom Lane wrote: > "'Bruce Momjian'" <bruce@momjian.us> writes: > > I am sure Tom can explain his reasoning. > > We don't normally release-note documentation changes. If this > wasn't purely a documentation change, then I was probably in error > to decide it didn't need to be in the notes. It was purely a documentation change, but it was a documented change in a long-standing and popular practice of not using too many shared buffers on Windows, so I thought it wise to add it. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
"'Bruce Momjian'" <bruce@momjian.us> writes:
> On Tue, Sep 19, 2017 at 12:22:39PM -0400, Tom Lane wrote:
>> We don't normally release-note documentation changes. If this
>> wasn't purely a documentation change, then I was probably in error
>> to decide it didn't need to be in the notes.
> It was purely a documentation change, but it was a documented change in a
> long-standing and popular practice of not using too many shared buffers
> on Windows, so I thought it wise to add it.
Well, if the intent of the note was to encourage people to raise
shared_buffers, it didn't do a very good job of that as written,
because I sure didn't understand it that way.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Sep 19, 2017 at 12:30:01PM -0400, Tom Lane wrote: > "'Bruce Momjian'" <bruce@momjian.us> writes: > > On Tue, Sep 19, 2017 at 12:22:39PM -0400, Tom Lane wrote: > >> We don't normally release-note documentation changes. If this > >> wasn't purely a documentation change, then I was probably in error > >> to decide it didn't need to be in the notes. > > > It was purely a documentation change, but it was a documented change in a > > long-standing and popular practice of not using too many shared buffers > > on Windows, so I thought it wise to add it. > > Well, if the intent of the note was to encourage people to raise > shared_buffers, it didn't do a very good job of that as written, > because I sure didn't understand it that way. Do you have any suggestions since it is not a code change that I can point to? My guess is that the limitation was removed years ago, but we only found out recently. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
"'Bruce Momjian'" <bruce@momjian.us> writes:
> On Tue, Sep 19, 2017 at 12:30:01PM -0400, Tom Lane wrote:
>> Well, if the intent of the note was to encourage people to raise
>> shared_buffers, it didn't do a very good job of that as written,
>> because I sure didn't understand it that way.
> Do you have any suggestions since it is not a code change that I can
> point to? My guess is that the limitation was removed years ago, but we
> only found out recently.
TBH, I think that's another reason for not release-noting it. There's
no concrete change to point to, and so it's hard to figure out what
to say. I'm not even very sure that we should be encouraging people
to change existing shared_buffers settings; the experiments that
Takayuki-san did were only on Windows 10, so we don't really know that
changing that would be a good idea on older Windows versions.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 09/19/2017 09:32 AM, 'Bruce Momjian' wrote: > On Tue, Sep 19, 2017 at 12:30:01PM -0400, Tom Lane wrote: >> "'Bruce Momjian'" <bruce@momjian.us> writes: >>> On Tue, Sep 19, 2017 at 12:22:39PM -0400, Tom Lane wrote: >>>> We don't normally release-note documentation changes. If this >>>> wasn't purely a documentation change, then I was probably in error >>>> to decide it didn't need to be in the notes. >> >>> It was purely a documentation change, but it was a documented change in a >>> long-standing and popular practice of not using too many shared buffers >>> on Windows, so I thought it wise to add it. >> >> Well, if the intent of the note was to encourage people to raise >> shared_buffers, it didn't do a very good job of that as written, >> because I sure didn't understand it that way. > > Do you have any suggestions since it is not a code change that I can > point to? My guess is that the limitation was removed years ago, but we > only found out recently. My guess is that the limitation was removed as of 9.3 with the work Haas did with shared buffers. Thus, yes it was years ago. I think that listing it regardless of the documentation change could be useful. Something like: """ Better support for large shared_buffers configurations including the Windows platform. Users are encouraged to review their shared_buffer settings against the size of their active data set and reconfigure appropriately. """ It is pretty much practitioner given that if your active data set can fit in shared_buffers and you aren't going to adversely affect the ability for the system to operate, that you should configure a high setting. I have seen settings as much as 96GB doing wonderfully in production. JD > -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL Centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://pgconf.us ***** Unless otherwise stated, opinions are my own. ***** -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: pgsql-hackers-owner@postgresql.org > [mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Tom Lane > TBH, I think that's another reason for not release-noting it. There's no > concrete change to point to, and so it's hard to figure out what to say. > I'm not even very sure that we should be encouraging people to change > existing shared_buffers settings; the experiments that Takayuki-san did > were only on Windows 10, so we don't really know that changing that would > be a good idea on older Windows versions. In fact, when I ran pgbench on Windows 2008 for some other unrelated reason, I found larger shared buffers (4GB, 8GB) waseffective, too. I didn't know pure documentation changes are not listed on the release notes. But I suggest listing them (of course, excludingmere typos), because the documentation is also important for users as well as programs. The documentation is alsopart of software product. If the documented behavior and the actual one differs and the user is wondering which is correct,he can know the answer only from the release note when the mismatch is fixed. I think the documentation changesare more useful for users than, for example, the following items: E.1.3.11. Source Code Improve behavior of pgindent (Piotr Stefaniak, Tom Lane) Allow WaitLatchOrSocket() to wait for socket connection on Windows (Andres Freund) Overhaul documentation build process (Alexander Lakhin) Use XSLT to build the PostgreSQL documentation (Peter Eisentraut) Regards Takayuki Tsunakawa -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers