Thread: PG 14 release notes, first draft
I have committed the first draft of the PG 14 release notes. You can
see the most current build of them here:
https://momjian.us/pgsql_docs/release-14.html
I need clarification on many items, and the document still needs its
items properly ordered, and markup added. I also expect a lot of
feedback.
I plan to work on completing this document this coming week in
preparation for beta next week.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Mon, 2021-05-10 at 02:03 -0400, Bruce Momjian wrote: > When using \e in psql, if the buffer is not modified by the editor, ignore the editor contents and leave the buffer unchanged(Laurenz Albe) > The \ef and \ev commands also now have this behavior. DOCS SAY BUFFER IS CLEARED. It's a bit more complicated: If you edit the current buffer with \e, the buffer is unchanged if you quit the editor. However, if you edit the previous statement, a file or the definition of a function or view, the query buffer is cleared if you quit the editor without saving. Suggested wording: When editing anything else than the current query buffer with \e, and you quit the editor, the query buffer is cleared. This makes the behavior less surprising and prevents the unintended re-execution of the previous statement. Yours, Laurenz Albe
On Mon, May 10, 2021 at 02:03:08AM -0400, Bruce Momjian wrote: > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > > I need clarification on many items, and the document still needs its > items properly ordered, and markup added. I also expect a lot of > feedback. There's a small typo: +Improve tab completion (Vignesh C,, Michael [...] (duplicated comma) Also +<para> +Avoid retrieval of CHECK constraints and DEFAULT exprs in data-only dump (Julien Rouhaud) +</para> + +<para> +IS THIS BACKWARD INCOMPATIBLE? +</para> +</listitem> The new behavior doesn't have any impact on the generated dump, as the modification is to avoid retrieving data that won't be used. For users, it only means maybe slight faster pg_dump execution, or slightly better change to be able to run a pg_dump --data-only if pg_constraint is corrupted but not the rest of the user data, so maybe it's not necessary to mention that in the release notes?
On Mon, May 10, 2021 at 3:03 PM Bruce Momjian <bruce@momjian.us> wrote: > > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html Thank you! > Add system view pg_stat_replication_slots to report replication slot activity (Sawada Masahiko, Amit Kapila) > > Function pg_stat_reset_replication_slot() resets slot statistics. THIS IS LOGICAL ONLY, BUT NO "LOGICAL" IN THE NAME? IIUC pg_stat_replication_slots view supports only logical slot for now. But we might have it show also physical slot in the future. I'm fine with the current view name and description but someone might want to use "logical replication slot" instead of just "replication slot". > IS "ACTIVITY" THE RIGHT WORD? The doc says "The pg_stat_replication_slots view will contain one row per logical replication slot, showing statistics about its usage.". So we can say "... to report replication slot statistics about its usage". Regards, -- Masahiko Sawada EDB: https://www.enterprisedb.com/
On Mon, 10 May 2021 at 18:03, Bruce Momjian <bruce@momjian.us> wrote: > I need clarification on many items, and the document still needs its > items properly ordered, and markup added. I also expect a lot of > feedback. Thanks for drafting that up. > Add executor method to cache results from the inner-side of joins (David Rowley) I think it would be more accurate to say "inner-side of nested loop joins". > Allow efficient retrieval of heap rows via tid (Edmund Horner, David Rowley) I'd say we already had that feature with TID Scan. Maybe it would be better to write: "Allow efficient heap scanning on ranges of tids (Edmund Horner, David Rowley)" > Improve the performance of parallel sequential scans (Thomas Munro, David Rowley) I think it is worth mentioning "I/O" before "performance". This change won't really help cases if all the table's pages are already in shared buffers. David
On Mon, 10 May 2021 at 08:03, Bruce Momjian <bruce@momjian.us> wrote: > > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > https://momjian.us/pgsql_docs/release-14.html > > I need clarification on many items, and the document still needs its > items properly ordered, and markup added. I also expect a lot of > feedback. I noticed that the improvement in bloat control in the HeapAM that I know of (3c3b8a4b, 0ff8bbde) weren't documented here. Although each can be considered minor, they together can decrease the bloating behaviour of certain workloads significantly (and limit the total damage), and in my opinion this should be mentioned. 3c3b8a4b: Returns space claimed for the line pointer array back to the page's empty space, so that it can also be used for tuple data. 0ff8bbde: Allows large tuples to be inserted on pages which have only a small amount of data, regardless of fillfactor. Together they should be able to help significantly in both bloat prevention and bloat reduction. > I plan to work on completing this document this coming week in > preparation for beta next week. Thanks! With regards, Matthias van de Meent
On 5/10/21 2:03 AM, Bruce Momjian wrote: > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > > I need clarification on many items, and the document still needs its > items properly ordered, and markup added. I also expect a lot of > feedback. > > I plan to work on completing this document this coming week in > preparation for beta next week. While only a small change, this commit does affect user visible behavior and so should probably be noted: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=b12bd4869b5e Joe -- Crunchy Data - http://crunchydata.com PostgreSQL Support for Secure Enterprises Consulting, Training, & Open Source Development
Hi, Bruce! On Mon, May 10, 2021 at 9:03 AM Bruce Momjian <bruce@momjian.us> wrote: > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > > I need clarification on many items, and the document still needs its > items properly ordered, and markup added. I also expect a lot of > feedback. > > I plan to work on completing this document this coming week in > preparation for beta next week. Thank you very much for your work! Let me provide a missing description for the items related to me. * Improve handling of compound words in to_tsquery() and websearch_to_tsquery() (Alexander Korotkov) Compound words are now transformed into parts connected with phrase search operators. For example, to_tsquery('pg_class') becomes 'pg <-> class' instead of 'pg & class'. This eliminates bug of handling compound words connected with the phrase operator and makes the search of compound words more strict. * Fix extra distance in phrase operators for quoted text in websearch_to_tsquery() (Alexander Korotkov) For example, websearch_to_tsquery('english', '"aaa: bbb"') becomes 'aaa <> bbb' instead of 'aaa <2> bbb'. Feel free to make stylistic and other corrections if needed. ------ Regards, Alexander Korotkov
2021年5月10日(月) 15:03 Bruce Momjian <bruce@momjian.us>: > > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > > I need clarification on many items, and the document still needs its > items properly ordered, and markup added. I also expect a lot of > feedback. > > I plan to work on completing this document this coming week in > preparation for beta next week. This misses the change of default value, and is a bit unclear: > Remove password_encryption's support for boolean values, e.g. true (Peter Eisentraut) > > Previous boolean values enabled md5. Now, only the md5 string does this. I'd suggest something along these lines: > The default for password_encryption is now "scram-sha-256" (Peter Eisentraut) > > The pseudo-boolean values "true", "on", "yes" and "1" are no longer accepted as an alias for "md5". (It hasn't been a true boolean setting since Pg 9.6). Regards Ian Barwick -- EnterpriseDB: https://www.enterprisedb.com
Thanks for putting it together.
I think these two should be merged:
| Remove containment operators @ and ~ from contrib modules cube, hstore, intarray, and seg (Justin Pryzby)
| Remove deprecated containment operators for built-in geometry data types (Justin Pryzby)
| Improve autovacuum's analyze of partitioned tables (Yuzuko Hosoya)
| DETAILS?
Should say: Autovacuum now analyzes partitioned tables.
| The server variable check_client_connection_interval allows supporting operating systems, e.g., Linux, to
automaticallycancel queries by disconnected clients.
The GUC is actually called client_connection_check_interval - the commit
message used the wrong name.
| This is particularly helpful for reducing index bloat on tables that frequently update indexed columns.
Does it mean "..where indexed columns are frequently updated"?
| Allow multiple foreign table scans to be run in parallel (Robert Haas, Kyotaro Horiguchi, Thomas Munro, Etsuro
Fujita)
I think it means multiple foreight table scan *nodes*
| If server variable compute_query_id is enabled, display the hash in pg_stat_activity, EXPLAIN VERBOSE, csvlog, and
optionallyin log_line_prefix (Julien Rouhaud)
I think needs details, like: "If disabled, then the hash might be computed by
an extension, instead".
Later, you say:
| Extension pg_stat_statements will need to enable hash computation via the compute_query_id server variable to
functionproperly. pg_stat_statements can now use a custom hash computation method.
Maybe it should say "will need hash computation to be enabled".
| Allow more than the common name (CN) to be matched for client certificate authentication (Andrew Dunstan)
Your description makes it sound like arbitrary attributes can be compared. But
the option just allows comparing CN or DN.
| Allow file system sync at the start of crash recovery on Linux (Thomas Munro)
I think this should describe the existing, default behavior:
Allow syncfs method to sync data directory during recovery;
The default behavior is to open and fsync every data file, and the new setting
recovery_init_sync_method=syncfs instead syncs each filesystem in the data
directory.
| Add date_bin function (John Naylor)
This truncate timestamps on an arbitrary interval.
Like date_trunc() but also supports eg. '15 minutes', and also uses an arbitrary "origin".
| Support negative indexes in split_part() (Nikhil Benesch)
| Negative values count from the last field going forward.
should say "start from the last field and count backward" ?
| Add configure option --with-openssl to behave like --with-ssl={openssl} (Daniel Gustafsson, Michael Paquier)
| The option --with-openssl is kept for compatibility.
I think this is backwards. The new option is with-ssl=openssl, and (as you
said) with-openssl is kept.
Should these be in the "compatibility" section?
| Force custom server variable names to match the pattern used for unquoted SQL identifiers (Tom Lane)
| Change password_encryption's default to scram-sha-256 (Peter Eisentraut)
| Change checkpoint_completion_target default to 0.9 (Stephen Frost)
| Reduce the default value of vacuum_cost_page_miss (Peter Geoghegan)
Nitpicks to follow:
| Allow some GiST index to be built by presorting the data (Andrey Borodin)
indexes
| with --with-lz4 support to enable this feature
I would change to say "to support" rather than "support to enable"
| Speed truncation of small tables on large shared buffer servers (Kirk Jamison)
"on servers with large settings of shared_buffers"
| Allow windowing functions to perform incremental sorts (David Rowley)
Just "window" functions
| Improve pg_stat_activity reporting for walsenders processes (Tom Lane)
walsender
| Previously these functions could only be executed by super-users, and still defaults do that.
..which is still the default behavior.
| This allows multiple queries to be send and only wait for completion when a specific synchronization message is sent.
be sent
| Enhance libpq libpq's target_session_attrs parameter options (Haribabu Kommi, Greg Nancarrow, Vignesh C, Tom Lane)
remove first "libpq"
| With the removal of the ! operator in this release, factorial() is the only built-in way to computer a factorial.
compute
| For example, GROUP BY CUBE (a,b), CUBE (b,c) will generated duplicate grouping combinations without DISTINCT.
will generate
| Allow VACUUM VERBOSE to report page deletion counts for each scan of an index (Peter Geoghegan)
I think "Allow" is wrong - should just say that VACUUM VERBOSE reports..
|By default, only the root of partitioned tables are imported.
*is* imported
Can these be merged:
Allow logical replication to stream long transactions to standbys (Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil
Sontakke)
Improve the logical replication API to allow streaming large in-progress transactions (Tomas Vondra, Dilip Kumar, Amit
Kapila)
Same as the last couple years, I checked for missing items in the release notes, running something like this. git log --cherry-pick --oneline origin/REL_13_STABLE...origin/master Should any of these be included? f82de5c46b Do COPY FROM encoding conversion/verification in larger chunks. 9e596b65f4 Add "LP_DEAD item?" column to GiST pageinspect functions 10a5b35a00 Report resource usage at the end of recovery 7e453634bb Add additional information in the vacuum error context. 1ea396362b Improve logging of bad parameter values in BIND messages. 86dc90056d Rework planning and execution of UPDATE and DELETE. a1115fa078 Postpone some more stuff out of ExecInitModifyTable. c5b7ba4e67 Postpone some stuff out of ExecInitModifyTable. 7db0cd2145 Set PD_ALL_VISIBLE and visibility map bits in COPY FREEZE 01e658fa74 Hash support for row types a929e17e5a Allow run-time pruning on nested Append/MergeAppend nodes 8c15a29745 Allow ALTER TYPE to update an existing type's typsubscript value. c7aba7c14e Support subscripting of arbitrary types, not only arrays. 7b94e99960 Remove catalog function currtid() 926fa801ac Remove undocumented IS [NOT] OF syntax. cd9c1b3e19 Rename PGPROC->vacuumFlags to statusFlags a04daa97a4 Remove es_result_relation_info from EState. 3d351d916b Redefine pg_class.reltuples to be -1 before the first VACUUM or ANALYZE. fea10a6434 Rename VariableCacheData.nextFullXid to nextXid. 9de9294b0c Stop archive recovery if WAL generated with wal_level=minimal is found. (see also 15251c0a6) f40c6969d0 Routine usage information schema tables b4af70cb21 Simplify state managed by VACUUM. 4753ef37e0 Use a WaitLatch for vacuum/autovacuum sleeping 9dd963ae25 Recycle nbtree pages deleted during same VACUUM. 3c3b8a4b26 Truncate line pointer array during VACUUM. ad1c36b070 Fix foreign-key selectivity estimation in the presence of constants.
On Sun, May 9, 2021 at 11:03 PM Bruce Momjian <bruce@momjian.us> wrote: > I have committed the first draft of the PG 14 release notes. This definitely isn't necessary, since the commit in question was a totally mechanical thing that cleaned up a minor inconsistency: Initialize work_mem and maintenance_work_mem using current guc.c default (Peter Geoghegan) Oversight in commit 848ae330a49, which increased the previous defaults for work_mem and maintenance_work_mem by 4X. IS THIS A BEHAVIORAL CHANGE? -- Peter Geoghegan
On Mon, May 10, 2021 at 4:44 AM Matthias van de Meent <boekewurm+postgres@gmail.com> wrote: > I noticed that the improvement in bloat control in the HeapAM that I > know of (3c3b8a4b, 0ff8bbde) weren't documented here. Although each > can be considered minor, they together can decrease the bloating > behaviour of certain workloads significantly (and limit the total > damage), and in my opinion this should be mentioned. > > 3c3b8a4b: Returns space claimed for the line pointer array back to the > page's empty space, so that it can also be used for tuple data. > > 0ff8bbde: Allows large tuples to be inserted on pages which have only > a small amount of data, regardless of fillfactor. +1 on mentioning both things. -- Peter Geoghegan
On Mon, May 10, 2021 at 7:00 AM Justin Pryzby <pryzby@telsasoft.com> wrote: > | Allow VACUUM VERBOSE to report page deletion counts for each scan of an index (Peter Geoghegan) > > I think "Allow" is wrong - should just say that VACUUM VERBOSE reports.. It's also not accurate, since the count of deleted pages was always shown by VACUUM VERBOSE (once per index scan). The new feature has us show pages deleted by the VACUUM that actually ran (not some earlier VACUUM) -- these are "newly deleted pages". I don't think that this item is worth mentioning, though -- it's just a nice to have. If Bruce removes it from the release notes entirely I won't object. In addition to the items that I commented on in my response to Matthias just now, I should point out the following item as worthy of inclusion: 9dd963ae25 Recycle nbtree pages deleted during same VACUUM. I suggest that this item be phrased more or less as follows: "Allow VACUUM to eagerly place newly deleted B-Tree pages in the Free Space Map. Previously VACUUM could only place preexisting deleted pages in the Free Space Map for recycling." -- Peter Geoghegan
On Mon, May 10, 2021 at 08:16:16AM +0200, Laurenz Albe wrote:
> On Mon, 2021-05-10 at 02:03 -0400, Bruce Momjian wrote:
> > When using \e in psql, if the buffer is not modified by the editor, ignore the editor contents and leave the buffer
unchanged(Laurenz Albe)
> > The \ef and \ev commands also now have this behavior. DOCS SAY BUFFER IS CLEARED.
>
> It's a bit more complicated: If you edit the current buffer with \e, the buffer is
> unchanged if you quit the editor.
> However, if you edit the previous statement, a file or the definition of a function
> or view, the query buffer is cleared if you quit the editor without saving.
>
> Suggested wording:
>
> When editing anything else than the current query buffer with \e, and you quit
> the editor, the query buffer is cleared. This makes the behavior less surprising
> and prevents the unintended re-execution of the previous statement.
OK, I figured it out. I was confused by \p because \? says:
test=> \?
Query Buffer
\e [FILE] [LINE] edit the query buffer (or file) with external editor
\ef [FUNCNAME [LINE]] edit function definition with external editor
\ev [VIEWNAME [LINE]] edit view definition with external editor
--> \p show the contents of the query buffer
\r reset (clear) the query buffer
...
but the documentaton says:
\p or \print
Print the current query buffer to the standard output. If
--> the current query buffer is empty, the most recently executed
--> query is printed instead.
I wasn't aware that \e loads the previous query if the buffer is empty.
I came up with this release note text:
<listitem>
<!--
Author: Tom Lane <tgl@sss.pgh.pa.us>
2021-04-03 [55873a00e] Improve psql's behavior when the editor is exited withou
-->
<para>
When editing the previous query or a file with psql's \e, ignore the
contents if the editor exits without saving (Laurenz Albe)
</para>
<para>
Previously, editing the previous query or a file and not saving the
editor contents would still execute the editor contents. The \ef and
\ev commands also now have this behavior.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 02:51:28PM +0800, Julien Rouhaud wrote: > On Mon, May 10, 2021 at 02:03:08AM -0400, Bruce Momjian wrote: > > I have committed the first draft of the PG 14 release notes. You can > > see the most current build of them here: > > > > https://momjian.us/pgsql_docs/release-14.html > > > > I need clarification on many items, and the document still needs its > > items properly ordered, and markup added. I also expect a lot of > > feedback. > > There's a small typo: > > +Improve tab completion (Vignesh C,, Michael [...] > > (duplicated comma) Fixed. > Also > > +<para> > +Avoid retrieval of CHECK constraints and DEFAULT exprs in data-only dump (Julien Rouhaud) > +</para> > + > +<para> > +IS THIS BACKWARD INCOMPATIBLE? > +</para> > +</listitem> > > The new behavior doesn't have any impact on the generated dump, as the > modification is to avoid retrieving data that won't be used. > > For users, it only means maybe slight faster pg_dump execution, or slightly > better change to be able to run a pg_dump --data-only if pg_constraint is > corrupted but not the rest of the user data, so maybe it's not necessary to > mention that in the release notes? Thanks, removed. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 05:28:24PM +0900, Masahiko Sawada wrote: > On Mon, May 10, 2021 at 3:03 PM Bruce Momjian <bruce@momjian.us> wrote: > > > > I have committed the first draft of the PG 14 release notes. You can > > see the most current build of them here: > > > > https://momjian.us/pgsql_docs/release-14.html > > Thank you! > > > Add system view pg_stat_replication_slots to report replication slot activity (Sawada Masahiko, Amit Kapila) > > > > Function pg_stat_reset_replication_slot() resets slot statistics. THIS IS LOGICAL ONLY, BUT NO "LOGICAL" IN THE NAME? > > IIUC pg_stat_replication_slots view supports only logical slot for > now. But we might have it show also physical slot in the future. I'm > fine with the current view name and description but someone might want > to use "logical replication slot" instead of just "replication slot". OK, I was just confirming we are happy with the name. > > > IS "ACTIVITY" THE RIGHT WORD? > > The doc says "The pg_stat_replication_slots view will contain one row > per logical replication slot, showing statistics about its usage.". So > we can say "... to report replication slot statistics about its > usage". OK, I think I prefer "activity" so will just keep that. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 08:52:44PM +1200, David Rowley wrote:
> On Mon, 10 May 2021 at 18:03, Bruce Momjian <bruce@momjian.us> wrote:
> > I need clarification on many items, and the document still needs its
> > items properly ordered, and markup added. I also expect a lot of
> > feedback.
>
> Thanks for drafting that up.
>
> > Add executor method to cache results from the inner-side of joins (David Rowley)
>
> I think it would be more accurate to say "inner-side of nested loop joins".
OK, thanks. I suspected that was true.
> > Allow efficient retrieval of heap rows via tid (Edmund Horner, David Rowley)
>
> I'd say we already had that feature with TID Scan. Maybe it would be
> better to write:
>
> "Allow efficient heap scanning on ranges of tids (Edmund Horner, David Rowley)"
I went with:
Allow efficient heap scanning of a range of tids (Edmund Horner,
David Rowley)
> > Improve the performance of parallel sequential scans (Thomas Munro, David Rowley)
>
> I think it is worth mentioning "I/O" before "performance". This
> change won't really help cases if all the table's pages are already in
> shared buffers.
I went with:
Improve the performance of parallel sequential I/O scans (Thomas Munro,
David Rowley)
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 01:44:12PM +0200, Matthias van de Meent wrote: > On Mon, 10 May 2021 at 08:03, Bruce Momjian <bruce@momjian.us> wrote: > > > > I have committed the first draft of the PG 14 release notes. You can > > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > > > > I need clarification on many items, and the document still needs its > > items properly ordered, and markup added. I also expect a lot of > > feedback. > > I noticed that the improvement in bloat control in the HeapAM that I > know of (3c3b8a4b, 0ff8bbde) weren't documented here. Although each > can be considered minor, they together can decrease the bloating > behaviour of certain workloads significantly (and limit the total > damage), and in my opinion this should be mentioned. > > 3c3b8a4b: Returns space claimed for the line pointer array back to the > page's empty space, so that it can also be used for tuple data. > > 0ff8bbde: Allows large tuples to be inserted on pages which have only > a small amount of data, regardless of fillfactor. > > Together they should be able to help significantly in both bloat > prevention and bloat reduction. I looked at those items. I try to mention performance items that enable new workloads or require some user action to benefit from it. I am not sure these two qualify, but can others comments? Thanks. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Mon, 2021-05-10 at 12:38 -0400, Bruce Momjian wrote:
> I came up with this release note text:
>
> <listitem>
> <!--
> Author: Tom Lane <tgl@sss.pgh.pa.us>
> 2021-04-03 [55873a00e] Improve psql's behavior when the editor is exited withou
> -->
>
> <para>
> When editing the previous query or a file with psql's \e, ignore the
> contents if the editor exits without saving (Laurenz Albe)
> </para>
>
> <para>
> Previously, editing the previous query or a file and not saving the
> editor contents would still execute the editor contents. The \ef and
> \ev commands also now have this behavior.
> </para>
> </listitem>
Thanks, that looks much better.
The second paragraph starts describing the previous behavior, but the second
sentence details on the changes. Perhaps it would be better to put that into
the first paragraph:
<para>
When editing the previous query or a file with psql's \e, or when a
view or function definition are edited with \ev or \ef, ignore the
contents if the editor exits without saving (Laurenz Albe)
</para>
<para>
Previously, editing the previous query or a file and not saving the
editor contents would still execute the editor contents.
</para>
Yours,
Laurenz Albe
On Mon, May 10, 2021 at 07:39:17PM +0200, Laurenz Albe wrote:
> On Mon, 2021-05-10 at 12:38 -0400, Bruce Momjian wrote:
> > I came up with this release note text:
> >
> > <listitem>
> > <!--
> > Author: Tom Lane <tgl@sss.pgh.pa.us>
> > 2021-04-03 [55873a00e] Improve psql's behavior when the editor is exited withou
> > -->
> >
> > <para>
> > When editing the previous query or a file with psql's \e, ignore the
> > contents if the editor exits without saving (Laurenz Albe)
> > </para>
> >
> > <para>
> > Previously, editing the previous query or a file and not saving the
> > editor contents would still execute the editor contents. The \ef and
> > \ev commands also now have this behavior.
> > </para>
> > </listitem>
>
> Thanks, that looks much better.
>
> The second paragraph starts describing the previous behavior, but the second
> sentence details on the changes. Perhaps it would be better to put that into
> the first paragraph:
>
> <para>
> When editing the previous query or a file with psql's \e, or when a
> view or function definition are edited with \ev or \ef, ignore the
> contents if the editor exits without saving (Laurenz Albe)
> </para>
>
> <para>
> Previously, editing the previous query or a file and not saving the
> editor contents would still execute the editor contents.
> </para>
Uh, I try to keep the first sentence short so people can scan it more
easily, so I am hesitant to make this change. I went with this change:
<listitem>
<!--
Author: Tom Lane <tgl@sss.pgh.pa.us>
2021-04-03 [55873a00e] Improve psql's behavior when the editor is exited withou
-->
<para>
When editing the previous query or a file with psql's \e, or using \ef and \ev, ignore the contents if the editor
exitswithout saving (Laurenz Albe)
</para>
<para>
Previously, such edits would still execute the editor contents.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
Hi Bruce,
Thanks for doing this work again!
> Add date_bin function (John Naylor)
>
> WHAT DOES THIS DO?
Hard to describe in a one-liner, but it lines up timestamps into regular intervals as specified by the user. It is more clear after seeing examples:
https://www.postgresql.org/docs/devel/functions-datetime.html#FUNCTIONS-DATETIME-BIN
> Dramatically improve Unicode normalization (John Naylor)
>
> WHAT OPERATIONS USE THIS?
PG13 added the normalize() function to normalize Unicode sequences, as well as the IS NORMALIZED syntax to test for that. The commits* here do not change behavior and only improve performance. As such, this really belongs in the performance section.
Thanks for doing this work again!
> Add date_bin function (John Naylor)
>
> WHAT DOES THIS DO?
Hard to describe in a one-liner, but it lines up timestamps into regular intervals as specified by the user. It is more clear after seeing examples:
https://www.postgresql.org/docs/devel/functions-datetime.html#FUNCTIONS-DATETIME-BIN
> Dramatically improve Unicode normalization (John Naylor)
>
> WHAT OPERATIONS USE THIS?
PG13 added the normalize() function to normalize Unicode sequences, as well as the IS NORMALIZED syntax to test for that. The commits* here do not change behavior and only improve performance. As such, this really belongs in the performance section.
*There is one additional commit that belongs to this entry:
Author: Michael Paquier <michael@paquier.xyz>
2020-10-11 [80f8eb79e] Use perfect hash for NFC and NFKC Unicode Normalization quick check
2020-10-11 [80f8eb79e] Use perfect hash for NFC and NFKC Unicode Normalization quick check
--
On Mon, 10 May 2021 at 19:34, Bruce Momjian <bruce@momjian.us> wrote: > > On Mon, May 10, 2021 at 01:44:12PM +0200, Matthias van de Meent wrote: > > On Mon, 10 May 2021 at 08:03, Bruce Momjian <bruce@momjian.us> wrote: > > > > > > I have committed the first draft of the PG 14 release notes. You can > > > see the most current build of them here: > > > https://momjian.us/pgsql_docs/release-14.html > > > > > > I need clarification on many items, and the document still needs its > > > items properly ordered, and markup added. I also expect a lot of > > > feedback. > > > > I noticed that the improvement in bloat control in the HeapAM that I > > know of (3c3b8a4b, 0ff8bbde) weren't documented here. Although each > > can be considered minor, they together can decrease the bloating > > behaviour of certain workloads significantly (and limit the total > > damage), and in my opinion this should be mentioned. > > > > 3c3b8a4b: Returns space claimed for the line pointer array back to the > > page's empty space, so that it can also be used for tuple data. > > > > 0ff8bbde: Allows large tuples to be inserted on pages which have only > > a small amount of data, regardless of fillfactor. > > > > Together they should be able to help significantly in both bloat > > prevention and bloat reduction. > > I looked at those items. I try to mention performance items that enable > new workloads or require some user action to benefit from it. 0ff8bbde Enables a workload that inserts (and non-locally updates) large (> FILLFACTOR %) tuples in tables that have a low FILLFACTOR. Previously this would fail dramatically by only inserting on new pages; this would extend the table indefinately. See the thread [0] 3c3b8a4b improves workloads with high local update-then-delete churn. Previously this would irreversably claim space on the page for tuple identifiers even when they were later deleted; now we can reclaim this space when a tuple is deleted from the page. I see these two improvements in a similar light as the bottom-up index deletion in btree: No user action required, works out-of-the-box, decreases bloat / disk usage, but good to note as it fixes (known) bloating footguns that a user might have encountered. > I am not sure these two qualify, but can others comments? Thanks. I'd like to refer to Peter Geoghegan's reply [1] upthread. Thank you for your effort, Matthias van de Meent [0] https://www.postgresql.org/message-id/flat/6e263217180649339720afe2176c50aa%40opammb0562.comp.optiver.com [1] https://www.postgresql.org/message-id/CAH2-Wz%3D-A%3DjRxpB2Owj3KQadCue7%2BNLqj56Q566ees7TapMRvA%40mail.gmail.com
On Mon, May 10, 2021 at 07:54:24AM -0700, Peter Geoghegan wrote:
> On Mon, May 10, 2021 at 4:44 AM Matthias van de Meent
> <boekewurm+postgres@gmail.com> wrote:
> > I noticed that the improvement in bloat control in the HeapAM that I
> > know of (3c3b8a4b, 0ff8bbde) weren't documented here. Although each
> > can be considered minor, they together can decrease the bloating
> > behaviour of certain workloads significantly (and limit the total
> > damage), and in my opinion this should be mentioned.
> >
> > 3c3b8a4b: Returns space claimed for the line pointer array back to the
> > page's empty space, so that it can also be used for tuple data.
> >
> > 0ff8bbde: Allows large tuples to be inserted on pages which have only
> > a small amount of data, regardless of fillfactor.
>
> +1 on mentioning both things.
OK, you are confirming what Matthias suggested. I added these two
items, which both seem to apply only to heap pages, not index pages:
---------------------------------------------------------------------------
<listitem>
<!--
Author: Peter Geoghegan <pg@bowt.ie>
2021-04-07 [3c3b8a4b2] Truncate line pointer array during VACUUM.
-->
<para>
Deallocate space reserved by trailing unused heap line pointers
(Matthias van de Meent, Peter Geoghegan)
</para>
</listitem>
---------------------------------------------------------------------------
<listitem>
<!--
Author: Noah Misch <noah@leadboat.com>
2021-03-30 [0ff8bbdee] Accept slightly-filled pages for tuples larger
than fill
-->
<para>
Allow wide tuples to be always added to almost-empty heap pages (John Naylor,
Floris van Nee)
</para>
<para>
Previously tuples whose insertion would have exceeded the page's fill
factor were instead added to new pages.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 3:58 PM Bruce Momjian <bruce@momjian.us> wrote: > OK, you are confirming what Matthias suggested. I added these two > items, which both seem to apply only to heap pages, not index pages: That's right -- these two relate to heap pages only. I think that Matthias compared these two to bottom-up index deletion because all three patches are concerned about avoiding "a permanent solution to a temporary problem". They're conceptually similar despite being in fairly different areas. Evidently Matthias has a similar mental model to my own when it comes to this stuff. Unfortunately the practical significance of the line pointer patch is hard to demonstrate with a benchmark. I believe that it is very useful on a sufficiently long timeline and with certain workloads because of the behavior it avoids. As I pointed out on that other thread recently, once you have irreversible bloat very small adverse events will eventually add up and cause big problems. When this happens it'll be very hard or impossible to detect, since it just looks like heap fragmentation. That said, it's clearly an issue with one of the TPC-C tables if you run BenchmarkSQL for days and days (just one table, though). So there is hard evidence that line pointer bloat could get really out of hand in at least some tables. -- Peter Geoghegan
Thanks for making the updates. On Tue, 11 May 2021 at 05:07, Bruce Momjian <bruce@momjian.us> wrote: > > On Mon, May 10, 2021 at 08:52:44PM +1200, David Rowley wrote: > > > Improve the performance of parallel sequential scans (Thomas Munro, David Rowley) > > > > I think it is worth mentioning "I/O" before "performance". This > > change won't really help cases if all the table's pages are already in > > shared buffers. > > I went with: > > Improve the performance of parallel sequential I/O scans (Thomas Munro, > David Rowley) I think I'd have gone with: "Improve I/O performance of parallel sequential scans (Thomas Munro, David Rowley)" The operation we're speeding up is called sequential scan. We don't have any operation that's named sequential I/O scan. David
On Mon, May 10, 2021 at 07:50:14AM -0400, Joe Conway wrote: > On 5/10/21 2:03 AM, Bruce Momjian wrote: > > I have committed the first draft of the PG 14 release notes. You can > > see the most current build of them here: > > > > https://momjian.us/pgsql_docs/release-14.html > > > > I need clarification on many items, and the document still needs its > > items properly ordered, and markup added. I also expect a lot of > > feedback. > > > > I plan to work on completing this document this coming week in > > preparation for beta next week. > > While only a small change, this commit does affect user visible behavior and > so should probably be noted: > > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=b12bd4869b5e I see your point. Here is the release entry I added: <listitem> <!-- Author: Joe Conway <mail@joeconway.com> 2021-03-31 [b12bd4869] Fix has_column_privilege function corner case --> <para> Return false for has_column_privilege() checks on non-existent or dropped columns (Joe Conway) </para> <para> Previously such columns returned an invalid column error. </para> </listitem> -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Tue, May 11, 2021 at 12:35:28PM +1200, David Rowley wrote:
> Thanks for making the updates.
>
> On Tue, 11 May 2021 at 05:07, Bruce Momjian <bruce@momjian.us> wrote:
> >
> > On Mon, May 10, 2021 at 08:52:44PM +1200, David Rowley wrote:
> > > > Improve the performance of parallel sequential scans (Thomas Munro, David Rowley)
> > >
> > > I think it is worth mentioning "I/O" before "performance". This
> > > change won't really help cases if all the table's pages are already in
> > > shared buffers.
> >
> > I went with:
> >
> > Improve the performance of parallel sequential I/O scans (Thomas Munro,
> > David Rowley)
>
> I think I'd have gone with:
>
> "Improve I/O performance of parallel sequential scans (Thomas Munro,
> David Rowley)"
>
> The operation we're speeding up is called sequential scan. We don't
> have any operation that's named sequential I/O scan.
OK, new text:
<listitem>
<!--
Author: David Rowley <drowley@postgresql.org>
2020-07-26 [56788d215] Allocate consecutive blocks during parallel seqscans
-->
<para>
Improve the I/O performance of parallel sequential scans (Thomas Munro, David Rowley)
</para>
<para>
This was done by allocating blocks in groups to parallel workers.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 04:14:56PM -0700, Peter Geoghegan wrote: > On Mon, May 10, 2021 at 3:58 PM Bruce Momjian <bruce@momjian.us> wrote: > > OK, you are confirming what Matthias suggested. I added these two > > items, which both seem to apply only to heap pages, not index pages: > > That's right -- these two relate to heap pages only. > > I think that Matthias compared these two to bottom-up index deletion > because all three patches are concerned about avoiding "a permanent > solution to a temporary problem". They're conceptually similar despite > being in fairly different areas. Evidently Matthias has a similar > mental model to my own when it comes to this stuff. Agreed, that is a very interesting distinction. > Unfortunately the practical significance of the line pointer patch is > hard to demonstrate with a benchmark. I believe that it is very useful > on a sufficiently long timeline and with certain workloads because of > the behavior it avoids. As I pointed out on that other thread > recently, once you have irreversible bloat very small adverse events > will eventually add up and cause big problems. When this happens it'll > be very hard or impossible to detect, since it just looks like heap > fragmentation. > > That said, it's clearly an issue with one of the TPC-C tables if you > run BenchmarkSQL for days and days (just one table, though). So there > is hard evidence that line pointer bloat could get really out of hand > in at least some tables. OK, once I dug into what you two were saying, I see the significance. I was frankly surprised we didn't already have these optimizations, and you are right they can lead to long-term problems. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 04:02:27PM +0300, Alexander Korotkov wrote: > Hi, Bruce! > > On Mon, May 10, 2021 at 9:03 AM Bruce Momjian <bruce@momjian.us> wrote: > > I have committed the first draft of the PG 14 release notes. You can > > see the most current build of them here: > > > > https://momjian.us/pgsql_docs/release-14.html > > > > I need clarification on many items, and the document still needs its > > items properly ordered, and markup added. I also expect a lot of > > feedback. > > > > I plan to work on completing this document this coming week in > > preparation for beta next week. > > Thank you very much for your work! > > Let me provide a missing description for the items related to me. > > * Improve handling of compound words in to_tsquery() and > websearch_to_tsquery() (Alexander Korotkov) > Compound words are now transformed into parts connected with phrase > search operators. For example, to_tsquery('pg_class') becomes 'pg <-> > class' instead of 'pg & class'. This eliminates bug of handling > compound words connected with the phrase operator and makes the search > of compound words more strict. OK, what symbols trigger this change? Underscore? What else? You are saying the previous code allowed 'pg' and 'class' anywhere in the string, while the new code requires them to be adjacent, which more closely matches the pattern. > * Fix extra distance in phrase operators for quoted text in > websearch_to_tsquery() (Alexander Korotkov) > For example, websearch_to_tsquery('english', '"aaa: bbb"') becomes > 'aaa <> bbb' instead of 'aaa <2> bbb'. So colon and space were considered to be two tokens between 'aaa' and 'bbb', while is really only one because both tokens are discarded? Is this true of any discarded tokens, e.g. ''"aaa ?:, bbb"'? -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 10:53:03PM +0900, Ian Lawrence Barwick wrote: > 2021年5月10日(月) 15:03 Bruce Momjian <bruce@momjian.us>: > > > > I have committed the first draft of the PG 14 release notes. You can > > see the most current build of them here: > > > > https://momjian.us/pgsql_docs/release-14.html > > > > I need clarification on many items, and the document still needs its > > items properly ordered, and markup added. I also expect a lot of > > feedback. > > > > I plan to work on completing this document this coming week in > > preparation for beta next week. > > This misses the change of default value, and is a bit unclear: > > > Remove password_encryption's support for boolean values, e.g. true (Peter Eisentraut) > > > > Previous boolean values enabled md5. Now, only the md5 string does this. > > I'd suggest something along these lines: > > > The default for password_encryption is now "scram-sha-256" (Peter Eisentraut) > > > > The pseudo-boolean values "true", "on", "yes" and "1" are no longer accepted as an alias for "md5". > > (It hasn't been a true boolean setting since Pg 9.6). I reworeded it to: <listitem> <!-- Author: Peter Eisentraut <peter@eisentraut.org> 2020-06-10 [c7eab0e97] Change default of password_encryption to scram-sha-256 --> <para> Limit the ways password_encryption can enable md5 hashing (Peter Eisentraut) </para> <para> Previously on/true/yes/1 values enabled md5. Now, only the string md5 does this. </para> </listitem> I also have this entry: <listitem> <!-- Author: Peter Eisentraut <peter@eisentraut.org> 2020-06-10 [c7eab0e97] Change default of password_encryption to scram-sha-256 --> <para> Change password_encryption's default to scram-sha-256 (Peter Eisentraut) </para> <para> Previously it was md5. </para> </listitem> I am hesitant to merge them since they are different changes. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 11:40 PM Justin Pryzby <pryzby@telsasoft.com> wrote: > Same as the last couple years, I checked for missing items in the release > notes, running something like this. > > git log --cherry-pick --oneline origin/REL_13_STABLE...origin/master > > Should any of these be included? > > 86dc90056d Rework planning and execution of UPDATE and DELETE. > a1115fa078 Postpone some more stuff out of ExecInitModifyTable. > c5b7ba4e67 Postpone some stuff out of ExecInitModifyTable. I was just about to ask Bruce what he thinks about these. To clarify, the first one is a big refactoring commit that allowed us to get rid of inheritance_planner(), a fairly inefficient way of planning updates/deletes on partitioned tables, especially when many partitions remain after pruning (or when pruning cannot be used). One may see the performance of update/deletes, especially on partitioned tables, to be generally improved as a result of this commit, but maybe not as significantly as to be mentioned in E.1.3.1.1. Partitioning or even E.1.3.1.4. General Performance. However, one user-visible feature that came out of this work is that updates/deletes can now use run-time pruning whereas they couldn't before. Maybe that ought to be mentioned. (This reminds me to send a patch to remove the note from 5.11.4. Partition Pruning that says that runtime pruning cannot be used for update/delete). The other two commits can lead to improved performance of update/deletes when there are many unpruned partitions in the plan, but runtime pruning (a new feature as mentioned above) leads to only one or few partitions to actually be updated/deleted from. I admit though that the cases for which performance has been improved still under-perform the cases that already performed better starting in v12, that is, the cases where the planner itself is able to trim down the plan to contain one or few partitions, so maybe nothing very big to see here just yet. You may want to take a look at the benchmark results I had posted here: https://www.postgresql.org/message-id/CA%2BHiwqEcawatEaUh1uTbZMEZTJeLzbroRTz9_X9Z5CFjTWJkhw%40mail.gmail.com Thank you. -- Amit Langote EDB: http://www.enterprisedb.com
On Mon, May 10, 2021 at 7:18 PM Bruce Momjian <bruce@momjian.us> wrote: > > On Mon, May 10, 2021 at 04:14:56PM -0700, Peter Geoghegan wrote: > > On Mon, May 10, 2021 at 3:58 PM Bruce Momjian <bruce@momjian.us> wrote: > > > OK, you are confirming what Matthias suggested. I added these two > > > items, which both seem to apply only to heap pages, not index pages: > > > > That's right -- these two relate to heap pages only. > > > > I think that Matthias compared these two to bottom-up index deletion > > because all three patches are concerned about avoiding "a permanent > > solution to a temporary problem". They're conceptually similar despite > > being in fairly different areas. Evidently Matthias has a similar > > mental model to my own when it comes to this stuff. > > Agreed, that is a very interesting distinction. BTW, I think that the wording of the bottom-up index deletion item should be changed to describe the result rather than the mechanism itself. The important idea is that non-HOT updaters are made to clean up their own mess before it gets out of hand (i.e. before a version-driven page split can go ahead), at least for those indexes whose columns are not logically modified by the UPDATE statement (usually most indexes on the table). We're making the updaters live within their means -- they cannot be allowed to avoid paying small incremental costs if that ultimately imposes a much larger, lasting cost on the system as a whole. This could be thought of as a negative externality. The index deletion mechanism itself is not that different to what we had before. The important point is how and when it kicks in, and the systematic effect of that over time. It's a subtractive thing, not an additive thing -- it's helpful because of what *doesn't* happen in Postgres 14. -- Peter Geoghegan
On Mon, May 10, 2021 at 7:30 PM Justin Pryzby <pryzby@telsasoft.com> wrote: > > Can these be merged: > Allow logical replication to stream long transactions to standbys (Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke) > Improve the logical replication API to allow streaming large in-progress transactions (Tomas Vondra, Dilip Kumar, AmitKapila) > I think we should merge the below two: Add support for streaming to built-in logical replication (Vondra, Dilip Kumar, Amit Kapila, Ajin Cherian, Nikhil Sontakke, Stas Kelvich) Allow logical replication to stream long transactions to standbys (Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke) and say something like: Allow logical replication to stream long in-progress transactions to standbys Previously transactions that exceeded logical_decoding_work_mem were written to disk until the transaction completed. Also, the commits that lead to the above feature is: 2020-09-03 [464824323] Add support for streaming to built-in logical replicatio 2020-08-08 [7259736a6] Implement streaming mode in ReorderBuffer. * Improve the logical replication API to allow streaming large in-progress transactions (Tomas Vondra, Dilip Kumar, Amit Kapila) The output functions begin with "stream". test_decoding also supports these. In the above, I think it is better to say Enhance instead of Improve. * Add system view pg_stat_replication_slots to report replication slot activity (Sawada Masahiko, Amit Kapila) I think for this item, we can give Vignesh C as author credit because he helped a lot enhance/stabilize this feature after the initial commit. See https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=f5fc2f5b23d1b1dff60f8ca5dc211161df47eda4 https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=cca57c1d9bf7eeba5b81115e0b82651cf3d8e4ea https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=592f00f8dec68038301467a904ac514eddabf6cd * Allow multiple xacts during table sync in logical replication (Peter Smith, Amit Kapila, and Takamichi Osumi) IMPORTANT? Yes, this is a big enough improvement in the initial table sync mechanism of logical replication that it is worth mentioning in Release Notes. One of the main user-visible effects this commit can have is that it will reduce the time for which we need to hold WAL on the server during the initial table sync phase. Then it can avoid large copies in case there is any error during the time when apply-worker and tablesync-worker synchronizes the apply location. * Enable logical replication to handle two phase commits (Ajin Cherian) This is controlled via pg_create_logical_replication_slot(). We still haven't supported two-phase commits for built-in logical replication but we have exposed APIs for decoding plugins to achieve that. Using these APIs, we are planning to support it for built-in logical replication but that will be probably done in PG-15. So, I suggest we can modify it to something like: * Enhance logical decoding APIs to handle two-phase commits This is controlled via pg_create_logical_replication_slot(). Most of the work for this feature is done via commits: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=a271a1b50e9bec07e2ef3a05e38e7285113e4ce6 https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=0aa8a01d04c8fe200b7a106878eebc3d0af9105c So, I think the author credit should be given to Ajin Cherian, Amit Kapila, Nikhil Sontakke, and Stas Kelvich * I think we should mention about commit: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=d7eb52d7181d83cf2363570f7a205b8eb1008dbc This allows Logical decoding to work efficiently in presence of a large amount of DDLs. This was a complaint from many users for long time. Apart from the discussion thread, see some of the earlier complaints: https://www.postgresql.org/message-id/flat/20f3de7675f83176253f607b5e199b228406c21c.camel@cybertec.at https://www.postgresql.org/message-id/flat/CAHoiPjzea6N0zuCi=+f9v_j94nfsy6y8SU7-=bp4=7qw6_i=Rg@mail.gmail.com and most recently at: https://www.postgresql.org/message-id/DB8PR10MB36126BC0DC3570637FDFED12EB4F9%40DB8PR10MB3612.EURPRD10.PROD.OUTLOOK.COM * Speed truncation of small tables on large shared buffer servers (Kirk Jamison) This optimization applies during recovery. Shouldn't we mention that in some way? -- With Regards, Amit Kapila.
On Mon, 2021-05-10 at 13:47 -0400, Bruce Momjian wrote: > Uh, I try to keep the first sentence short so people can scan it more > easily, so I am hesitant to make this change. I went with this change: > > <listitem> > <!-- > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2021-04-03 [55873a00e] Improve psql's behavior when the editor is exited withou > --> > > <para> > When editing the previous query or a file with psql's \e, or using \ef and \ev, ignore the contents if the editor exitswithout saving (Laurenz Albe) > </para> > > <para> > Previously, such edits would still execute the editor contents. > </para> > </listitem> That is fine with me and probably better than my suggestion. Yours, Laurenz Albe
On Tue, May 11, 2021 at 5:34 AM Bruce Momjian <bruce@momjian.us> wrote: > On Mon, May 10, 2021 at 04:02:27PM +0300, Alexander Korotkov wrote: > > Hi, Bruce! > > > > On Mon, May 10, 2021 at 9:03 AM Bruce Momjian <bruce@momjian.us> wrote: > > > I have committed the first draft of the PG 14 release notes. You can > > > see the most current build of them here: > > > > > > https://momjian.us/pgsql_docs/release-14.html > > > > > > I need clarification on many items, and the document still needs its > > > items properly ordered, and markup added. I also expect a lot of > > > feedback. > > > > > > I plan to work on completing this document this coming week in > > > preparation for beta next week. > > > > Thank you very much for your work! > > > > Let me provide a missing description for the items related to me. > > > > * Improve handling of compound words in to_tsquery() and > > websearch_to_tsquery() (Alexander Korotkov) > > Compound words are now transformed into parts connected with phrase > > search operators. For example, to_tsquery('pg_class') becomes 'pg <-> > > class' instead of 'pg & class'. This eliminates bug of handling > > compound words connected with the phrase operator and makes the search > > of compound words more strict. > > OK, what symbols trigger this change? Underscore? What else? Any symbol, which is recognized as a separator by full-text parser, but not tsquery parser. Fulltext search is extensible and allowing pluggable parsers. In principle, we could dig the exact set of symbols, but I'm not sure this worth the effort. > You are > saying the previous code allowed 'pg' and 'class' anywhere in the > string, while the new code requires them to be adjacent, which more > closely matches the pattern. Yes, that's it. > > * Fix extra distance in phrase operators for quoted text in > > websearch_to_tsquery() (Alexander Korotkov) > > For example, websearch_to_tsquery('english', '"aaa: bbb"') becomes > > 'aaa <> bbb' instead of 'aaa <2> bbb'. > > So colon and space were considered to be two tokens between 'aaa' and > 'bbb', while is really only one because both tokens are discarded? Is > this true of any discarded tokens, e.g. ''"aaa ?:, bbb"'? Yes, that's true for any discarded tokens. ------ Regards, Alexander Korotkov
On 5/10/21 9:56 PM, Bruce Momjian wrote: > On Mon, May 10, 2021 at 07:50:14AM -0400, Joe Conway wrote: >> On 5/10/21 2:03 AM, Bruce Momjian wrote: >> > I have committed the first draft of the PG 14 release notes. You can >> > see the most current build of them here: >> > >> > https://momjian.us/pgsql_docs/release-14.html >> > >> > I need clarification on many items, and the document still needs its >> > items properly ordered, and markup added. I also expect a lot of >> > feedback. >> > >> > I plan to work on completing this document this coming week in >> > preparation for beta next week. >> >> While only a small change, this commit does affect user visible behavior and >> so should probably be noted: >> >> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=b12bd4869b5e > > I see your point. Here is the release entry I added: Not quite -- maybe something like the below: > <para> > Return false for has_column_privilege() checks on non-existent or dropped columns (Joe Conway) Always return NULL for has_column_privilege() checks on non-existent or dropped column by attnum (Joe Conway) > <para> > Previously such columns returned an invalid column error. Previously existence of such columns were ignored when caller had table level privileges. -- Crunchy Data - http://crunchydata.com PostgreSQL Support for Secure Enterprises Consulting, Training, & Open Source Development
First Bruce, thank you for compiling the release notes draft. Comments inline: On 5/10/21 11:56 PM, Amit Kapila wrote: > On Mon, May 10, 2021 at 7:30 PM Justin Pryzby <pryzby@telsasoft.com> wrote: >> >> Can these be merged: >> Allow logical replication to stream long transactions to standbys (Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke) >> Improve the logical replication API to allow streaming large in-progress transactions (Tomas Vondra, Dilip Kumar, AmitKapila) >> > > I think we should merge the below two: > Add support for streaming to built-in logical replication (Vondra, > Dilip Kumar, Amit Kapila, Ajin Cherian, Nikhil Sontakke, Stas Kelvich) > > Allow logical replication to stream long transactions to standbys > (Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke) > > and say something like: > Allow logical replication to stream long in-progress transactions to standbys > Previously transactions that exceeded logical_decoding_work_mem were > written to disk until the transaction completed. I was going to post a question about the above bullet point(s), so this clarification helps -- thanks! Two comments on the above: 1. I do believe ther eis a missing "Tomas" in the first release note. 2. Would you want to s/standbys/subscribers/ given an instance can be in read/write mode but still receive logical changes? Or did this patch affect only subscribers that are also standbys? Thanks, Jonathan
Attachment
I think these should be merged: | Remove factorial operators ! and !! (Mark Dilger) | Add documentation for the factorial() function (Peter Eisentraut) | This can be disabled by turning client options "sslsni" off. option (no ess) | Allow REINDEX to process all child tables and indexes of a partitioned table (Justin Pryzby, Michael Paquier) It should actually say "all child tables or indexes of a partitioned relation", since you can REINDEX INDEX partitioned_index. | Add bit_xor XOR aggregate function (Alexey Bashtanov) Remove XOR ? | Remove support for postfix (right-unary) operators (Mark Dilger) | pg_dump and pg_upgrade will warn if post-fix operators are being dumped. postfix not post-fix | Remove temporary files after backend crashes (Euler Taveira) crash (singular) | Increase warning time and hard limit before transaction id and multi-transaction wraparound (Noah Misch) | This should reduce the number of failures without warning. I'd say: This should reduce the possibility of failures that occur without having issued warnings about wraparound. -- Justin
On Mon, May 10, 2021 at 09:00:05AM -0500, Justin Pryzby wrote:
> Thanks for putting it together.
>
> I think these two should be merged:
> | Remove containment operators @ and ~ from contrib modules cube, hstore, intarray, and seg (Justin Pryzby)
> | Remove deprecated containment operators for built-in geometry data types (Justin Pryzby)
Agreed, merged.
>
> | Improve autovacuum's analyze of partitioned tables (Yuzuko Hosoya)
> | DETAILS?
>
> Should say: Autovacuum now analyzes partitioned tables.
Agreed, updated.
> | The server variable check_client_connection_interval allows supporting operating systems, e.g., Linux, to
automaticallycancel queries by disconnected clients.
> The GUC is actually called client_connection_check_interval - the commit
> message used the wrong name.
OK, fixed.
> | This is particularly helpful for reducing index bloat on tables that frequently update indexed columns.
> Does it mean "..where indexed columns are frequently updated"?
Yeah, that needs help. ;-) I updated it to :
This is particularly helpful for reducing index bloat on tables
whose indexed columns are frequently updated.
> | Allow multiple foreign table scans to be run in parallel (Robert Haas, Kyotaro Horiguchi, Thomas Munro, Etsuro
Fujita)
> I think it means multiple foreight table scan *nodes*
How is this?
Allow a query referencing multiple foreign tables to perform foreign
table scans in parallel (Robert Haas, Kyotaro Horiguchi, Thomas Munro,
Etsuro Fujita)
I am worried "nodes" is too vague.
> | If server variable compute_query_id is enabled, display the hash in pg_stat_activity, EXPLAIN VERBOSE, csvlog, and
optionallyin log_line_prefix (Julien Rouhaud)
> I think needs details, like: "If disabled, then the hash might be computed by
> an extension, instead".
I ended up with:
<para>
If server variable compute_query_id is enabled, display the hash
in pg_stat_activity, EXPLAIN VERBOSE, csvlog, and optionally in
log_line_prefix (Julien Rouhaud)
</para>
<para>
A query id computed by an extension will also be displayed.
</para>
> Later, you say:
> | Extension pg_stat_statements will need to enable hash computation via the compute_query_id server variable to
functionproperly. pg_stat_statements can now use a custom hash computation method.
> Maybe it should say "will need hash computation to be enabled".
Here is the updated entry:
<listitem>
<!--
Author: Bruce Momjian <bruce@momjian.us>
2021-04-07 [5fd9dfa5f] Move pg_stat_statements query jumbling to core.
-->
<para>
Move query hash computation from pg_stat_statements to the core server (Julien Rouhaud)
</para>
<para>
Extension pg_stat_statements will now need to enable query hash computation to function properly.
This can be done by enabling the server variable compute_query_id or by using an extension with a custom hash
computationmethod.
</para>
</listitem>
> | Allow more than the common name (CN) to be matched for client certificate authentication (Andrew Dunstan)
> Your description makes it sound like arbitrary attributes can be compared. But
> the option just allows comparing CN or DN.
OK, new text is:
<listitem>
<!--
Author: Andrew Dunstan <andrew@dunslane.net>
2021-03-29 [6d7a6feac] Allow matching the DN of a client certificate for
authen
-->
<para>
Allow the certificate's distinguished name (DN) to be matched for client
certificate authentication (Andrew Dunstan)
</para>
<para>
The new pg_hba.conf keyword "clientname=DN" allows comparison with
non-CN certificate attributes and can be combined with ident maps.
</para>
</listitem>
> | Allow file system sync at the start of crash recovery on Linux (Thomas Munro)
> I think this should describe the existing, default behavior:
> Allow syncfs method to sync data directory during recovery;
> The default behavior is to open and fsync every data file, and the new setting
> recovery_init_sync_method=syncfs instead syncs each filesystem in the data
> directory.
I went with this text:
<listitem>
<!--
Author: Thomas Munro <tmunro@postgresql.org>
2021-03-20 [61752afb2] Provide recovery_init_sync_method=syncfs.
-->
<para>
Allow file system sync at the start of crash recovery on Linux (Thomas
Munro)
</para>
<para>
By default, Postgres opens and fsyncs every data file at the start of
crash recovery.
This new setting, recovery_init_sync_method=syncfs, instead syncs each
filesystem used by the database cluster.
This allows for faster recovery on systems with many database files.
</para>
</listitem>
> | Add date_bin function (John Naylor)
> This truncate timestamps on an arbitrary interval.
> Like date_trunc() but also supports eg. '15 minutes', and also uses an arbitrary "origin".
OK, so what I think it returns is the greatest datetime that is a
multiple of interval values added to origin which is not greater than
the target date, right? Am I the only one who finds this unclear?
Doesn't our documentation of this feature need to explain this?
> | Support negative indexes in split_part() (Nikhil Benesch)
> | Negative values count from the last field going forward.
> should say "start from the last field and count backward" ?
Yes, fixed with your wording.
> | Add configure option --with-openssl to behave like --with-ssl={openssl} (Daniel Gustafsson, Michael Paquier)
> | The option --with-openssl is kept for compatibility.
> I think this is backwards. The new option is with-ssl=openssl, and (as you
> said) with-openssl is kept.
Agreed:
Add configure option --with-ssl={openssl} to behave like
--with-openssl (Daniel Gustafsson, Michael Paquier)
> Should these be in the "compatibility" section?
>
> | Force custom server variable names to match the pattern used for unquoted SQL identifiers (Tom Lane)
Yes.
> | Change password_encryption's default to scram-sha-256 (Peter Eisentraut)
Yes, I can see this impacting people. I move it and added text to
highlight the incompatibility:
<listitem>
<!--
Author: Peter Eisentraut <peter@eisentraut.org>
2020-06-10 [c7eab0e97] Change default of password_encryption to scram-sha-256
-->
<para>
Change password_encryption's default to scram-sha-256 (Peter Eisentraut)
</para>
<para>
Previously it was md5. All new passwords will be stored as
SHA256 unless this server variable is changed or the password is
already md5-hashed.
</para>
</listitem>
>
> | Change checkpoint_completion_target default to 0.9 (Stephen Frost)
I don' think that is an incompatibility since it only affects
performance, and can be easily changed.
> | Reduce the default value of vacuum_cost_page_miss (Peter Geoghegan)
>
> Nitpicks to follow:
>
> | Allow some GiST index to be built by presorting the data (Andrey Borodin)
> indexes
Fixed.
> | with --with-lz4 support to enable this feature
> I would change to say "to support" rather than "support to enable"
Yes, better.
> | Speed truncation of small tables on large shared buffer servers (Kirk Jamison)
> "on servers with large settings of shared_buffers"
I went with:
Speed truncation of small tables on clusters with a large number of
shared buffers (Kirk Jamison)
> | Allow windowing functions to perform incremental sorts (David Rowley)
> Just "window" functions
OK, fixed.
> | Improve pg_stat_activity reporting for walsenders processes (Tom Lane)
> walsender
Fixed.
> | Previously these functions could only be executed by super-users, and still defaults do that.
> ..which is still the default behavior.
Updated to:
Previously these functions could only be executed by
super-users, and this is still the default.
> | This allows multiple queries to be send and only wait for completion when a specific synchronization message is
sent.
> be sent
Fixed.
> | Enhance libpq libpq's target_session_attrs parameter options (Haribabu Kommi, Greg Nancarrow, Vignesh C, Tom Lane)
> remove first "libpq"
Fixed.
> | With the removal of the ! operator in this release, factorial() is the only built-in way to computer a factorial.
> compute
Fixed.
> | For example, GROUP BY CUBE (a,b), CUBE (b,c) will generated duplicate grouping combinations without DISTINCT.
>
> will generate
Fixed.
> | Allow VACUUM VERBOSE to report page deletion counts for each scan of an index (Peter Geoghegan)
>
> I think "Allow" is wrong - should just say that VACUUM VERBOSE reports..
Updated to:
Have VACUUM VERBOSE report page deletion counts for each scan of
an index (Peter Geoghegan)
> |By default, only the root of partitioned tables are imported.
> *is* imported
Fixed.
> Can these be merged:
> Allow logical replication to stream long transactions to standbys (Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil
Sontakke)
> Improve the logical replication API to allow streaming large in-progress transactions (Tomas Vondra, Dilip Kumar,
AmitKapila)
I am not sure. I do think we need to merge some of the logical
replication items, but I am not sure which ones yet.
Thanks for all the good feedback.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 08:53:54PM -0700, Peter Geoghegan wrote:
> On Mon, May 10, 2021 at 7:18 PM Bruce Momjian <bruce@momjian.us> wrote:
> >
> > On Mon, May 10, 2021 at 04:14:56PM -0700, Peter Geoghegan wrote:
> > > On Mon, May 10, 2021 at 3:58 PM Bruce Momjian <bruce@momjian.us> wrote:
> > > > OK, you are confirming what Matthias suggested. I added these two
> > > > items, which both seem to apply only to heap pages, not index pages:
> > >
> > > That's right -- these two relate to heap pages only.
> > >
> > > I think that Matthias compared these two to bottom-up index deletion
> > > because all three patches are concerned about avoiding "a permanent
> > > solution to a temporary problem". They're conceptually similar despite
> > > being in fairly different areas. Evidently Matthias has a similar
> > > mental model to my own when it comes to this stuff.
> >
> > Agreed, that is a very interesting distinction.
>
> BTW, I think that the wording of the bottom-up index deletion item
> should be changed to describe the result rather than the mechanism
> itself. The important idea is that non-HOT updaters are made to clean
> up their own mess before it gets out of hand (i.e. before a
> version-driven page split can go ahead), at least for those indexes
> whose columns are not logically modified by the UPDATE statement
> (usually most indexes on the table). We're making the updaters live
> within their means -- they cannot be allowed to avoid paying small
> incremental costs if that ultimately imposes a much larger, lasting
> cost on the system as a whole. This could be thought of as a negative
> externality.
>
> The index deletion mechanism itself is not that different to what we
> had before. The important point is how and when it kicks in, and the
> systematic effect of that over time. It's a subtractive thing, not an
> additive thing -- it's helpful because of what *doesn't* happen in
> Postgres 14.
I updated this to:
<listitem>
<!--
Author: Peter Geoghegan <pg@bowt.ie>
2021-01-13 [9dc718bdf] Pass down "logically unchanged index" hint.
Author: Peter Geoghegan <pg@bowt.ie>
2021-01-13 [d168b6668] Enhance nbtree index tuple deletion.
-->
<para>
Allow index additions to remove expired btree index entries to prevent page
splits (Peter Geoghegan)
</para>
<para>
This is particularly helpful for reducing index bloat on tables whose
indexed columns are frequently updated.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Tue, May 11, 2021 at 06:57:19AM -0400, Joe Conway wrote: > On 5/10/21 9:56 PM, Bruce Momjian wrote: > > On Mon, May 10, 2021 at 07:50:14AM -0400, Joe Conway wrote: > > > On 5/10/21 2:03 AM, Bruce Momjian wrote: > > > > I have committed the first draft of the PG 14 release notes. You can > > > > see the most current build of them here: > > > > > https://momjian.us/pgsql_docs/release-14.html > > > > > I need clarification on many items, and the document still needs > > > its > > > > items properly ordered, and markup added. I also expect a lot of > > > > feedback. > > > > > I plan to work on completing this document this coming week in > > > > preparation for beta next week. > > > > > > While only a small change, this commit does affect user visible behavior and > > > so should probably be noted: > > > > > > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=b12bd4869b5e > > > > I see your point. Here is the release entry I added: > > Not quite -- maybe something like the below: > > > <para> > > Return false for has_column_privilege() checks on non-existent or dropped columns (Joe Conway) > > Always return NULL for has_column_privilege() checks on non-existent or > dropped column by attnum (Joe Conway) > > > <para> > > Previously such columns returned an invalid column error. > > Previously existence of such columns were ignored when caller had table > level privileges. I can't reproduce the NULL using column name text: test=> SELECT CURRENT_USER; current_user -------------- postgres (1 row) test=> CREATE TABLE test (x int, y int); CREATE TABLE test=> CREATE USER test; CREATE ROLE test=> \c test test You are now connected to database "test" as user "test". test=> SELECT has_column_privilege('test', 'x', 'SELECT'); has_column_privilege ---------------------- f test=> SELECT has_column_privilege('test', 'z', 'SELECT'); ERROR: column "z" of relation "test" does not exist but I was able to show it using attribute numbers, as you explained: test=> SELECT has_column_privilege('test', 1::smallint, 'SELECT'); has_column_privilege ---------------------- f (1 row) test=> SELECT has_column_privilege('test', 2::smallint, 'SELECT'); has_column_privilege ---------------------- f (1 row) test=> SELECT has_column_privilege('test', 3::smallint, 'SELECT'); has_column_privilege ---------------------- (1 row) What is the logic that attribute numbers should behave differently than attribute names? Here is the updated text: <listitem> <!-- Author: Joe Conway <mail@joeconway.com> 2021-03-31 [b12bd4869] Fix has_column_privilege function corner case --> <para> Return false for has_column_privilege() checks on non-existent or dropped columns when using attribute numbers (Joe Conway) </para> <para> Previously such attribute numbers returned an invalid column error. </para> </listitem> -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 02:03:08AM -0400, Bruce Momjian wrote: > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > > I need clarification on many items, and the document still needs its > items properly ordered, and markup added. I also expect a lot of > feedback. > > I plan to work on completing this document this coming week in > preparation for beta next week. Just to clarify, I am trying to process the release note feedback as quickly as possible, and committing changes 1-2 times a day. However, the feedback is currently coming in quicker than I can process it --- eventually I will catch up. All the feedback has been very useful. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On 5/11/21 11:11 AM, Bruce Momjian wrote: > On Tue, May 11, 2021 at 06:57:19AM -0400, Joe Conway wrote: >> On 5/10/21 9:56 PM, Bruce Momjian wrote: >> > On Mon, May 10, 2021 at 07:50:14AM -0400, Joe Conway wrote: >> > > On 5/10/21 2:03 AM, Bruce Momjian wrote: >> > > > I have committed the first draft of the PG 14 release notes. You can >> > > > see the most current build of them here: >> > > > > https://momjian.us/pgsql_docs/release-14.html >> > > > > I need clarification on many items, and the document still needs >> > > its >> > > > items properly ordered, and markup added. I also expect a lot of >> > > > feedback. >> > > > > I plan to work on completing this document this coming week in >> > > > preparation for beta next week. >> > > >> > > While only a small change, this commit does affect user visible behavior and >> > > so should probably be noted: >> > > >> > > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=b12bd4869b5e >> > >> > I see your point. Here is the release entry I added: >> >> Not quite -- maybe something like the below: >> >> > <para> >> > Return false for has_column_privilege() checks on non-existent or dropped columns (Joe Conway) >> >> Always return NULL for has_column_privilege() checks on non-existent or >> dropped column by attnum (Joe Conway) >> >> > <para> >> > Previously such columns returned an invalid column error. >> >> Previously existence of such columns were ignored when caller had table >> level privileges. > > I can't reproduce the NULL using column name text: > test=> SELECT has_column_privilege('test', 'z', 'SELECT'); > ERROR: column "z" of relation "test" does not exist That is the way it is supposed to work when the column is specified by name. The patch did not change that in any way. > but I was able to show it using attribute numbers, as you explained: Yes, that was what the patch fixed. Joe -- Crunchy Data - http://crunchydata.com PostgreSQL Support for Secure Enterprises Consulting, Training, & Open Source Development
On Tue, May 11, 2021 at 11:26:48AM -0400, Joe Conway wrote:
> On 5/11/21 11:11 AM, Bruce Momjian wrote:
> > > Previously existence of such columns were ignored when caller had table
> > > level privileges.
> >
> > I can't reproduce the NULL using column name text:
>
> > test=> SELECT has_column_privilege('test', 'z', 'SELECT');
> > ERROR: column "z" of relation "test" does not exist
>
> That is the way it is supposed to work when the column is specified by name.
> The patch did not change that in any way.
I am just confused why attribute numbers are handled differently than
attribute names.
> > but I was able to show it using attribute numbers, as you explained:
>
> Yes, that was what the patch fixed.
Thanks.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On 5/11/21 11:37 AM, Bruce Momjian wrote:
> On Tue, May 11, 2021 at 11:26:48AM -0400, Joe Conway wrote:
>> On 5/11/21 11:11 AM, Bruce Momjian wrote:
>> > > Previously existence of such columns were ignored when caller had table
>> > > level privileges.
>> >
>> > I can't reproduce the NULL using column name text:
>>
>> > test=> SELECT has_column_privilege('test', 'z', 'SELECT');
>> > ERROR: column "z" of relation "test" does not exist
>>
>> That is the way it is supposed to work when the column is specified by name.
>> The patch did not change that in any way.
>
> I am just confused why attribute numbers are handled differently than
> attribute names.
I am not entirely sure, but that boat sailed a long time ago and really
has nothing to do with this patch ;-)
This is the code comment that predates the patch but is the reason
behind the change:
------------
/*
* has_any_column_privilege variants
* These are all named "has_any_column_privilege" at the SQL level.
* They take various combinations of relation name, relation OID,
* user name, user OID, or implicit user = current_user.
*
* The result is a boolean value: true if user has the indicated
* privilege for any column of the table, false if not. The variants
* that take a relation OID return NULL if the OID doesn't exist.
*/
------------
The patch made that last sentence true in the corner cases.
Joe
--
Crunchy Data - http://crunchydata.com
PostgreSQL Support for Secure Enterprises
Consulting, Training, & Open Source Development
On Tue, May 11, 2021 at 12:31:01PM -0400, Joe Conway wrote:
> On 5/11/21 11:37 AM, Bruce Momjian wrote:
> > On Tue, May 11, 2021 at 11:26:48AM -0400, Joe Conway wrote:
> > > On 5/11/21 11:11 AM, Bruce Momjian wrote:
> > > > > Previously existence of such columns were ignored when caller had table
> > > > > level privileges.
> > > > > I can't reproduce the NULL using column name text:
> > >
> > > > test=> SELECT has_column_privilege('test', 'z', 'SELECT');
> > > > ERROR: column "z" of relation "test" does not exist
> > >
> > > That is the way it is supposed to work when the column is specified by name.
> > > The patch did not change that in any way.
> >
> > I am just confused why attribute numbers are handled differently than
> > attribute names.
>
> I am not entirely sure, but that boat sailed a long time ago and really has
> nothing to do with this patch ;-)
It just feels like this change makes the function's behavior less
consistent.
> This is the code comment that predates the patch but is the reason behind
> the change:
>
> ------------
> /*
> * has_any_column_privilege variants
> * These are all named "has_any_column_privilege" at the SQL level.
> * They take various combinations of relation name, relation OID,
> * user name, user OID, or implicit user = current_user.
> *
> * The result is a boolean value: true if user has the indicated
> * privilege for any column of the table, false if not. The variants
> * that take a relation OID return NULL if the OID doesn't exist.
> */
> ------------
>
> The patch made that last sentence true in the corner cases.
Well, the example I showed was for attribute numbers but relation names,
which isn't mentioned in this comment.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On 5/11/21 1:30 PM, Bruce Momjian wrote:
> On Tue, May 11, 2021 at 12:31:01PM -0400, Joe Conway wrote:
>> On 5/11/21 11:37 AM, Bruce Momjian wrote:
>> > On Tue, May 11, 2021 at 11:26:48AM -0400, Joe Conway wrote:
>> > > On 5/11/21 11:11 AM, Bruce Momjian wrote:
>> > > > > Previously existence of such columns were ignored when caller had table
>> > > > > level privileges.
>> > > > > I can't reproduce the NULL using column name text:
>> > >
>> > > > test=> SELECT has_column_privilege('test', 'z', 'SELECT');
>> > > > ERROR: column "z" of relation "test" does not exist
>> > >
>> > > That is the way it is supposed to work when the column is specified by name.
>> > > The patch did not change that in any way.
>> >
>> > I am just confused why attribute numbers are handled differently than
>> > attribute names.
>>
>> I am not entirely sure, but that boat sailed a long time ago and really has
>> nothing to do with this patch ;-)
>
> It just feels like this change makes the function's behavior less
> consistent.
See Tom's commit message here:
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=3d0f68dd30612
In particular:
"The variants of these functions that take
numeric inputs (OIDs or column numbers) are
supposed to return NULL rather than failing
on bad input; this rule reduces problems with
snapshot skew when queries apply the functions
to all rows of a catalog."
Joe
--
Crunchy Data - http://crunchydata.com
PostgreSQL Support for Secure Enterprises
Consulting, Training, & Open Source Development
Joe Conway <mail@joeconway.com> writes: > On 5/11/21 1:30 PM, Bruce Momjian wrote: >> It just feels like this change makes the function's behavior less >> consistent. > See Tom's commit message here: > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=3d0f68dd30612 > In particular: > "The variants of these functions that take > numeric inputs (OIDs or column numbers) are > supposed to return NULL rather than failing > on bad input; this rule reduces problems with > snapshot skew when queries apply the functions > to all rows of a catalog." Yeah, the null-return-for-bad-numeric-input behavior is important. Perhaps a case could be made for returning null for bad text input too, but I don't recall that anybody has asked for that. A case could also be made that changing the behavior on the text side would break applications that expect the current behavior. So I'm disinclined to make a wholesale change there, without more evidence that it's a good idea. regards, tom lane
On Tue, May 11, 2021 at 03:51:39PM -0400, Tom Lane wrote: > Joe Conway <mail@joeconway.com> writes: > > On 5/11/21 1:30 PM, Bruce Momjian wrote: > >> It just feels like this change makes the function's behavior less > >> consistent. > > > See Tom's commit message here: > > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=3d0f68dd30612 > > > In particular: > > > "The variants of these functions that take > > numeric inputs (OIDs or column numbers) are > > supposed to return NULL rather than failing > > on bad input; this rule reduces problems with > > snapshot skew when queries apply the functions > > to all rows of a catalog." > > Yeah, the null-return-for-bad-numeric-input behavior is important. > Perhaps a case could be made for returning null for bad text > input too, but I don't recall that anybody has asked for that. > > A case could also be made that changing the behavior on the text > side would break applications that expect the current behavior. > So I'm disinclined to make a wholesale change there, without more > evidence that it's a good idea. OK, as long as we thought about this, I am fine. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Tue, May 11, 2021 at 01:16:38PM +0300, Alexander Korotkov wrote:
> > OK, what symbols trigger this change? Underscore? What else?
>
> Any symbol, which is recognized as a separator by full-text parser,
> but not tsquery parser. Fulltext search is extensible and allowing
> pluggable parsers. In principle, we could dig the exact set of
> symbols, but I'm not sure this worth the effort.
>
> > You are
> > saying the previous code allowed 'pg' and 'class' anywhere in the
> > string, while the new code requires them to be adjacent, which more
> > closely matches the pattern.
>
> Yes, that's it.
>
> > > * Fix extra distance in phrase operators for quoted text in
> > > websearch_to_tsquery() (Alexander Korotkov)
> > > For example, websearch_to_tsquery('english', '"aaa: bbb"') becomes
> > > 'aaa <> bbb' instead of 'aaa <2> bbb'.
> >
> > So colon and space were considered to be two tokens between 'aaa' and
> > 'bbb', while is really only one because both tokens are discarded? Is
> > this true of any discarded tokens, e.g. ''"aaa ?:, bbb"'?
>
> Yes, that's true for any discarded tokens.
I can up with this text for these two items. I think it still needs ro
be more specific:
<listitem>
<!--
Author: Alexander Korotkov <akorotkov@postgresql.org>
2021-01-31 [0c4f355c6] Fix parsing of complex morphs to tsquery
-->
<para>
Fix to_tsquery() and websearch_to_tsquery() to properly parse
certain discarded tokens in quotes (Alexander Korotkov)
</para>
<para>
Certain discarded tokens, like underscore, caused the output
of these functions to produce incorrect tsquery output, e.g.,
websearch_to_tsquery('"pg_class pg"') used to output '( pg &
class ) <-> pg', but now outputs 'pg <-> class <-> pg'.
</para>
</listitem>
<listitem>
<!--
Author: Alexander Korotkov <akorotkov@postgresql.org>
2021-05-03 [eb086056f] Make websearch_to_tsquery() parse text in quotes as a si
-->
<para>
Fix websearch_to_tsquery() to properly parse multiple adjacent
discarded tokens in quotes (Alexander Korotkov)
</para>
<para>
Previously, quoted text that contained multiple adjacent discarded
tokens were treated as multiple tokens, causing incorrect tsquery
output, e.g., websearch_to_tsquery('"aaa: bbb"') used to output
'aaa <2> bbb', but now outputs 'aaa <-> bbb'.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 08:08:01AM -0700, Peter Geoghegan wrote: > On Mon, May 10, 2021 at 7:00 AM Justin Pryzby <pryzby@telsasoft.com> wrote: > > | Allow VACUUM VERBOSE to report page deletion counts for each scan of an index (Peter Geoghegan) > > > > I think "Allow" is wrong - should just say that VACUUM VERBOSE reports.. > > It's also not accurate, since the count of deleted pages was always > shown by VACUUM VERBOSE (once per index scan). The new feature has us > show pages deleted by the VACUUM that actually ran (not some earlier > VACUUM) -- these are "newly deleted pages". > > I don't think that this item is worth mentioning, though -- it's just > a nice to have. If Bruce removes it from the release notes entirely I > won't object. OK, removed. Additional debug output usually isn't important mentioning --- when you need it, you will see it. > In addition to the items that I commented on in my response to > Matthias just now, I should point out the following item as worthy of > inclusion: > > 9dd963ae25 Recycle nbtree pages deleted during same VACUUM. > > I suggest that this item be phrased more or less as follows: > > "Allow VACUUM to eagerly place newly deleted B-Tree pages in the Free > Space Map. Previously VACUUM could only place preexisting deleted > pages in the Free Space Map for recycling." So, previously it could only place pages that were deleted from the previous vacuum on the free list? -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Tue, May 11, 2021 at 1:41 PM Bruce Momjian <bruce@momjian.us> wrote: > > "Allow VACUUM to eagerly place newly deleted B-Tree pages in the Free > > Space Map. Previously VACUUM could only place preexisting deleted > > pages in the Free Space Map for recycling." > > So, previously it could only place pages that were deleted from the > previous vacuum on the free list? Yes, exactly -- that's how it worked, even though in general there was no way to predict when the next VACUUM would be. Now VACUUM will usually manage to do everything in the same VACUUM operation, starting from deleting the index tuples that make the page empty, then deleting the page, and finally putting the page in the FSM. Makes a big difference when the workload has lots of page deletions, say due to range deletions. -- Peter Geoghegan
On Tue, May 11, 2021 at 09:26:36AM +0530, Amit Kapila wrote: > On Mon, May 10, 2021 at 7:30 PM Justin Pryzby <pryzby@telsasoft.com> wrote: > > > > Can these be merged: > > Allow logical replication to stream long transactions to standbys (Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke) > > Improve the logical replication API to allow streaming large in-progress transactions (Tomas Vondra, Dilip Kumar, AmitKapila) > > > > I think we should merge the below two: > Add support for streaming to built-in logical replication (Vondra, > Dilip Kumar, Amit Kapila, Ajin Cherian, Nikhil Sontakke, Stas Kelvich) > > Allow logical replication to stream long transactions to standbys > (Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke) > > and say something like: > Allow logical replication to stream long in-progress transactions to standbys > Previously transactions that exceeded logical_decoding_work_mem were > written to disk until the transaction completed. OK, updated as you suggested. > Also, the commits that lead to the above feature is: > 2020-09-03 [464824323] Add support for streaming to built-in logical replicatio > 2020-08-08 [7259736a6] Implement streaming mode in ReorderBuffer. Yes, these were already listed above that item. > * Improve the logical replication API to allow streaming large > in-progress transactions (Tomas Vondra, Dilip Kumar, Amit Kapila) > The output functions begin with "stream". test_decoding also supports these. > > In the above, I think it is better to say Enhance instead of Improve. OK, word changed. > * Add system view pg_stat_replication_slots to report replication slot > activity (Sawada Masahiko, Amit Kapila) > > I think for this item, we can give Vignesh C as author credit because > he helped a lot enhance/stabilize this feature after the initial > commit. See > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=f5fc2f5b23d1b1dff60f8ca5dc211161df47eda4 > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=cca57c1d9bf7eeba5b81115e0b82651cf3d8e4ea > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=592f00f8dec68038301467a904ac514eddabf6cd OK, I have added him: Add system view pg_stat_replication_slots to report replication slot activity (Sawada Masahiko, Amit Kapila, Vignesh C) > * Allow multiple xacts during table sync in logical replication (Peter > Smith, Amit Kapila, and Takamichi Osumi) > > IMPORTANT? > > Yes, this is a big enough improvement in the initial table sync > mechanism of logical replication that it is worth mentioning in > Release Notes. One of the main user-visible effects this commit can > have is that it will reduce the time for which we need to hold WAL on > the server during the initial table sync phase. Then it can avoid > large copies in case there is any error during the time when > apply-worker and tablesync-worker synchronizes the apply location. OK, thanks. > * Enable logical replication to handle two phase commits (Ajin Cherian) > This is controlled via pg_create_logical_replication_slot(). > > We still haven't supported two-phase commits for built-in logical > replication but we have exposed APIs for decoding plugins to achieve > that. Using these APIs, we are planning to support it for built-in > logical replication but that will be probably done in PG-15. So, I > suggest we can modify it to something like: > > * Enhance logical decoding APIs to handle two-phase commits > This is controlled via pg_create_logical_replication_slot(). OK, updated as you suggested. > Most of the work for this feature is done via commits: > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=a271a1b50e9bec07e2ef3a05e38e7285113e4ce6 > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=0aa8a01d04c8fe200b7a106878eebc3d0af9105c > > So, I think the author credit should be given to Ajin Cherian, Amit > Kapila, Nikhil Sontakke, and Stas Kelvich OK, the new item text is: <listitem> <!-- Author: Amit Kapila <akapila@postgresql.org> 2020-12-30 [0aa8a01d0] Extend the output plugin API to allow decoding of prepar Author: Amit Kapila <akapila@postgresql.org> 2021-01-04 [a271a1b50] Allow decoding at prepare time in ReorderBuffer. Author: Amit Kapila <akapila@postgresql.org> 2021-03-03 [19890a064] Add option to enable two_phase commits via pg_create_log --> <para> Enhance logical decoding APIs to handle two-phase commits (Ajin Cherian, Amit Kapila, Nikhil Sontakke, Stas Kelvich) </para> <para> This is controlled via pg_create_logical_replication_slot(). </para> </listitem> > * I think we should mention about commit: > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=d7eb52d7181d83cf2363570f7a205b8eb1008dbc > > This allows Logical decoding to work efficiently in presence of a > large amount of DDLs. This was a complaint from many users for long > time. Apart from the discussion thread, see some of the earlier > complaints: > https://www.postgresql.org/message-id/flat/20f3de7675f83176253f607b5e199b228406c21c.camel@cybertec.at > https://www.postgresql.org/message-id/flat/CAHoiPjzea6N0zuCi=+f9v_j94nfsy6y8SU7-=bp4=7qw6_i=Rg@mail.gmail.com > > and most recently at: > https://www.postgresql.org/message-id/DB8PR10MB36126BC0DC3570637FDFED12EB4F9%40DB8PR10MB3612.EURPRD10.PROD.OUTLOOK.COM OK, I added this: <listitem> <!-- Author: Amit Kapila <akapila@postgresql.org> 2020-10-15 [d7eb52d71] Execute invalidation messages for each XLOG_XACT_INVALID --> <para> Allow logical decoding to more efficently process cache invalidation messages </para> <para> This allows Logical decoding to work efficiently in presence of a large amount of DDL. </para> </listitem> > * Speed truncation of small tables on large shared buffer servers (Kirk Jamison) > > This optimization applies during recovery. Shouldn't we mention that > in some way? Yes, new text: Speed truncation of small tables during recovery on clusters with a large number of shared buffers (Kirk Jamison) -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 08:08:01AM -0700, Peter Geoghegan wrote:
> On Mon, May 10, 2021 at 7:00 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > | Allow VACUUM VERBOSE to report page deletion counts for each scan of an index (Peter Geoghegan)
> >
> > I think "Allow" is wrong - should just say that VACUUM VERBOSE reports..
>
> It's also not accurate, since the count of deleted pages was always
> shown by VACUUM VERBOSE (once per index scan). The new feature has us
> show pages deleted by the VACUUM that actually ran (not some earlier
> VACUUM) -- these are "newly deleted pages".
>
> I don't think that this item is worth mentioning, though -- it's just
> a nice to have. If Bruce removes it from the release notes entirely I
> won't object.
>
> In addition to the items that I commented on in my response to
> Matthias just now, I should point out the following item as worthy of
> inclusion:
>
> 9dd963ae25 Recycle nbtree pages deleted during same VACUUM.
>
> I suggest that this item be phrased more or less as follows:
>
> "Allow VACUUM to eagerly place newly deleted B-Tree pages in the Free
> Space Map. Previously VACUUM could only place preexisting deleted
> pages in the Free Space Map for recycling."
OK, god it. New text is:
<listitem>
<!--
Author: Peter Geoghegan <pg@bowt.ie>
2021-03-21 [9dd963ae2] Recycle nbtree pages deleted during same VACUUM.
-->
<para>
Allow VACUUM to eagerly add newly deleted btree pages in the free
space map (Peter Geoghegan)
</para>
<para>
Previously VACUUM could only place preexisting deleted pages in
the free space map.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Tue, May 11, 2021 at 09:13:31AM -0400, Jonathan Katz wrote: > First Bruce, thank you for compiling the release notes draft. Comments > inline: > > On 5/10/21 11:56 PM, Amit Kapila wrote: > > On Mon, May 10, 2021 at 7:30 PM Justin Pryzby <pryzby@telsasoft.com> wrote: > >> > >> Can these be merged: > >> Allow logical replication to stream long transactions to standbys (Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke) > >> Improve the logical replication API to allow streaming large in-progress transactions (Tomas Vondra, Dilip Kumar, AmitKapila) > >> > > > > I think we should merge the below two: > > Add support for streaming to built-in logical replication (Vondra, > > Dilip Kumar, Amit Kapila, Ajin Cherian, Nikhil Sontakke, Stas Kelvich) > > > > Allow logical replication to stream long transactions to standbys > > (Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke) > > > > and say something like: > > Allow logical replication to stream long in-progress transactions to standbys > > Previously transactions that exceeded logical_decoding_work_mem were > > written to disk until the transaction completed. > > I was going to post a question about the above bullet point(s), so this > clarification helps -- thanks! > > Two comments on the above: > > 1. I do believe ther eis a missing "Tomas" in the first release note. Yes, I saw that and fixed, thanks. > 2. Would you want to s/standbys/subscribers/ given an instance can be in > read/write mode but still receive logical changes? Or did this patch > affect only subscribers that are also standbys? I don't know. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Tue, May 11, 2021 at 08:24:22AM -0500, Justin Pryzby wrote: > I think these should be merged: > > | Remove factorial operators ! and !! (Mark Dilger) > | Add documentation for the factorial() function (Peter Eisentraut) > > | This can be disabled by turning client options "sslsni" off. > > option (no ess) Fixed. > | Allow REINDEX to process all child tables and indexes of a partitioned table (Justin Pryzby, Michael Paquier) > > It should actually say "all child tables or indexes of a partitioned relation", > since you can REINDEX INDEX partitioned_index. OK, fixed, good point. I had not thought of indexes. > | Add bit_xor XOR aggregate function (Alexey Bashtanov) > > Remove XOR ? Uh, I thought XOR was describing the function, since the name might not be clear. No? > | Remove support for postfix (right-unary) operators (Mark Dilger) > | pg_dump and pg_upgrade will warn if post-fix operators are being dumped. > > postfix not post-fix Oops, fixed. :-) > | Remove temporary files after backend crashes (Euler Taveira) > > crash (singular) Uh, I thought I would say "all crashes" here. > | Increase warning time and hard limit before transaction id and multi-transaction wraparound (Noah Misch) > | This should reduce the number of failures without warning. > > I'd say: This should reduce the possibility of failures that occur without > having issued warnings about wraparound. OK, added, thanks. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On 5/11/21 5:12 PM, Bruce Momjian wrote: > On Tue, May 11, 2021 at 09:13:31AM -0400, Jonathan Katz wrote: >> First Bruce, thank you for compiling the release notes draft. Comments >> inline: >> >> On 5/10/21 11:56 PM, Amit Kapila wrote: >>> and say something like: >>> Allow logical replication to stream long in-progress transactions to standbys >>> Previously transactions that exceeded logical_decoding_work_mem were >>> written to disk until the transaction completed. >> > >> 2. Would you want to s/standbys/subscribers/ given an instance can be in >> read/write mode but still receive logical changes? Or did this patch >> affect only subscribers that are also standbys? > > I don't know. That question was geared more towards Amit as he proposed the revised text :) Thanks, Jonathan
Attachment
.On Tue, May 11, 2021 at 11:31 PM Bruce Momjian <bruce@momjian.us> wrote:
> On Tue, May 11, 2021 at 01:16:38PM +0300, Alexander Korotkov wrote:
> > > OK, what symbols trigger this change? Underscore? What else?
> >
> > Any symbol, which is recognized as a separator by full-text parser,
> > but not tsquery parser. Fulltext search is extensible and allowing
> > pluggable parsers. In principle, we could dig the exact set of
> > symbols, but I'm not sure this worth the effort.
> >
> > > You are
> > > saying the previous code allowed 'pg' and 'class' anywhere in the
> > > string, while the new code requires them to be adjacent, which more
> > > closely matches the pattern.
> >
> > Yes, that's it.
> >
> > > > * Fix extra distance in phrase operators for quoted text in
> > > > websearch_to_tsquery() (Alexander Korotkov)
> > > > For example, websearch_to_tsquery('english', '"aaa: bbb"') becomes
> > > > 'aaa <> bbb' instead of 'aaa <2> bbb'.
> > >
> > > So colon and space were considered to be two tokens between 'aaa' and
> > > 'bbb', while is really only one because both tokens are discarded? Is
> > > this true of any discarded tokens, e.g. ''"aaa ?:, bbb"'?
> >
> > Yes, that's true for any discarded tokens.
>
> I can up with this text for these two items. I think it still needs ro
> be more specific:
>
> <listitem>
> <!--
> Author: Alexander Korotkov <akorotkov@postgresql.org>
> 2021-01-31 [0c4f355c6] Fix parsing of complex morphs to tsquery
> -->
>
> <para>
> Fix to_tsquery() and websearch_to_tsquery() to properly parse
> certain discarded tokens in quotes (Alexander Korotkov)
> </para>
This relates not just to quotes. Original problem relates to quotes
in websearch_to_tsquery() and phrase operator in to_tsquery(). But
the solution changes output for all query operands containing
discarded tokens.
Could we try this?
Make to_tsquery() and websearch_to_tsquery() produce more strict
output for query parts containing discarded tokens. In particular,
this makes to_tsquery() and websearch_to_tsquery() properly parse the
discarded tokens in phrase search operands and quotes correspondingly.
> <para>
> Certain discarded tokens, like underscore, caused the output
> of these functions to produce incorrect tsquery output, e.g.,
> websearch_to_tsquery('"pg_class pg"') used to output '( pg &
> class ) <-> pg', but now outputs 'pg <-> class <-> pg'.
> </para>
> </listitem>
This part looks good to me. I'd just suggest to extend the example to
to_tsquery() as well.
Certain discarded tokens, like underscore, caused the output of these
functions to produce incorrect tsquery output, e.g., both
websearch_to_tsquery('"pg_class pg"') and to_tsquery('pg_class <->
pg') used to output '( pg & class ) <-> pg', but now both output 'pg
<-> class <-> pg'.
> <listitem>
> <!--
> Author: Alexander Korotkov <akorotkov@postgresql.org>
> 2021-05-03 [eb086056f] Make websearch_to_tsquery() parse text in quotes as a si
> -->
>
> <para>
> Fix websearch_to_tsquery() to properly parse multiple adjacent
> discarded tokens in quotes (Alexander Korotkov)
> </para>
>
> <para>
> Previously, quoted text that contained multiple adjacent discarded
> tokens were treated as multiple tokens, causing incorrect tsquery
> output, e.g., websearch_to_tsquery('"aaa: bbb"') used to output
> 'aaa <2> bbb', but now outputs 'aaa <-> bbb'.
> </para>
> </listitem>
This item looks good to me.
------
Regards,
Alexander Korotkov
Thanks you for the review. I did skip these items based on the criteria I normally use, but it is good for people to review this list to see if my analysis or criteria is wrong. --------------------------------------------------------------------------- On Mon, May 10, 2021 at 09:40:45AM -0500, Justin Pryzby wrote: > Same as the last couple years, I checked for missing items in the release > notes, running something like this. > > git log --cherry-pick --oneline origin/REL_13_STABLE...origin/master > > Should any of these be included? > > f82de5c46b Do COPY FROM encoding conversion/verification in larger chunks. > 9e596b65f4 Add "LP_DEAD item?" column to GiST pageinspect functions > > 10a5b35a00 Report resource usage at the end of recovery > 7e453634bb Add additional information in the vacuum error context. > 1ea396362b Improve logging of bad parameter values in BIND messages. > > 86dc90056d Rework planning and execution of UPDATE and DELETE. > a1115fa078 Postpone some more stuff out of ExecInitModifyTable. > c5b7ba4e67 Postpone some stuff out of ExecInitModifyTable. > > 7db0cd2145 Set PD_ALL_VISIBLE and visibility map bits in COPY FREEZE > 01e658fa74 Hash support for row types > a929e17e5a Allow run-time pruning on nested Append/MergeAppend nodes > 8c15a29745 Allow ALTER TYPE to update an existing type's typsubscript value. > c7aba7c14e Support subscripting of arbitrary types, not only arrays. > > 7b94e99960 Remove catalog function currtid() > 926fa801ac Remove undocumented IS [NOT] OF syntax. > cd9c1b3e19 Rename PGPROC->vacuumFlags to statusFlags > a04daa97a4 Remove es_result_relation_info from EState. > > 3d351d916b Redefine pg_class.reltuples to be -1 before the first VACUUM or ANALYZE. > fea10a6434 Rename VariableCacheData.nextFullXid to nextXid. > 9de9294b0c Stop archive recovery if WAL generated with wal_level=minimal is found. (see also 15251c0a6) > > f40c6969d0 Routine usage information schema tables > > b4af70cb21 Simplify state managed by VACUUM. > 4753ef37e0 Use a WaitLatch for vacuum/autovacuum sleeping > 9dd963ae25 Recycle nbtree pages deleted during same VACUUM. > 3c3b8a4b26 Truncate line pointer array during VACUUM. > > ad1c36b070 Fix foreign-key selectivity estimation in the presence of constants. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Tue, May 11, 2021 at 11:57:10AM +0900, Amit Langote wrote: > On Mon, May 10, 2021 at 11:40 PM Justin Pryzby <pryzby@telsasoft.com> wrote: > > Same as the last couple years, I checked for missing items in the release > > notes, running something like this. > > > > git log --cherry-pick --oneline origin/REL_13_STABLE...origin/master > > > > Should any of these be included? > > > > 86dc90056d Rework planning and execution of UPDATE and DELETE. > > a1115fa078 Postpone some more stuff out of ExecInitModifyTable. > > c5b7ba4e67 Postpone some stuff out of ExecInitModifyTable. > > I was just about to ask Bruce what he thinks about these. > > To clarify, the first one is a big refactoring commit that allowed us > to get rid of inheritance_planner(), a fairly inefficient way of > planning updates/deletes on partitioned tables, especially when many > partitions remain after pruning (or when pruning cannot be used). One > may see the performance of update/deletes, especially on partitioned > tables, to be generally improved as a result of this commit, but maybe > not as significantly as to be mentioned in E.1.3.1.1. Partitioning or > even E.1.3.1.4. General Performance. However, one user-visible > feature that came out of this work is that updates/deletes can now use > run-time pruning whereas they couldn't before. Maybe that ought to be > mentioned. (This reminds me to send a patch to remove the note from > 5.11.4. Partition Pruning that says that runtime pruning cannot be > used for update/delete). > > The other two commits can lead to improved performance of > update/deletes when there are many unpruned partitions in the plan, > but runtime pruning (a new feature as mentioned above) leads to only > one or few partitions to actually be updated/deleted from. I admit > though that the cases for which performance has been improved still > under-perform the cases that already performed better starting in v12, > that is, the cases where the planner itself is able to trim down the > plan to contain one or few partitions, so maybe nothing very big to > see here just yet. You may want to take a look at the benchmark > results I had posted here: > https://www.postgresql.org/message-id/CA%2BHiwqEcawatEaUh1uTbZMEZTJeLzbroRTz9_X9Z5CFjTWJkhw%40mail.gmail.com Seems we might want to have a general release note item that mentions improved update/delete performance for partitioned tables, yes? I think the run-time pruning and single-parition pullup are significant. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 07:53:08AM -0700, Peter Geoghegan wrote: > On Sun, May 9, 2021 at 11:03 PM Bruce Momjian <bruce@momjian.us> wrote: > > I have committed the first draft of the PG 14 release notes. > > This definitely isn't necessary, since the commit in question was a > totally mechanical thing that cleaned up a minor inconsistency: > > Initialize work_mem and maintenance_work_mem using current guc.c > default (Peter Geoghegan) > > Oversight in commit 848ae330a49, which increased the previous defaults > for work_mem and maintenance_work_mem by 4X. IS THIS A BEHAVIORAL > CHANGE? OK, that's what I thought --- removed. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On 2021-May-11, Bruce Momjian wrote: > > 86dc90056d Rework planning and execution of UPDATE and DELETE. > > a1115fa078 Postpone some more stuff out of ExecInitModifyTable. > > c5b7ba4e67 Postpone some stuff out of ExecInitModifyTable. Are these the set that means UPDATE/DELETE can now do partition pruning? I think that's a significant improvement, because table partitioning becomes usable in situations where it previously wasn't, so let's mention it. > > c7aba7c14e Support subscripting of arbitrary types, not only arrays. There's an entry that says we allow subscripting of jsonb, but the feature is more general than that: our subscripting is now extensible, and jsonb is just the first implementation. I think this should be mentioned. > > f40c6969d0 Routine usage information schema tables This is one small step towards compliance, I think, so it should be mentioned. -- Álvaro Herrera 39°49'30"S 73°17'W
On Tue, May 11, 2021 at 10:35:23AM -0400, Bruce Momjian wrote: > > | Allow more than the common name (CN) to be matched for client certificate authentication (Andrew Dunstan) > > Your description makes it sound like arbitrary attributes can be compared. But > > the option just allows comparing CN or DN. > > OK, new text is: > > <para> > Allow the certificate's distinguished name (DN) to be matched for client > certificate authentication (Andrew Dunstan) > </para> > > <para> > The new pg_hba.conf keyword "clientname=DN" allows comparison with > non-CN certificate attributes and can be combined with ident maps. > </para> > </listitem> I think this part is still misleading. The option just allows DN/CN, so it's strange to say "non-CN attributes". > > | Add date_bin function (John Naylor) > > This truncate timestamps on an arbitrary interval. > > Like date_trunc() but also supports eg. '15 minutes', and also uses an arbitrary "origin". > > OK, so what I think it returns is the greatest datetime that is a > multiple of interval values added to origin which is not greater than > the target date, right? Am I the only one who finds this unclear? > Doesn't our documentation of this feature need to explain this? I think the documentation is okay, myself: https://www.postgresql.org/docs/devel/functions-datetime.html#FUNCTIONS-DATETIME-BIN | The function date_bin “bins” the input timestamp into the specified interval (the stride)... Anyway, the release notes have to be at least as succiently as that. +Allow VACUUM to eagerly add newly deleted btree pages in the free space map (Peter Geoghegan) say added "to" the FSM -- Justin
On Wed, May 12, 2021 at 12:19:37AM +0300, Alexander Korotkov wrote:
> This relates not just to quotes. Original problem relates to quotes
> in websearch_to_tsquery() and phrase operator in to_tsquery(). But
> the solution changes output for all query operands containing
> discarded tokens.
>
> Could we try this?
>
> Make to_tsquery() and websearch_to_tsquery() produce more strict
> output for query parts containing discarded tokens. In particular,
> this makes to_tsquery() and websearch_to_tsquery() properly parse the
> discarded tokens in phrase search operands and quotes correspondingly.
> > <para>
> > Certain discarded tokens, like underscore, caused the output
> > of these functions to produce incorrect tsquery output, e.g.,
> > websearch_to_tsquery('"pg_class pg"') used to output '( pg &
> > class ) <-> pg', but now outputs 'pg <-> class <-> pg'.
> > </para>
> > </listitem>
>
> This part looks good to me. I'd just suggest to extend the example to
> to_tsquery() as well.
>
> Certain discarded tokens, like underscore, caused the output of these
> functions to produce incorrect tsquery output, e.g., both
> websearch_to_tsquery('"pg_class pg"') and to_tsquery('pg_class <->
> pg') used to output '( pg & class ) <-> pg', but now both output 'pg
> <-> class <-> pg'.
OK, I went with this:
<listitem>
<!--
Author: Alexander Korotkov <akorotkov@postgresql.org>
2021-01-31 [0c4f355c6] Fix parsing of complex morphs to tsquery
-->
<para>
Fix to_tsquery() and websearch_to_tsquery() to properly parse
query text containing discarded tokens (Alexander Korotkov)
</para>
<para>
Certain discarded tokens, like underscore, caused the output of
these functions to produce incorrect tsquery output, e.g., both
websearch_to_tsquery('"pg_class pg"') and to_tsquery('pg_class
<-> pg') used to output '( pg & class ) <-> pg',
but now both output 'pg <-> class <-> pg'.
</para>
</listitem>
> > <listitem>
> > <!--
> > Author: Alexander Korotkov <akorotkov@postgresql.org>
> > 2021-05-03 [eb086056f] Make websearch_to_tsquery() parse text in quotes as a si
> > -->
> >
> > <para>
> > Fix websearch_to_tsquery() to properly parse multiple adjacent
> > discarded tokens in quotes (Alexander Korotkov)
> > </para>
> >
> > <para>
> > Previously, quoted text that contained multiple adjacent discarded
> > tokens were treated as multiple tokens, causing incorrect tsquery
> > output, e.g., websearch_to_tsquery('"aaa: bbb"') used to output
> > 'aaa <2> bbb', but now outputs 'aaa <-> bbb'.
> > </para>
> > </listitem>
>
> This item looks good to me.
Good, thanks.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Tue, May 11, 2021 at 05:13:21PM -0500, Justin Pryzby wrote: > On Tue, May 11, 2021 at 10:35:23AM -0400, Bruce Momjian wrote: > > > | Allow more than the common name (CN) to be matched for client certificate authentication (Andrew Dunstan) > > > Your description makes it sound like arbitrary attributes can be compared. But > > > the option just allows comparing CN or DN. > > > > OK, new text is: > > > > <para> > > Allow the certificate's distinguished name (DN) to be matched for client > > certificate authentication (Andrew Dunstan) > > </para> > > > > <para> > > The new pg_hba.conf keyword "clientname=DN" allows comparison with > > non-CN certificate attributes and can be combined with ident maps. > > </para> > > </listitem> > > I think this part is still misleading. The option just allows DN/CN, so it's > strange to say "non-CN attributes". OK, so this is where I am confused. I searched for distinguished name (DN) and came up with DN being a concatentation of all the fields provided to the certificate signing request (CSR). Is that right? Wouldn't people test _parts_ of the DN, rather than all of it. The test in the patch seems to do that: + "# MAPNAME SYSTEM-USERNAME PG-USERNAME\n", + "dn \"CN=ssltestuser-dn,OU=Testing,OU=Engineering,O=PGDG\" ssltestuser\n", + "dnre \"/^.*OU=Testing,.*\$\" ssltestuser\n", + "cn ssltestuser-dn ssltestuser\n"; I think someone need to explain to me exactly what the DN is and how it is used. Sorry. > > > | Add date_bin function (John Naylor) > > > This truncate timestamps on an arbitrary interval. > > > Like date_trunc() but also supports eg. '15 minutes', and also uses an arbitrary "origin". > > > > OK, so what I think it returns is the greatest datetime that is a > > multiple of interval values added to origin which is not greater than > > the target date, right? Am I the only one who finds this unclear? > > Doesn't our documentation of this feature need to explain this? > > I think the documentation is okay, myself: > https://www.postgresql.org/docs/devel/functions-datetime.html#FUNCTIONS-DATETIME-BIN > | The function date_bin “bins” the input timestamp into the specified interval (the stride)... OK, if everyone else is happy with it, I am fine. I now have: <listitem> <!-- Author: Peter Eisentraut <peter@eisentraut.org> 2021-03-24 [49ab61f0b] Add date_bin function Author: Peter Eisentraut <peter@eisentraut.org> 2021-04-10 [496e58bb0] Improve behavior of date_bin with origin in the future --> <para> Add date_bin function (John Naylor) </para> <para> The function date_bin "bins" the input timestamp into a specified interval aligned with a specified origin. </para> </listitem> > Anyway, the release notes have to be at least as succiently as that. > > +Allow VACUUM to eagerly add newly deleted btree pages in the free space map (Peter Geoghegan) > > say added "to" the FSM Yes, fixed. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Tue, May 11, 2021 at 6:43 PM Jonathan S. Katz <jkatz@postgresql.org> wrote: > > First Bruce, thank you for compiling the release notes draft. Comments > inline: > > On 5/10/21 11:56 PM, Amit Kapila wrote: > > On Mon, May 10, 2021 at 7:30 PM Justin Pryzby <pryzby@telsasoft.com> wrote: > >> > >> Can these be merged: > >> Allow logical replication to stream long transactions to standbys (Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke) > >> Improve the logical replication API to allow streaming large in-progress transactions (Tomas Vondra, Dilip Kumar, AmitKapila) > >> > > > > I think we should merge the below two: > > Add support for streaming to built-in logical replication (Vondra, > > Dilip Kumar, Amit Kapila, Ajin Cherian, Nikhil Sontakke, Stas Kelvich) > > > > Allow logical replication to stream long transactions to standbys > > (Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke) > > > > and say something like: > > Allow logical replication to stream long in-progress transactions to standbys > > Previously transactions that exceeded logical_decoding_work_mem were > > written to disk until the transaction completed. > > I was going to post a question about the above bullet point(s), so this > clarification helps -- thanks! > > Two comments on the above: > > 1. I do believe ther eis a missing "Tomas" in the first release note. > 2. Would you want to s/standbys/subscribers/ given an instance can be in > read/write mode but still receive logical changes? > Right, that makes sense to me. -- With Regards, Amit Kapila.
On Wed, May 12, 2021 at 2:36 AM Bruce Momjian <bruce@momjian.us> wrote: > > > OK, I added this: > > <listitem> > <!-- > Author: Amit Kapila <akapila@postgresql.org> > 2020-10-15 [d7eb52d71] Execute invalidation messages for each XLOG_XACT_INVALID > --> > > <para> > Allow logical decoding to more efficently process cache invalidation messages > </para> > > <para> > This allows Logical decoding to work efficiently in presence of a large amount of DDL. > </para> > </listitem> > Dilip Kumar is the primary author of this work. Thanks for all the updates. -- With Regards, Amit Kapila.
On Wed, May 12, 2021 at 6:23 AM Bruce Momjian <bruce@momjian.us> wrote: > On Tue, May 11, 2021 at 11:57:10AM +0900, Amit Langote wrote: > > On Mon, May 10, 2021 at 11:40 PM Justin Pryzby <pryzby@telsasoft.com> wrote: > > > Same as the last couple years, I checked for missing items in the release > > > notes, running something like this. > > > > > > git log --cherry-pick --oneline origin/REL_13_STABLE...origin/master > > > > > > Should any of these be included? > > > > > > 86dc90056d Rework planning and execution of UPDATE and DELETE. > > > a1115fa078 Postpone some more stuff out of ExecInitModifyTable. > > > c5b7ba4e67 Postpone some stuff out of ExecInitModifyTable. > > > > I was just about to ask Bruce what he thinks about these. > > > > To clarify, the first one is a big refactoring commit that allowed us > > to get rid of inheritance_planner(), a fairly inefficient way of > > planning updates/deletes on partitioned tables, especially when many > > partitions remain after pruning (or when pruning cannot be used). One > > may see the performance of update/deletes, especially on partitioned > > tables, to be generally improved as a result of this commit, but maybe > > not as significantly as to be mentioned in E.1.3.1.1. Partitioning or > > even E.1.3.1.4. General Performance. However, one user-visible > > feature that came out of this work is that updates/deletes can now use > > run-time pruning whereas they couldn't before. Maybe that ought to be > > mentioned. (This reminds me to send a patch to remove the note from > > 5.11.4. Partition Pruning that says that runtime pruning cannot be > > used for update/delete). > > > > The other two commits can lead to improved performance of > > update/deletes when there are many unpruned partitions in the plan, > > but runtime pruning (a new feature as mentioned above) leads to only > > one or few partitions to actually be updated/deleted from. I admit > > though that the cases for which performance has been improved still > > under-perform the cases that already performed better starting in v12, > > that is, the cases where the planner itself is able to trim down the > > plan to contain one or few partitions, so maybe nothing very big to > > see here just yet. You may want to take a look at the benchmark > > results I had posted here: > > https://www.postgresql.org/message-id/CA%2BHiwqEcawatEaUh1uTbZMEZTJeLzbroRTz9_X9Z5CFjTWJkhw%40mail.gmail.com > > Seems we might want to have a general release note item that mentions > improved update/delete performance for partitioned tables, yes? Yes, that would not be entirely wrong. The cases that benefit more from these commits are those where the planner is not able to prune many (or any) partitions, such as, for example, if a generic plan is chosen for prepared update/delete queries. The cases where the planner *is* able to prune most partitions should run faster with v14 but only marginally; that case was already extensively improved by the work done in v12 release. > I think > the run-time pruning and single-parition pullup are significant. Assuming, by "single-partition pullup", you mean run-time pruning now allows the executor to touch only one affected partition out of the many contained in a update/delete plan, then yes, that's new in v14. -- Amit Langote EDB: http://www.enterprisedb.com
On Wed, May 12, 2021 at 6:47 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote: > On 2021-May-11, Bruce Momjian wrote: > > > > 86dc90056d Rework planning and execution of UPDATE and DELETE. > > > a1115fa078 Postpone some more stuff out of ExecInitModifyTable. > > > c5b7ba4e67 Postpone some stuff out of ExecInitModifyTable. > > Are these the set that means UPDATE/DELETE can now do partition pruning? > I think that's a significant improvement, because table partitioning > becomes usable in situations where it previously wasn't, so let's > mention it. Thanks. To be clear, update/delete can now do *run-time* partition pruning, while plan-time pruning has already been working since v12 (commit 428b260f87). How? The plan for update/delete now looks very similar to that for an equivalent select query -- Append/MergeAppend to scan partitions, with ModifyTable on top to do the actual update/delete -- and because Append/MergeAppend nodes carry run-time pruning info, update/delete now get that ability. -- Amit Langote EDB: http://www.enterprisedb.com
On Tue, May 11, 2021 at 10:45:04PM -0400, Bruce Momjian wrote: > On Tue, May 11, 2021 at 05:13:21PM -0500, Justin Pryzby wrote: > > On Tue, May 11, 2021 at 10:35:23AM -0400, Bruce Momjian wrote: > > > > | Allow more than the common name (CN) to be matched for client certificate authentication (Andrew Dunstan) > > > > Your description makes it sound like arbitrary attributes can be compared. But > > > > the option just allows comparing CN or DN. > > > > > > OK, new text is: > > > > > > <para> > > > Allow the certificate's distinguished name (DN) to be matched for client > > > certificate authentication (Andrew Dunstan) > > > </para> > > > > > > <para> > > > The new pg_hba.conf keyword "clientname=DN" allows comparison with > > > non-CN certificate attributes and can be combined with ident maps. > > > </para> > > > </listitem> > > > > I think this part is still misleading. The option just allows DN/CN, so it's > > strange to say "non-CN attributes". > > OK, so this is where I am confused. I searched for distinguished name > (DN) and came up with DN being a concatentation of all the fields > provided to the certificate signing request (CSR). Is that right? > Wouldn't people test _parts_ of the DN, rather than all of it. +Andrew The full DN is probably not the postgres username, so the docs suggest that: | This option is probably best used in conjunction with a username map. You're right that clientname=DN allows testing *parts*, of the DN, but I don't know if there's any reason to believe that's the typical use case. The primary utility of clientname=DN seems to be that the CN alone is (or can be) ambiguous - matching on the full DN is intended to resolve that. I think the release notes should focus on this. Matching parts of the DN (other than the CN) seems like a 2ndary use. Maybe a variation on your original words is better. | Allow the distinguished name (DN) to be matched for client certificate authentication (Andrew Dunstan) | Previously, matching was done only the common name (DN). | With a username map, the DN can be matched in full or in part. > The test in the patch seems to do that: > > + "# MAPNAME SYSTEM-USERNAME PG-USERNAME\n", > + "dn \"CN=ssltestuser-dn,OU=Testing,OU=Engineering,O=PGDG\" ssltestuser\n", > + "dnre \"/^.*OU=Testing,.*\$\" ssltestuser\n", > + "cn ssltestuser-dn ssltestuser\n"; > > I think someone need to explain to me exactly what the DN is and how it > is used. Sorry. -- Justin
On Wed, May 12, 2021 at 09:15:55AM +0530, Amit Kapila wrote: > On Tue, May 11, 2021 at 6:43 PM Jonathan S. Katz <jkatz@postgresql.org> wrote: > > I was going to post a question about the above bullet point(s), so this > > clarification helps -- thanks! > > > > Two comments on the above: > > > > 1. I do believe ther eis a missing "Tomas" in the first release note. > > 2. Would you want to s/standbys/subscribers/ given an instance can be in > > read/write mode but still receive logical changes? > > > > Right, that makes sense to me. Change made. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Wed, May 12, 2021 at 09:18:27AM +0530, Amit Kapila wrote: > On Wed, May 12, 2021 at 2:36 AM Bruce Momjian <bruce@momjian.us> wrote: > > > > > > OK, I added this: > > > > <listitem> > > <!-- > > Author: Amit Kapila <akapila@postgresql.org> > > 2020-10-15 [d7eb52d71] Execute invalidation messages for each XLOG_XACT_INVALID > > --> > > > > <para> > > Allow logical decoding to more efficently process cache invalidation messages > > </para> > > > > <para> > > This allows Logical decoding to work efficiently in presence of a large amount of DDL. > > </para> > > </listitem> > > > > Dilip Kumar is the primary author of this work. > > Thanks for all the updates. Thanks, added. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Wed, May 12, 2021 at 10:06:05AM -0500, Justin Pryzby wrote:
> On Tue, May 11, 2021 at 10:45:04PM -0400, Bruce Momjian wrote:
> > OK, so this is where I am confused. I searched for distinguished name
> > (DN) and came up with DN being a concatentation of all the fields
> > provided to the certificate signing request (CSR). Is that right?
> > Wouldn't people test _parts_ of the DN, rather than all of it.
>
> +Andrew
>
> The full DN is probably not the postgres username, so the docs suggest that:
> | This option is probably best used in conjunction with a username map.
>
> You're right that clientname=DN allows testing *parts*, of the DN, but I don't
> know if there's any reason to believe that's the typical use case.
>
> The primary utility of clientname=DN seems to be that the CN alone is (or can
> be) ambiguous - matching on the full DN is intended to resolve that. I think
> the release notes should focus on this.
OK, that matches what I thought. Here is my updated version:
<listitem>
<!--
Author: Andrew Dunstan <andrew@dunslane.net>
2021-03-29 [6d7a6feac] Allow matching the DN of a client certificate for
authen
-->
<para>
Allow the certificate's distinguished name (DN) to be matched for client
certificate authentication (Andrew Dunstan)
</para>
<para>
--> The new pg_hba.conf keyword "clientname=DN" allows comparison with
--> certificate attributes beyond the CN and can be combined with ident
--> maps.
</para>
</listitem>
Technically these are attributes of the certificate signing request
(CSR), but I didn't want to mention that here.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Tue, May 11, 2021 at 11:57:10AM +0900, Amit Langote wrote: > On Mon, May 10, 2021 at 11:40 PM Justin Pryzby <pryzby@telsasoft.com> wrote: > > 86dc90056d Rework planning and execution of UPDATE and DELETE. > > a1115fa078 Postpone some more stuff out of ExecInitModifyTable. > > c5b7ba4e67 Postpone some stuff out of ExecInitModifyTable. > > I was just about to ask Bruce what he thinks about these. > > To clarify, the first one is a big refactoring commit that allowed us > to get rid of inheritance_planner(), a fairly inefficient way of > planning updates/deletes on partitioned tables, especially when many > partitions remain after pruning (or when pruning cannot be used). One > may see the performance of update/deletes, especially on partitioned > tables, to be generally improved as a result of this commit, but maybe > not as significantly as to be mentioned in E.1.3.1.1. Partitioning or > even E.1.3.1.4. General Performance. However, one user-visible > feature that came out of this work is that updates/deletes can now use > run-time pruning whereas they couldn't before. Maybe that ought to be > mentioned. (This reminds me to send a patch to remove the note from > 5.11.4. Partition Pruning that says that runtime pruning cannot be > used for update/delete). > > The other two commits can lead to improved performance of > update/deletes when there are many unpruned partitions in the plan, > but runtime pruning (a new feature as mentioned above) leads to only > one or few partitions to actually be updated/deleted from. I admit > though that the cases for which performance has been improved still > under-perform the cases that already performed better starting in v12, > that is, the cases where the planner itself is able to trim down the > plan to contain one or few partitions, so maybe nothing very big to > see here just yet. You may want to take a look at the benchmark > results I had posted here: > https://www.postgresql.org/message-id/CA%2BHiwqEcawatEaUh1uTbZMEZTJeLzbroRTz9_X9Z5CFjTWJkhw%40mail.gmail.com OK, I added this entry: <listitem> <!-- Author: Tom Lane <tgl@sss.pgh.pa.us> 2021-03-31 [86dc90056] Rework planning and execution of UPDATE and DELETE. Author: Tom Lane <tgl@sss.pgh.pa.us> 2021-04-06 [a1115fa07] Postpone some more stuff out of ExecInitModifyTable. Author: Tom Lane <tgl@sss.pgh.pa.us> 2021-04-06 [c5b7ba4e6] Postpone some stuff out of ExecInitModifyTable. --> <para> Improve the performance of updates/deletes on partitioned tables when only a few partitions are affected (Amit Langote, Tom Lane) </para> <para> This also allows run-time pruning of updates/deletes on partitioned tables. </para> </listitem> -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Tue, May 11, 2021 at 05:47:27PM -0400, Álvaro Herrera wrote:
> On 2021-May-11, Bruce Momjian wrote:
>
> > > 86dc90056d Rework planning and execution of UPDATE and DELETE.
> > > a1115fa078 Postpone some more stuff out of ExecInitModifyTable.
> > > c5b7ba4e67 Postpone some stuff out of ExecInitModifyTable.
>
> Are these the set that means UPDATE/DELETE can now do partition pruning?
> I think that's a significant improvement, because table partitioning
> becomes usable in situations where it previously wasn't, so let's
> mention it.
>
> > > c7aba7c14e Support subscripting of arbitrary types, not only arrays.
>
> There's an entry that says we allow subscripting of jsonb, but the
> feature is more general than that: our subscripting is now extensible,
> and jsonb is just the first implementation. I think this should be
> mentioned.
OK, I added it into the existing entry:
<listitem>
<!--
Author: Alexander Korotkov <akorotkov@postgresql.org>
2021-01-31 [676887a3b] Implementation of subscripting for jsonb
Author: Alexander Korotkov <akorotkov@postgresql.org>
2021-01-31 [81fcc72e6] Filling array gaps during jsonb subscripting
Author: Alexander Korotkov <akorotkov@postgresql.org>
2021-01-31 [aa6e46daf] Throw error when assigning jsonb scalar instead of a com
-->
<para>
Allow subscripting of jsonb (Dmitry Dolgov)
</para>
<para>
Subscripting can be used to extract from and assign to jsonb documents.
Subscripting is now extensible so extensions and built-in data type can make use of this.
</para>
</listitem>
> > > f40c6969d0 Routine usage information schema tables
>
> This is one small step towards compliance, I think, so it should be
> mentioned.
I added this item:
<listitem>
<!--
Author: Peter Eisentraut <peter@eisentraut.org>
2021-02-17 [f40c6969d] Routine usage information schema tables
-->
<para>
Implement information schema view routine_column_usage to track columns
referenced by function and procedure default expressions (Peter Eisentraut)
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On 2021-May-12, Bruce Momjian wrote:
> OK, I added it into the existing entry:
>
> <listitem>
> <!--
> Author: Alexander Korotkov <akorotkov@postgresql.org>
> 2021-01-31 [676887a3b] Implementation of subscripting for jsonb
> Author: Alexander Korotkov <akorotkov@postgresql.org>
> 2021-01-31 [81fcc72e6] Filling array gaps during jsonb subscripting
> Author: Alexander Korotkov <akorotkov@postgresql.org>
> 2021-01-31 [aa6e46daf] Throw error when assigning jsonb scalar instead of a com
> -->
>
> <para>
> Allow subscripting of jsonb (Dmitry Dolgov)
> </para>
>
> <para>
> Subscripting can be used to extract from and assign to jsonb documents.
> Subscripting is now extensible so extensions and built-in data type can make use of this.
> </para>
> </listitem>
I dunno, this seems to approach the feature from the specifics rather
than its full generality. I would have said something like
Make subscripting user-extensible for "container" types
<!-- maybe "array-like types" -->
Subscripting JSONB is now integrated in core, and adding user-defined
subscripting for other types is possible.
--
Álvaro Herrera Valdivia, Chile
Bob [Floyd] used to say that he was planning to get a Ph.D. by the "green
stamp method," namely by saving envelopes addressed to him as 'Dr. Floyd'.
After collecting 500 such letters, he mused, a university somewhere in
Arizona would probably grant him a degree. (Don Knuth)
On 2021-May-10, Bruce Momjian wrote: > I reworeded it to: > > <listitem> > <!-- > Author: Peter Eisentraut <peter@eisentraut.org> > 2020-06-10 [c7eab0e97] Change default of password_encryption to scram-sha-256 > --> > > <para> > Limit the ways password_encryption can enable md5 hashing (Peter Eisentraut) > </para> > > <para> > Previously on/true/yes/1 values enabled md5. Now, only the string md5 does this. > </para> > </listitem> > > I also have this entry: > > <listitem> > <!-- > Author: Peter Eisentraut <peter@eisentraut.org> > 2020-06-10 [c7eab0e97] Change default of password_encryption to scram-sha-256 > --> > > <para> > Change password_encryption's default to scram-sha-256 (Peter Eisentraut) > </para> > > <para> > Previously it was md5. > </para> > </listitem> > > I am hesitant to merge them since they are different changes. Different? The commit IDs look pretty similar to me, not to say exactly identical. Reading these, it's pretty unclear what password_encryption *is* in the first place. The item should say "the password_encryption server parameter" -- that's what we have in the pg10 note for precursor commit eb61136dc. I would suggest <para> Change the default of the password_encryption server parameter to scram-sha-256. </para> <para> Also, the legacy (and undocumented) boolean-like values which were previously synonyms of <literal>md5</literal> are no longer accepted. </para> -- Álvaro Herrera Valdivia, Chile "The Gord often wonders why people threaten never to come back after they've been told never to return" (www.actsofgord.com)
On Mon, May 10, 2021 at 02:42:08PM -0400, John Naylor wrote: > Hi Bruce, > > Thanks for doing this work again! > > > Add date_bin function (John Naylor) > > > > WHAT DOES THIS DO? > > Hard to describe in a one-liner, but it lines up timestamps into regular > intervals as specified by the user. It is more clear after seeing examples: > > https://www.postgresql.org/docs/devel/functions-datetime.html# > FUNCTIONS-DATETIME-BIN OK, I just copied the doc sentence into the release notes. > > Dramatically improve Unicode normalization (John Naylor) > > > > WHAT OPERATIONS USE THIS? > > PG13 added the normalize() function to normalize Unicode sequences, as well as > the IS NORMALIZED syntax to test for that. The commits* here do not change > behavior and only improve performance. As such, this really belongs in the > performance section. > > *There is one additional commit that belongs to this entry: > > Author: Michael Paquier <michael@paquier.xyz> > 2020-10-11 [80f8eb79e] Use perfect hash for NFC and NFKC Unicode Normalization > quick check OK, updated text: <listitem> <!-- Author: Michael Paquier <michael@paquier.xyz> 2020-10-11 [80f8eb79e] Use perfect hash for NFC and NFKC Unicode Normalization Author: Michael Paquier <michael@paquier.xyz> 2020-10-23 [783f0cc64] Improve performance of Unicode {de,re}composition in the --> <para> Dramatically improve Unicode normalization (John Naylor) </para> <para> This speeds normalize() and IS NORMALIZED. </para> </listitem> -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Wed, May 12, 2021 at 06:06:31PM -0400, Álvaro Herrera wrote:
> On 2021-May-12, Bruce Momjian wrote:
>
> > OK, I added it into the existing entry:
> >
> > <listitem>
> > <!--
> > Author: Alexander Korotkov <akorotkov@postgresql.org>
> > 2021-01-31 [676887a3b] Implementation of subscripting for jsonb
> > Author: Alexander Korotkov <akorotkov@postgresql.org>
> > 2021-01-31 [81fcc72e6] Filling array gaps during jsonb subscripting
> > Author: Alexander Korotkov <akorotkov@postgresql.org>
> > 2021-01-31 [aa6e46daf] Throw error when assigning jsonb scalar instead of a com
> > -->
> >
> > <para>
> > Allow subscripting of jsonb (Dmitry Dolgov)
> > </para>
> >
> > <para>
> > Subscripting can be used to extract from and assign to jsonb documents.
> > Subscripting is now extensible so extensions and built-in data type can make use of this.
> > </para>
> > </listitem>
>
> I dunno, this seems to approach the feature from the specifics rather
> than its full generality. I would have said something like
>
> Make subscripting user-extensible for "container" types
> <!-- maybe "array-like types" -->
>
> Subscripting JSONB is now integrated in core, and adding user-defined
> subscripting for other types is possible.
I wanted to highlight the more useful aspect of this change, which I
think is JSBONB subscripting. Here is an updated version. Changes?
<listitem>
<!--
Author: Alexander Korotkov <akorotkov@postgresql.org>
2021-01-31 [676887a3b] Implementation of subscripting for jsonb
Author: Alexander Korotkov <akorotkov@postgresql.org>
2021-01-31 [81fcc72e6] Filling array gaps during jsonb subscripting
Author: Alexander Korotkov <akorotkov@postgresql.org>
2021-01-31 [aa6e46daf] Throw error when assigning jsonb scalar instead of a com
-->
<para>
Allow subscripting of JSONB and simplify the implementation of subscripting
(Dmitry Dolgov)
</para>
<para>
JSONB subscripting can be used to extract from and assign to JSONB
documents. Extensions and built-in data types can now implement
subscripting more easily.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Wed, May 12, 2021 at 06:19:28PM -0400, Álvaro Herrera wrote:
> > I am hesitant to merge them since they are different changes.
>
> Different? The commit IDs look pretty similar to me, not to say exactly
> identical.
>
> Reading these, it's pretty unclear what password_encryption *is* in the
> first place. The item should say "the password_encryption server
> parameter" -- that's what we have in the pg10 note for precursor commit
> eb61136dc.
>
> I would suggest
>
> <para>
> Change the default of the password_encryption server parameter to
> scram-sha-256.
> </para>
>
> <para>
> Also, the legacy (and undocumented) boolean-like values which were
> previously synonyms of <literal>md5</literal> are no longer accepted.
> </para>
OK, updated text:
<listitem>
<!--
Author: Peter Eisentraut <peter@eisentraut.org>
2020-06-10 [c7eab0e97] Change default of password_encryption to scram-sha-256
-->
<para>
Change the default of the password_encryption server parameter
to scram-sha-256 (Peter Eisentraut)
</para>
<para>
Previously it was md5. All new passwords will be stored as SHA256
unless this server variable is changed or the password is already
md5-hashed. Also, the legacy (and undocumented) boolean-like
values which were previously synonyms of <literal>md5</literal>
are no longer accepted.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Thu, May 13, 2021 at 2:39 AM Bruce Momjian <bruce@momjian.us> wrote: > On Tue, May 11, 2021 at 11:57:10AM +0900, Amit Langote wrote: > > On Mon, May 10, 2021 at 11:40 PM Justin Pryzby <pryzby@telsasoft.com> wrote: > > > 86dc90056d Rework planning and execution of UPDATE and DELETE. > > > a1115fa078 Postpone some more stuff out of ExecInitModifyTable. > > > c5b7ba4e67 Postpone some stuff out of ExecInitModifyTable. > > > > I was just about to ask Bruce what he thinks about these. > > > > To clarify, the first one is a big refactoring commit that allowed us > > to get rid of inheritance_planner(), a fairly inefficient way of > > planning updates/deletes on partitioned tables, especially when many > > partitions remain after pruning (or when pruning cannot be used). One > > may see the performance of update/deletes, especially on partitioned > > tables, to be generally improved as a result of this commit, but maybe > > not as significantly as to be mentioned in E.1.3.1.1. Partitioning or > > even E.1.3.1.4. General Performance. However, one user-visible > > feature that came out of this work is that updates/deletes can now use > > run-time pruning whereas they couldn't before. Maybe that ought to be > > mentioned. (This reminds me to send a patch to remove the note from > > 5.11.4. Partition Pruning that says that runtime pruning cannot be > > used for update/delete). > > > > The other two commits can lead to improved performance of > > update/deletes when there are many unpruned partitions in the plan, > > but runtime pruning (a new feature as mentioned above) leads to only > > one or few partitions to actually be updated/deleted from. I admit > > though that the cases for which performance has been improved still > > under-perform the cases that already performed better starting in v12, > > that is, the cases where the planner itself is able to trim down the > > plan to contain one or few partitions, so maybe nothing very big to > > see here just yet. You may want to take a look at the benchmark > > results I had posted here: > > https://www.postgresql.org/message-id/CA%2BHiwqEcawatEaUh1uTbZMEZTJeLzbroRTz9_X9Z5CFjTWJkhw%40mail.gmail.com > > OK, I added this entry: Thank you. > <listitem> > <!-- > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2021-03-31 [86dc90056] Rework planning and execution of UPDATE and DELETE. > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2021-04-06 [a1115fa07] Postpone some more stuff out of ExecInitModifyTable. > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2021-04-06 [c5b7ba4e6] Postpone some stuff out of ExecInitModifyTable. > --> > > <para> > Improve the performance of updates/deletes on partitioned tables > when only a few partitions are affected (Amit Langote, Tom Lane) > </para> > > <para> > This also allows run-time pruning of updates/deletes on partitioned > tables. > </para> > </listitem> How about writing the 2nd line instead as: Updates/deletes on partitioned tables can now use execution-time partition pruning. We don't seem to use the term "run-time pruning" anywhere else in the documentation, and "pruning of updates/deletes" sounds strange. -- Amit Langote EDB: http://www.enterprisedb.com
On Thu, May 13, 2021 at 02:46:58PM +0900, Amit Langote wrote:
> How about writing the 2nd line instead as:
>
> Updates/deletes on partitioned tables can now use execution-time
> partition pruning.
>
> We don't seem to use the term "run-time pruning" anywhere else in the
> documentation, and "pruning of updates/deletes" sounds strange.
Good point, updated text:
<listitem>
<!--
Author: Tom Lane <tgl@sss.pgh.pa.us>
2021-03-31 [86dc90056] Rework planning and execution of UPDATE and DELETE.
Author: Tom Lane <tgl@sss.pgh.pa.us>
2021-04-06 [a1115fa07] Postpone some more stuff out of ExecInitModifyTable.
Author: Tom Lane <tgl@sss.pgh.pa.us>
2021-04-06 [c5b7ba4e6] Postpone some stuff out of ExecInitModifyTable.
-->
<para>
Improve the performance of updates/deletes on partitioned tables
when only a few partitions are affected (Amit Langote, Tom Lane)
</para>
<para>
This also allows updates/deletes on partitioned tables to use
execution-time partition pruning.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 02:03:08AM -0400, Bruce Momjian wrote: > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > > I need clarification on many items, and the document still needs its > items properly ordered, and markup added. I also expect a lot of > feedback. > > I plan to work on completing this document this coming week in > preparation for beta next week. I have ordered the items in each section. My next job is to add markup and indenting to the XML. FYI, the PG 14 release item count is much higher than usual: release-7.4: 263 release-8.0: 230 release-8.1: 174 release-8.2: 215 release-8.3: 214 release-8.4: 314 release-9.0: 237 release-9.1: 203 release-9.2: 238 release-9.3: 177 release-9.4: 211 release-9.5: 193 release-9.6: 214 release-10: 189 release-11: 170 release-12: 180 release-13: 178 release-14: 220 PG 14 is a 23% increase over our previous release. I think the cause is either more hackers/sponsors, Covid, or both. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Thu, May 13, 2021 at 11:59 PM Bruce Momjian <bruce@momjian.us> wrote: > On Thu, May 13, 2021 at 02:46:58PM +0900, Amit Langote wrote: > > How about writing the 2nd line instead as: > > > > Updates/deletes on partitioned tables can now use execution-time > > partition pruning. > > > > We don't seem to use the term "run-time pruning" anywhere else in the > > documentation, and "pruning of updates/deletes" sounds strange. > > Good point, updated text: > > <listitem> > <!-- > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2021-03-31 [86dc90056] Rework planning and execution of UPDATE and DELETE. > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2021-04-06 [a1115fa07] Postpone some more stuff out of ExecInitModifyTable. > Author: Tom Lane <tgl@sss.pgh.pa.us> > 2021-04-06 [c5b7ba4e6] Postpone some stuff out of ExecInitModifyTable. > --> > > <para> > Improve the performance of updates/deletes on partitioned tables > when only a few partitions are affected (Amit Langote, Tom Lane) > </para> > > <para> > This also allows updates/deletes on partitioned tables to use > execution-time partition pruning. > </para> > </listitem> Thank you. -- Amit Langote EDB: http://www.enterprisedb.com
On Mon, May 10, 2021 at 09:40:45AM -0500, Justin Pryzby wrote: > Should any of these be included? New SQL-accessible functionality should be included: > 8c15a29745 Allow ALTER TYPE to update an existing type's typsubscript value. These should be merged: > 756ab29124 Allow pageinspect to inspect GiST indexes (Andrey Borodin, Heikki Linnakangas) > 9e596b65f4 Add "LP_DEAD item?" column to GiST pageinspect functions I'm undecided on this one: > 7db0cd2145 Set PD_ALL_VISIBLE and visibility map bits in COPY FREEZE People who didn't used to use FREEZE (because it didn't work or otherwise) might want to start using it. I'm withdrawing these, as modifications to existing log messages don't need to be included: > 10a5b35a00 Report resource usage at the end of recovery > 7e453634bb Add additional information in the vacuum error context. > 1ea396362b Improve logging of bad parameter values in BIND messages. -- Justin
On Thu, May 13, 2021 at 09:01:41PM -0500, Justin Pryzby wrote: > These should be merged: > > 756ab29124 Allow pageinspect to inspect GiST indexes (Andrey Borodin, Heikki Linnakangas) > > 9e596b65f4 Add "LP_DEAD item?" column to GiST pageinspect functions Sorry, this was my error while reconciling my list with yours. Your notes only have one item for these, which is correct. Possibly you'd want to include the 9e59 commit in the comment (but it wouldn't have avoided my own confusion, tough). -- Justin
On Thu, May 13, 2021 at 09:08:37PM -0500, Justin Pryzby wrote:
> On Thu, May 13, 2021 at 09:01:41PM -0500, Justin Pryzby wrote:
> > These should be merged:
> > > 756ab29124 Allow pageinspect to inspect GiST indexes (Andrey Borodin, Heikki Linnakangas)
> > > 9e596b65f4 Add "LP_DEAD item?" column to GiST pageinspect functions
>
> Sorry, this was my error while reconciling my list with yours.
> Your notes only have one item for these, which is correct.
>
> Possibly you'd want to include the 9e59 commit in the comment (but it wouldn't
> have avoided my own confusion, tough).
OK, I added that commit:
<listitem>
<!--
Author: Heikki Linnakangas <heikki.linnakangas@iki.fi>
2021-01-13 [756ab2912] Add functions to 'pageinspect' to inspect GiST indexes.
Author: Peter Geoghegan <pg@bowt.ie>
2021-02-14 [9e596b65f] Add "LP_DEAD item?" column to GiST pageinspect functions
-->
<para>
Allow pageinspect to inspect GiST indexes (Andrey Borodin,
Heikki Linnakangas)
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Thu, May 13, 2021 at 09:01:41PM -0500, Justin Pryzby wrote:
> On Mon, May 10, 2021 at 09:40:45AM -0500, Justin Pryzby wrote:
> > Should any of these be included?
>
> New SQL-accessible functionality should be included:
> > 8c15a29745 Allow ALTER TYPE to update an existing type's typsubscript value.
OK, text is:
<listitem>
<!--
Author: Tom Lane <tgl@sss.pgh.pa.us>
2020-12-11 [8c15a2974] Allow ALTER TYPE to update an existing type's typsubscri
-->
<para>
Allow ALTER TYPE to specify or remove a SUBSCRIPT handler
(Tom Lane)
</para>
</listitem>
> I'm undecided on this one:
> > 7db0cd2145 Set PD_ALL_VISIBLE and visibility map bits in COPY FREEZE
>
> People who didn't used to use FREEZE (because it didn't work or otherwise)
> might want to start using it.
Yes, that seems very important:
<listitem>
<!--
Author: Tomas Vondra <tomas.vondra@postgresql.org>
2021-01-17 [7db0cd214] Set PD_ALL_VISIBLE and visibility map bits in COPY FREEZ
-->
<para>
Have COPY FREEZE appropriately update page visibility bits
(Anastasia Lubennikova, Pavan Deolasee, Jeff Janes)
</para>
</listitem>
This one one of those, "I must be confused, since we certainly already
did this before."
> I'm withdrawing these, as modifications to existing log messages don't need to
> be included:
>
> > 10a5b35a00 Report resource usage at the end of recovery
> > 7e453634bb Add additional information in the vacuum error context.
> > 1ea396362b Improve logging of bad parameter values in BIND messages.
OK.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
Bruce Momjian <bruce@momjian.us> writes:
> On Thu, May 13, 2021 at 09:01:41PM -0500, Justin Pryzby wrote:
>> New SQL-accessible functionality should be included:
>>> 8c15a29745 Allow ALTER TYPE to update an existing type's typsubscript value.
> OK, text is:
> <listitem>
> <!--
> Author: Tom Lane <tgl@sss.pgh.pa.us>
> 2020-12-11 [8c15a2974] Allow ALTER TYPE to update an existing type's typsubscri
> -->
> <para>
> Allow ALTER TYPE to specify or remove a SUBSCRIPT handler
> (Tom Lane)
> </para>
> </listitem>
I don't understand why we'd bother to list that as a separate bullet item.
It's an expected part of the custom-subscript capability, I'd think.
regards, tom lane
On Fri, May 14, 2021 at 03:39:39PM -0400, Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > On Thu, May 13, 2021 at 09:01:41PM -0500, Justin Pryzby wrote: > >> New SQL-accessible functionality should be included: > >>> 8c15a29745 Allow ALTER TYPE to update an existing type's typsubscript value. > > > OK, text is: > > > <listitem> > > <!-- > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > 2020-12-11 [8c15a2974] Allow ALTER TYPE to update an existing type's typsubscri > > --> > > > <para> > > Allow ALTER TYPE to specify or remove a SUBSCRIPT handler > > (Tom Lane) > > </para> > > </listitem> > > I don't understand why we'd bother to list that as a separate bullet item. > It's an expected part of the custom-subscript capability, I'd think. I am not sure either. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Fri, May 14, 2021 at 04:01:32PM -0400, Bruce Momjian wrote: > On Fri, May 14, 2021 at 03:39:39PM -0400, Tom Lane wrote: > > Bruce Momjian <bruce@momjian.us> writes: > > > On Thu, May 13, 2021 at 09:01:41PM -0500, Justin Pryzby wrote: > > >> New SQL-accessible functionality should be included: > > >>> 8c15a29745 Allow ALTER TYPE to update an existing type's typsubscript value. > > > > > OK, text is: > > > > > <listitem> > > > <!-- > > > Author: Tom Lane <tgl@sss.pgh.pa.us> > > > 2020-12-11 [8c15a2974] Allow ALTER TYPE to update an existing type's typsubscri > > > --> > > > > > <para> > > > Allow ALTER TYPE to specify or remove a SUBSCRIPT handler > > > (Tom Lane) > > > </para> > > > </listitem> > > > > I don't understand why we'd bother to list that as a separate bullet item. > > It's an expected part of the custom-subscript capability, I'd think. > > I am not sure either. Removed. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On 2021-May-12, Bruce Momjian wrote: > OK, updated text: > > <listitem> > <!-- > Author: Peter Eisentraut <peter@eisentraut.org> > 2020-06-10 [c7eab0e97] Change default of password_encryption to scram-sha-256 > --> > > <para> > Change the default of the password_encryption server parameter > to scram-sha-256 (Peter Eisentraut) > </para> > > <para> > Previously it was md5. All new passwords will be stored as SHA256 > unless this server variable is changed or the password is already > md5-hashed. Also, the legacy (and undocumented) boolean-like > values which were previously synonyms of <literal>md5</literal> > are no longer accepted. > </para> > </listitem> Thanks, looks ok as far as what the original point was about. I have to say that this sentence is a bit odd: "All new passwords will be stored as sha256 unless ... the password is already md5-hashed". Does this mean that if you change a password for a user whose password was md5, the new one is stored as md5 too even if the setting is scram-sha-256? Or if "the password" means an old password, then why is it a new password? -- Álvaro Herrera Valdivia, Chile
On 2021-May-10, Bruce Momjian wrote: > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html Thanks for putting this all together. > <listitem> > <!-- > Author: Alvaro Herrera <alvherre@alvh.no-ip.org> > 2021-04-08 [0827e8af7] autovacuum: handle analyze for partitioned tables > --> > > <para> > Autovacuum now analyzes partitioned tables (Yuzuko Hosoya) > </para> I would add something like: "The insert, update and delete tuple counts from partitions are propagated to their parent tables, so autovacuum knows to process them too." Because I spent a considerable amount of time on this patch, I think it's fair to list me as co-author, after Hosoya-san's name. Same with the PQtrace() item. Thanks -- Álvaro Herrera Valdivia, Chile
Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> I, um, again think that the item should not be primarily about JSONB,
> but about our (IMO groundbreaking) extensible subscripting work.
+1. Extensible subscripting is THE feature here, using it for jsonb
is just an application.
regards, tom lane
On Sat, May 15, 2021 at 07:05:35PM -0400, Álvaro Herrera wrote:
> On 2021-May-12, Bruce Momjian wrote:
>
> > OK, updated text:
> >
> > <listitem>
> > <!--
> > Author: Peter Eisentraut <peter@eisentraut.org>
> > 2020-06-10 [c7eab0e97] Change default of password_encryption to scram-sha-256
> > -->
> >
> > <para>
> > Change the default of the password_encryption server parameter
> > to scram-sha-256 (Peter Eisentraut)
> > </para>
> >
> > <para>
> > Previously it was md5. All new passwords will be stored as SHA256
> > unless this server variable is changed or the password is already
> > md5-hashed. Also, the legacy (and undocumented) boolean-like
> > values which were previously synonyms of <literal>md5</literal>
> > are no longer accepted.
> > </para>
> > </listitem>
>
> Thanks, looks ok as far as what the original point was about.
>
> I have to say that this sentence is a bit odd: "All new passwords will
> be stored as sha256 unless ... the password is already md5-hashed".
> Does this mean that if you change a password for a user whose password
> was md5, the new one is stored as md5 too even if the setting is
> scram-sha-256? Or if "the password" means an old password, then why is
> it a new password?
OK, what I was trying to say was that if you dump/restore, and the old
password was md5, the newly-restored password will be md5, but it was
very unclear. I changed it to this:
<listitem>
<!--
Author: Peter Eisentraut <peter@eisentraut.org>
2020-06-10 [c7eab0e97] Change default of password_encryption to scram-sha-256
Author: Peter Eisentraut <peter@eisentraut.org>
2020-06-10 [c7eab0e97] Change default of password_encryption to scram-sha-256
-->
<para>
Change the default of the password_encryption server parameter to
scram-sha-256 (Peter Eisentraut)
</para>
<para>
Previously it was md5. All new passwords will be stored as SHA256
unless this server variable is changed or the password is specified
in md5 format. Also, the legacy (and undocumented) boolean-like
values which were previously synonyms for <literal>md5</literal>
are no longer accepted.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Sat, May 15, 2021 at 10:27:08PM -0400, Tom Lane wrote:
> Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> > I, um, again think that the item should not be primarily about JSONB,
> > but about our (IMO groundbreaking) extensible subscripting work.
>
> +1. Extensible subscripting is THE feature here, using it for jsonb
> is just an application.
OK, I moved the entry into 'data types', and listed two entries instead
of one:
<listitem>
<!--
Author: Tom Lane <tgl@sss.pgh.pa.us>
2020-12-09 [c7aba7c14] Support subscripting of arbitrary types, not only arrays
-->
<para>
Allow extensions and built-in data types to implement subscripting (Tom Lane)
</para>
<para>
Previously subscript handling was hard-coded into the server and
was difficult to add to date types.
</para>
</listitem>
<listitem>
<!--
Author: Alexander Korotkov <akorotkov@postgresql.org>
2021-01-31 [676887a3b] Implementation of subscripting for jsonb
Author: Alexander Korotkov <akorotkov@postgresql.org>
2021-01-31 [81fcc72e6] Filling array gaps during jsonb subscripting
Author: Alexander Korotkov <akorotkov@postgresql.org>
2021-01-31 [aa6e46daf] Throw error when assigning jsonb scalar instead of a com
-->
<para>
Allow subscripting of JSONB (Dmitry Dolgov)
</para>
<para>
JSONB subscripting can be used to extract from and assign to
JSONB documents.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Sat, May 15, 2021 at 07:16:18PM -0400, Álvaro Herrera wrote: > On 2021-May-10, Bruce Momjian wrote: > > > I have committed the first draft of the PG 14 release notes. You can > > see the most current build of them here: > > > > https://momjian.us/pgsql_docs/release-14.html > > Thanks for putting this all together. > > > <listitem> > > <!-- > > Author: Alvaro Herrera <alvherre@alvh.no-ip.org> > > 2021-04-08 [0827e8af7] autovacuum: handle analyze for partitioned tables > > --> > > > > <para> > > Autovacuum now analyzes partitioned tables (Yuzuko Hosoya) > > </para> > > I would add something like: > > "The insert, update and delete tuple counts from partitions are > propagated to their parent tables, so autovacuum knows to process them > too." OK, updated text: <listitem> <!-- Author: Alvaro Herrera <alvherre@alvh.no-ip.org> 2021-04-08 [0827e8af7] autovacuum: handle analyze for partitioned tables --> <para> Autovacuum now analyzes partitioned tables (Yuzuko Hosoya, Álvaro Herrera) </para> <para> Insert, update, and delete tuple counts from partitions are now propagated to their parent tables so autovacuum knows when to process them. </para> </listitem> > Because I spent a considerable amount of time on this patch, I think > it's fair to list me as co-author, after Hosoya-san's name. Same with > the PQtrace() item. Done, thanks. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
Bruce Momjian <bruce@momjian.us> writes:
> OK, I moved the entry into 'data types', and listed two entries instead
> of one:
> <para>
> Allow extensions and built-in data types to implement subscripting (Tom Lane)
> </para>
Better, but Dmitry Dolgov gets the credit not me.
> <para>
> Previously subscript handling was hard-coded into the server and
> was difficult to add to date types.
> </para>
> </listitem>
"Difficult"? I don't think it was possible at all. Perhaps
Previously subscript handling was hard-coded into the server,
so that subscripting could only be applied to array types.
This change allows subscript notation to be used to extract
or assign portions of a value of any type for which the concept
makes sense.
> JSONB subscripting can be used to extract from and assign to
> JSONB documents.
Perhaps "extract and assign to portions of JSONB documents" or
something like that?
regards, tom lane
On Sun, May 16, 2021 at 11:17:46PM -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > OK, I moved the entry into 'data types', and listed two entries instead
> > of one:
>
> > <para>
> > Allow extensions and built-in data types to implement subscripting (Tom Lane)
> > </para>
>
> Better, but Dmitry Dolgov gets the credit not me.
OK, fixed.
> > <para>
> > Previously subscript handling was hard-coded into the server and
> > was difficult to add to date types.
> > </para>
> > </listitem>
>
> "Difficult"? I don't think it was possible at all. Perhaps
I used difficult to mean you had to hack the server code to enable it,
but your sentence below is better since it highlights usage beyond
arrays.
> Previously subscript handling was hard-coded into the server,
> so that subscripting could only be applied to array types.
> This change allows subscript notation to be used to extract
> or assign portions of a value of any type for which the concept
> makes sense.
>
> > JSONB subscripting can be used to extract from and assign to
> > JSONB documents.
>
> Perhaps "extract and assign to portions of JSONB documents" or
> something like that?
OK, updated text for both items:
<listitem>
<!--
Author: Tom Lane <tgl@sss.pgh.pa.us>
2020-12-09 [c7aba7c14] Support subscripting of arbitrary types, not only arrays
-->
<para>
Allow extensions and built-in data types to implement subscripting (Dmitry Dolgov)
</para>
<para>
Previously subscript handling was hard-coded into the server,
so that subscripting could only be applied to array types. This
change allows subscript notation to be used to extract or assign
portions of a value of any type for which the concept makes sense.
</para>
</listitem>
<listitem>
<!--
Author: Alexander Korotkov <akorotkov@postgresql.org>
2021-01-31 [676887a3b] Implementation of subscripting for jsonb
Author: Alexander Korotkov <akorotkov@postgresql.org>
2021-01-31 [81fcc72e6] Filling array gaps during jsonb subscripting
Author: Alexander Korotkov <akorotkov@postgresql.org>
2021-01-31 [aa6e46daf] Throw error when assigning jsonb scalar instead of a com
-->
<para>
Allow subscripting of JSONB (Dmitry Dolgov)
</para>
<para>
JSONB subscripting can be used to extract and assign to portions
of JSONB documents.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On 2021/05/10 15:03, Bruce Momjian wrote: > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html Thanks a lot!! > I need clarification on many items, and the document still needs its > items properly ordered, and markup added. I also expect a lot of > feedback. > <para> > Add function pg_backend_memory_contexts() to output the memory contexts of arbitrary backends (Atsushi Torikoshi) Could you fix the function name from pg_backend_memory_contexts() to pg_log_backend_memory_contexts()? That is, "log_" is necessary just after "pg_" in the function name. > <para> > Add lock wait time to pg_locks (Atsushi Torikoshi) Could you change "lock wait time" to "lock wait start time"? Because the pg_locks.waitstart column that was added in v14 reports the time when the server process *started* waiting for the lock. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION
On Mon, May 17, 2021 at 04:18:27PM +0900, Fujii Masao wrote: > > > On 2021/05/10 15:03, Bruce Momjian wrote: > > I have committed the first draft of the PG 14 release notes. You can > > see the most current build of them here: > > > > https://momjian.us/pgsql_docs/release-14.html > > Thanks a lot!! > > > > I need clarification on many items, and the document still needs its > > items properly ordered, and markup added. I also expect a lot of > > feedback. > > > <para> > > Add function pg_backend_memory_contexts() to output the memory contexts of arbitrary backends (Atsushi Torikoshi) > > Could you fix the function name from pg_backend_memory_contexts() to > pg_log_backend_memory_contexts()? That is, "log_" is necessary just after > "pg_" in the function name. Ah, I see my mistake, fixed. > > <para> > > Add lock wait time to pg_locks (Atsushi Torikoshi) > > Could you change "lock wait time" to "lock wait start time"? Because > the pg_locks.waitstart column that was added in v14 reports the time > when the server process *started* waiting for the lock. Yes, good point, fixed. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
Hi, On 5/10/21 2:03 AM, Bruce Momjian wrote: > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > > I need clarification on many items, and the document still needs its > items properly ordered, and markup added. I also expect a lot of > feedback. > > I plan to work on completing this document this coming week in > preparation for beta next week. Again, thanks for compiling the release notes. I have published the first draft of the release announcement on -advocacy, you can find the original email here: https://www.postgresql.org/message-id/f82ccf97-d02c-0974-27b8-e0f5e4c1de02%40postgresql.org If you are interested in reviewing the release announcement, please provide feedback on the -advocacy mailing list. Thanks! Jonathan
Attachment
On 2021-May-16, Bruce Momjian wrote: > OK, what I was trying to say was that if you dump/restore, and the old > password was md5, the newly-restored password will be md5, but it was > very unclear. I changed it to this: > > <listitem> > <!-- > Author: Peter Eisentraut <peter@eisentraut.org> > 2020-06-10 [c7eab0e97] Change default of password_encryption to scram-sha-256 > Author: Peter Eisentraut <peter@eisentraut.org> > 2020-06-10 [c7eab0e97] Change default of password_encryption to scram-sha-256 > --> > > <para> > Change the default of the password_encryption server parameter to > scram-sha-256 (Peter Eisentraut) > </para> > > <para> > Previously it was md5. All new passwords will be stored as SHA256 > unless this server variable is changed or the password is specified > in md5 format. Also, the legacy (and undocumented) boolean-like > values which were previously synonyms for <literal>md5</literal> > are no longer accepted. > </para> > </listitem> Ah, yeah, now I understand. WFM. Thanks -- Álvaro Herrera 39°49'30"S 73°17'W "La fuerza no está en los medios físicos sino que reside en una voluntad indomable" (Gandhi)
I propose that the B-Tree index item (bottom-up index deletion, "Allow btree index additions to remove expired index entries to prevent page splits") link back to the documentation for those that want to drill down: https://www.postgresql.org/docs/14/btree-implementation.html#BTREE-DELETION The same approach was used for the Postgres 13 B-Tree deduplication release note item (same page from the docs, even). I understand that it's kind of hard to get the point of the feature across within the confines of the release notes, which have a very wide audience. I believe that the practical take-away for more experienced users should be something like "significantly ameliorates write-amplification problems in indexes with non-HOT updates". But that's quite a mouthful. Having a hyperlink to the docs seems like a sensible compromise. I really just want experienced users to be able to make the connection between this feature and well-publicized problems with Postgres index bloat/VACUUM. Thanks -- Peter Geoghegan
On Mon, May 17, 2021 at 12:45 PM Peter Geoghegan <pg@bowt.ie> wrote: > https://www.postgresql.org/docs/14/btree-implementation.html#BTREE-DELETION This linke currently 404s -- https://www.postgresql.org/docs/devel/btree-implementation.html#BTREE-DELETION works, though. -- Peter Geoghegan
On Mon, May 17, 2021 at 12:45:44PM -0700, Peter Geoghegan wrote: > I propose that the B-Tree index item (bottom-up index deletion, "Allow > btree index additions to remove expired index entries to prevent page > splits") link back to the documentation for those that want to drill > down: > > https://www.postgresql.org/docs/14/btree-implementation.html#BTREE-DELETION > > The same approach was used for the Postgres 13 B-Tree deduplication > release note item (same page from the docs, even). > > I understand that it's kind of hard to get the point of the feature > across within the confines of the release notes, which have a very > wide audience. I believe that the practical take-away for more > experienced users should be something like "significantly ameliorates > write-amplification problems in indexes with non-HOT updates". But > that's quite a mouthful. Having a hyperlink to the docs seems like a > sensible compromise. I really just want experienced users to be able > to make the connection between this feature and well-publicized > problems with Postgres index bloat/VACUUM. Yes, I am going to add links to everything, but haven't had time yet. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 3:03 PM Bruce Momjian <bruce@momjian.us> wrote: > > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > I think we need to mention in the release note that vacuum_cleanup_index_scale_factor GUC parameter has been removed and vacuum_cleanup_index_scale_factor storage parameter has been deprecated (please refer to commit 9f3665fb and effdd3f3b63). Regards, -- Masahiko Sawada EDB: https://www.enterprisedb.com/
On Tue, May 18, 2021 at 06:28:49PM +0900, Masahiko Sawada wrote: > On Mon, May 10, 2021 at 3:03 PM Bruce Momjian <bruce@momjian.us> wrote: > > > > I have committed the first draft of the PG 14 release notes. You can > > see the most current build of them here: > > > > https://momjian.us/pgsql_docs/release-14.html > > > > I think we need to mention in the release note that > vacuum_cleanup_index_scale_factor GUC parameter has been removed and > vacuum_cleanup_index_scale_factor storage parameter has been > deprecated (please refer to commit 9f3665fb and effdd3f3b63). Looking at the full commit message: commit 9f3665fbfc Author: Peter Geoghegan <pg@bowt.ie> Date: Wed Mar 10 16:27:01 2021 -0800 Don't consider newly inserted tuples in nbtree VACUUM. Remove the entire idea of "stale stats" within nbtree VACUUM (stop caring about stats involving the number of inserted tuples). Also remove the vacuum_cleanup_index_scale_factor GUC/param on the master branch (though just disable them on postgres 13). The vacuum_cleanup_index_scale_factor/stats interface made the nbtree AM partially responsible for deciding when pg_class.reltuples stats needed to be updated. This seems contrary to the spirit of the index AM API, though -- it is not actually necessary for an index AM's bulk delete and cleanup callbacks to provide accurate stats when it happens to be inconvenient. The core code owns that. (Index AMs have the authority to perform or not perform certain kinds of deferred cleanup based on their own considerations, such as page deletion and recycling, but that has little to do with pg_class.reltuples/num_index_tuples.) This issue was fairly harmless until the introduction of the autovacuum_vacuum_insert_threshold feature by commit b07642db, which had an undesirable interaction with the vacuum_cleanup_index_scale_factor mechanism: it made insert-driven autovacuums perform full index scans, even though there is no real benefit to doing so. This has been tied to a regression with an append-only insert benchmark [1]. Also have remaining cases that perform a full scan of an index during a cleanup-only nbtree VACUUM indicate that the final tuple count is only an estimate. This prevents vacuumlazy.c from setting the index's pg_class.reltuples in those cases (it will now only update pg_class when vacuumlazy.c had TIDs for nbtree to bulk delete). This arguably fixes an oversight in deduplication-related bugfix commit 48e12913. [1] https://smalldatum.blogspot.com/2021/01/insert-benchmark-postgres-is-still.html Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com> Discussion: https://postgr.es/m/CAD21AoA4WHthN5uU6+WScZ7+J_RcEjmcuH94qcoUPuB42ShXzg@mail.gmail.com --> Backpatch: 13-, where autovacuum_vacuum_insert_threshold was added. This was backpatched into PG 13.3, which was released last week: <listitem> <!-- Author: Peter Geoghegan <pg@bowt.ie> Branch: master [9f3665fbf] 2021-03-10 16:27:01 -0800 Branch: REL_13_STABLE [9663d1244] 2021-03-10 16:26:58 -0800 Author: Peter Geoghegan <pg@bowt.ie> Branch: master [5f8727f5a] 2021-03-10 17:07:57 -0800 Branch: REL_13_STABLE [1fc5a5738] 2021-03-10 17:07:55 -0800 --> <para> Disable the <varname>vacuum_cleanup_index_scale_factor</varname> parameter and storage option (Peter Geoghegan) </para> <para> The notion of tracking <quote>stale</quote> index statistics proved to interact badly with the <varname>autovacuum_vacuum_insert_threshold</varname> parameter, resulting in unnecessary full-index scans and consequent degradation of autovacuum performance. The latter mechanism seems superior, so remove the stale-statistics logic. The control parameter for that, <varname>vacuum_cleanup_index_scale_factor</varname>, will be removed entirely in v14. In v13, it remains present to avoid breaking existing configuration files, but it no longer does anything. </para> </listitem> Therefore, it didn't show up in my src/tools/git_changelog output, and I did not include it. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Tue, May 18, 2021 at 11:07 PM Bruce Momjian <bruce@momjian.us> wrote: > > On Tue, May 18, 2021 at 06:28:49PM +0900, Masahiko Sawada wrote: > > On Mon, May 10, 2021 at 3:03 PM Bruce Momjian <bruce@momjian.us> wrote: > > > > > > I have committed the first draft of the PG 14 release notes. You can > > > see the most current build of them here: > > > > > > https://momjian.us/pgsql_docs/release-14.html > > > > > > > I think we need to mention in the release note that > > vacuum_cleanup_index_scale_factor GUC parameter has been removed and > > vacuum_cleanup_index_scale_factor storage parameter has been > > deprecated (please refer to commit 9f3665fb and effdd3f3b63). > > Looking at the full commit message: > > commit 9f3665fbfc > Author: Peter Geoghegan <pg@bowt.ie> > Date: Wed Mar 10 16:27:01 2021 -0800 > > Don't consider newly inserted tuples in nbtree VACUUM. > > Remove the entire idea of "stale stats" within nbtree VACUUM (stop > caring about stats involving the number of inserted tuples). Also > remove the vacuum_cleanup_index_scale_factor GUC/param on the master > branch (though just disable them on postgres 13). > > The vacuum_cleanup_index_scale_factor/stats interface made the nbtree AM > partially responsible for deciding when pg_class.reltuples stats needed > to be updated. This seems contrary to the spirit of the index AM API, > though -- it is not actually necessary for an index AM's bulk delete and > cleanup callbacks to provide accurate stats when it happens to be > inconvenient. The core code owns that. (Index AMs have the authority > to perform or not perform certain kinds of deferred cleanup based on > their own considerations, such as page deletion and recycling, but that > has little to do with pg_class.reltuples/num_index_tuples.) > > This issue was fairly harmless until the introduction of the > autovacuum_vacuum_insert_threshold feature by commit b07642db, which had > an undesirable interaction with the vacuum_cleanup_index_scale_factor > mechanism: it made insert-driven autovacuums perform full index scans, > even though there is no real benefit to doing so. This has been tied to > a regression with an append-only insert benchmark [1]. > > Also have remaining cases that perform a full scan of an index during a > cleanup-only nbtree VACUUM indicate that the final tuple count is only > an estimate. This prevents vacuumlazy.c from setting the index's > pg_class.reltuples in those cases (it will now only update pg_class when > vacuumlazy.c had TIDs for nbtree to bulk delete). This arguably fixes > an oversight in deduplication-related bugfix commit 48e12913. > > [1] https://smalldatum.blogspot.com/2021/01/insert-benchmark-postgres-is-still.html > > Author: Peter Geoghegan <pg@bowt.ie> > Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com> > Discussion: https://postgr.es/m/CAD21AoA4WHthN5uU6+WScZ7+J_RcEjmcuH94qcoUPuB42ShXzg@mail.gmail.com > --> Backpatch: 13-, where autovacuum_vacuum_insert_threshold was added. > > This was backpatched into PG 13.3, which was released last week: > > <listitem> > <!-- > Author: Peter Geoghegan <pg@bowt.ie> > Branch: master [9f3665fbf] 2021-03-10 16:27:01 -0800 > Branch: REL_13_STABLE [9663d1244] 2021-03-10 16:26:58 -0800 > Author: Peter Geoghegan <pg@bowt.ie> > Branch: master [5f8727f5a] 2021-03-10 17:07:57 -0800 > Branch: REL_13_STABLE [1fc5a5738] 2021-03-10 17:07:55 -0800 > --> > <para> > Disable the <varname>vacuum_cleanup_index_scale_factor</varname> > parameter and storage option (Peter Geoghegan) > </para> > > <para> > The notion of tracking <quote>stale</quote> index statistics proved > to interact badly with > the <varname>autovacuum_vacuum_insert_threshold</varname> parameter, > resulting in unnecessary full-index scans and consequent degradation > of autovacuum performance. The latter mechanism seems superior, so > remove the stale-statistics logic. The control parameter for that, > <varname>vacuum_cleanup_index_scale_factor</varname>, will be > removed entirely in v14. In v13, it remains present to avoid > breaking existing configuration files, but it no longer does > anything. > </para> > </listitem> > > Therefore, it didn't show up in my src/tools/git_changelog output, and I > did not include it. > Thanks for your explanation. I understood and agreed not to include it in PG14 release note. Regards, -- Masahiko Sawada EDB: https://www.enterprisedb.com/
On Tue, May 18, 2021 at 10:07:25AM -0400, Bruce Momjian wrote: > On Tue, May 18, 2021 at 06:28:49PM +0900, Masahiko Sawada wrote: > > On Mon, May 10, 2021 at 3:03 PM Bruce Momjian <bruce@momjian.us> wrote: > > > > > > I have committed the first draft of the PG 14 release notes. You can > > > see the most current build of them here: > > > > > > https://momjian.us/pgsql_docs/release-14.html > > > > I think we need to mention in the release note that > > vacuum_cleanup_index_scale_factor GUC parameter has been removed and > > vacuum_cleanup_index_scale_factor storage parameter has been > > deprecated (please refer to commit 9f3665fb and effdd3f3b63). > > Looking at the full commit message: > > commit 9f3665fbfc > Author: Peter Geoghegan <pg@bowt.ie> > Date: Wed Mar 10 16:27:01 2021 -0800 > > Don't consider newly inserted tuples in nbtree VACUUM. > > Remove the entire idea of "stale stats" within nbtree VACUUM (stop > caring about stats involving the number of inserted tuples). Also > remove the vacuum_cleanup_index_scale_factor GUC/param on the master > branch (though just disable them on postgres 13). > This was backpatched into PG 13.3, which was released last week: > remove the stale-statistics logic. The control parameter for that, > <varname>vacuum_cleanup_index_scale_factor</varname>, will be > removed entirely in v14. In v13, it remains present to avoid > breaking existing configuration files, but it no longer does > anything. > </para> > </listitem> > > Therefore, it didn't show up in my src/tools/git_changelog output, and I > did not include it. Normally, stuff that was backpatched isn't included in major release notes, since the change would/could normally happen during a minor -> minor+1 release. As things stand, in this case I think it *should* be included, since the backpatched change isn't the same as the change to HEAD (removing the GUC). The git_changelog output might well be wrong in this case (or, arguably, the "remove the GUC entirely" should've been a separate master-only commit than the "make the GUC do nothing" commit). However, Peter indicated an intent to add a reloption to disable the vacuum optimization, so maybe the removal of the GUC could be documented at that time. -- Justin
On Tue, May 18, 2021 at 7:44 AM Justin Pryzby <pryzby@telsasoft.com> wrote: > As things stand, in this case I think it *should* be included, since the > backpatched change isn't the same as the change to HEAD (removing the GUC). > The git_changelog output might well be wrong in this case (or, arguably, the > "remove the GUC entirely" should've been a separate master-only commit than the > "make the GUC do nothing" commit). I suppose that's true -- maybe it should be listed separately, because the GUC is removed in 14 only. > However, Peter indicated an intent to add a reloption to disable the vacuum > optimization, so maybe the removal of the GUC could be documented at that time. This is unrelated to the bypass indexes in VACUUM thing. -- Peter Geoghegan
On Tue, May 18, 2021 at 09:44:09AM -0500, Justin Pryzby wrote: > As things stand, in this case I think it *should* be included, since the > backpatched change isn't the same as the change to HEAD (removing the GUC). > The git_changelog output might well be wrong in this case (or, arguably, the > "remove the GUC entirely" should've been a separate master-only commit than the > "make the GUC do nothing" commit). I think having the same commit message for different patches to different branches is an unwise behavior, particularly if the commit is release-note worthy. (I think it is fine if the patch is purely mechanical and hence not release-note worthy.) The master patch is hash 9f3665fbfc and the PG 13 patch is hash 9663d12446. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Tue, May 18, 2021 at 07:51:00AM -0700, Peter Geoghegan wrote: > On Tue, May 18, 2021 at 7:44 AM Justin Pryzby <pryzby@telsasoft.com> wrote: > > As things stand, in this case I think it *should* be included, since the > > backpatched change isn't the same as the change to HEAD (removing the GUC). > > The git_changelog output might well be wrong in this case (or, arguably, the > > "remove the GUC entirely" should've been a separate master-only commit than the > > "make the GUC do nothing" commit). > > I suppose that's true -- maybe it should be listed separately, because > the GUC is removed in 14 only. OK, this is a mess then. Would someone please give me the full text for this, including the commit hash(es)? Is the PG 13.3 release note text accurate? -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Tue, May 18, 2021 at 8:09 AM Bruce Momjian <bruce@momjian.us> wrote: > > I suppose that's true -- maybe it should be listed separately, because > > the GUC is removed in 14 only. > > OK, this is a mess then. Would someone please give me the full text for > this, including the commit hash(es)? Is the PG 13.3 release note text > accurate? The 13.3 release notes say this: """ Disable the vacuum_cleanup_index_scale_factor parameter and storage option (Peter Geoghegan) The notion of tracking “stale” index statistics proved to interact badly with the autovacuum_vacuum_insert_threshold parameter, resulting in unnecessary full-index scans and consequent degradation of autovacuum performance. The latter mechanism seems superior, so remove the stale-statistics logic. The control parameter for that, vacuum_cleanup_index_scale_factor, will be removed entirely in v14. In v13, it remains present to avoid breaking existing configuration files, but it no longer does anything. """ I think that this is slightly inaccurate, though that's probably of little consequence. The autovacuum_vacuum_insert_threshold GUC was in fact removed in v14, but the reloption was ultimately not removed from HEAD/v14 for compatibility reasons. This is not apparent just from commit 9f3665fb -- there was a clean-up commit (commit effdd3f3) that added the reloption back following further discussion. So the equivalent reloption remains in place though is disabled, just for compatibility purposes -- on v13 and v14. The GUC is where v13 and v14 differ -- only v13 still has the GUC to avoid breaking user's postgresql.conf files (though it's also delisted). Deprecating a reloption is much harder than deprecating a GUC. In my opinion this should be interpreted as already handled by the backpatch to 13.3, and so not necessary to handle again now -- despite the GUC thing. It's possible that no users set the GUC at all, because it wasn't particularly well thought out. This entire situation is probably unprecedented (we just don't deprecate reloptions very often), so I defer to your judgement, Bruce. -- Peter Geoghegan
On Tue, May 18, 2021 at 08:34:55AM -0700, Peter Geoghegan wrote: > In my opinion this should be interpreted as already handled by the > backpatch to 13.3, and so not necessary to handle again now -- despite > the GUC thing. It's possible that no users set the GUC at all, because > it wasn't particularly well thought out. This entire situation is > probably unprecedented (we just don't deprecate reloptions very > often), so I defer to your judgement, Bruce. I am thiking the vacuum_cleanup_index_scale_factor existance in a PG 14 postgresql.conf will throw an error, unlike 13.x, so I do think we need to mention this so people will know to remove it from their postgresql.conf before upgrades, right? I don't think the PG 13.3 release note mention really makes it clear it has to be removed. In a dump/restore, so we retain the reloption vacuum_cleanup_index_scale_factor and just ignore it, or drop it on restore? I am hoping it is the later. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Tue, May 18, 2021 at 8:48 AM Bruce Momjian <bruce@momjian.us> wrote: > I am thiking the vacuum_cleanup_index_scale_factor existance in a PG 14 > postgresql.conf will throw an error, unlike 13.x, so I do think we need > to mention this so people will know to remove it from their > postgresql.conf before upgrades, right? I don't think the PG 13.3 > release note mention really makes it clear it has to be removed. In a > dump/restore, so we retain the reloption > vacuum_cleanup_index_scale_factor and just ignore it, or drop it on > restore? I am hoping it is the later. There is no dump/restore hazard on upgrade to 14, since the vacuum_cleanup_index_scale_factor reloption remains in place (it's just not in psql tab completion anymore, nor is it documented, etc). It is possible (though I would certainly say unlikely) that the vacuum_cleanup_index_scale_factor GUC will be in somebody's postgresql.conf from an earlier version, and won't be recognized on upgrade to v14. So maybe you need to say something about that particular issue -- which could be framed as finishing off the process started by the 13.3 commit. But that's it. -- Peter Geoghegan
On Tue, May 18, 2021 at 08:54:56AM -0700, Peter Geoghegan wrote: > On Tue, May 18, 2021 at 8:48 AM Bruce Momjian <bruce@momjian.us> wrote: > > I am thiking the vacuum_cleanup_index_scale_factor existance in a PG 14 > > postgresql.conf will throw an error, unlike 13.x, so I do think we need > > to mention this so people will know to remove it from their > > postgresql.conf before upgrades, right? I don't think the PG 13.3 > > release note mention really makes it clear it has to be removed. In a > > dump/restore, so we retain the reloption > > vacuum_cleanup_index_scale_factor and just ignore it, or drop it on > > restore? I am hoping it is the later. > > There is no dump/restore hazard on upgrade to 14, since the > vacuum_cleanup_index_scale_factor reloption remains in place (it's > just not in psql tab completion anymore, nor is it documented, etc). So it is the former behavior --- "so we retain the reloption vacuum_cleanup_index_scale_factor and just ignore it"? > It is possible (though I would certainly say unlikely) that the > vacuum_cleanup_index_scale_factor GUC will be in somebody's > postgresql.conf from an earlier version, and won't be recognized on > upgrade to v14. So maybe you need to say something about that > particular issue -- which could be framed as finishing off the process > started by the 13.3 commit. But that's it. Yes. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
We do retain the reloption, but ignore it. Purely to avoid the dump and reload hazard. I think that you could reasonably tell users that it's gone completely, because it does nothing on either 13 or 14. It's hidden from them to the extent that that's possible.
Peter Geoghegan
(Sent from my phone)
(Sent from my phone)
On Tue, May 18, 2021 at 09:01:44AM -0700, Peter Geoghegan wrote:
> We do retain the reloption, but ignore it. Purely to avoid the dump and reload
> hazard. I think that you could reasonably tell users that it's gone completely,
> because it does nothing on either 13 or 14. It's hidden from them to the extent
> that that's possible.
I went with this release note text:
<listitem>
<!--
Author: Peter Geoghegan <pg@bowt.ie>
2021-03-10 [9f3665fbf] Don't consider newly inserted tuples in nbtree VACUUM.
Author: Peter Geoghegan <pg@bowt.ie>
2021-03-11 [effdd3f3b] Add back vacuum_cleanup_index_scale_factor parameter.
-->
<para>
Remove server variable vacuum_cleanup_index_scale_factor (Peter Geoghegan)
</para>
<para>
This setting was disabled in PostgreSQL version 13.3.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Tue, May 18, 2021 at 12:17 PM Bruce Momjian <bruce@momjian.us> wrote: > I went with this release note text: That seems reasonable -- thanks! -- Peter Geoghegan
These sound weird since markup was added in 6a5bde7d4: https://www.postgresql.org/docs/devel/release-14.html | Remove server and Chapter 34 support for the version 2 wire protocol (Heikki Linnakangas) ... | Pass doubled quote marks in Chapter 36 SQL command strings literally (Tom Lane) -Remove server and libpq support for the version 2 wire protocol (Heikki Linnakangas) +Remove server and <xref linkend="libpq"/> support for the version 2 <link linkend="protocol">wire protocol</link> (HeikkiLinnakangas) > Force custom server variable names to match the pattern used for unquoted SQL identifiers (Tom Lane) Say "Require" not force? > This setting was disabled in PostgreSQL version 13.3. "disabled" sounds like it was set to "off". Maybe say it was ignored. > Add long-running queries to be canceled if the client disconnects (Sergey Cherkashin, Thomas Munro) Should say: Allow > The server variable client_connection_check_interval allows supporting operating systems, e.g., Linux, to automaticallycancel queries by disconnected clients. say "some operating systems" ? > This can be disabled by turning client option "sslsni" off. "turning off" | Add %P to log_line_prefix to report the parallel group leader (Justin Pryzby) Maybe it should say "Allow %P in log_line_prefix to ...", otherwise it sounds like the default was changed. | Reduce the default value of vacuum_cost_page_miss (Peter Geoghegan) | This new default better reflects current hardware capabilities. Also say: the previous default was 10.
On Wed, May 19, 2021 at 09:39:08AM -0500, Justin Pryzby wrote: > These sound weird since markup was added in 6a5bde7d4: > https://www.postgresql.org/docs/devel/release-14.html > | Remove server and Chapter 34 support for the version 2 wire protocol (Heikki Linnakangas) > ... > | Pass doubled quote marks in Chapter 36 SQL command strings literally (Tom Lane) > -Remove server and libpq support for the version 2 wire protocol (Heikki Linnakangas) > +Remove server and <xref linkend="libpq"/> support for the version 2 <link linkend="protocol">wire protocol</link> (HeikkiLinnakangas) Agreed, fixed. > > Force custom server variable names to match the pattern used for unquoted SQL identifiers (Tom Lane) > Say "Require" not force? Agreed, fixed. > > This setting was disabled in PostgreSQL version 13.3. > "disabled" sounds like it was set to "off". Maybe say it was ignored. OK, I went with this: This setting was ignored starting in <productname>PostgreSQL</productname> version 13.3. > > Add long-running queries to be canceled if the client disconnects (Sergey Cherkashin, Thomas Munro) > Should say: Allow Yes. > > The server variable client_connection_check_interval allows supporting operating systems, e.g., Linux, to automaticallycancel queries by disconnected clients. > say "some operating systems" ? Agreed, done. > > This can be disabled by turning client option "sslsni" off. > "turning off" Agreed. > | Add %P to log_line_prefix to report the parallel group leader (Justin Pryzby) > > Maybe it should say "Allow %P in log_line_prefix to ...", otherwise it sounds > like the default was changed. I am not sure, but I changed it as you suggested: Allow %P in <varname>log_line_prefix</varname> to report the parallel group leader (Justin Pryzby) > | Reduce the default value of vacuum_cost_page_miss (Peter Geoghegan) > | This new default better reflects current hardware capabilities. > Also say: the previous default was 10. Uh, we didn't report the new value, so why report the old one? -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Thu, May 20, 2021 at 03:44:46PM -0400, Bruce Momjian wrote: > > | Reduce the default value of vacuum_cost_page_miss (Peter Geoghegan) > > | This new default better reflects current hardware capabilities. > > Also say: the previous default was 10. > > Uh, we didn't report the new value, so why report the old one? Good point. For symmetry with this one, maybe the old and new values should be included? |Change checkpoint_completion_target default to 0.9 (Stephen Frost) |The previous default was 0.5. -- Justin
On Thu, May 20, 2021 at 02:55:18PM -0500, Justin Pryzby wrote: > On Thu, May 20, 2021 at 03:44:46PM -0400, Bruce Momjian wrote: > > > | Reduce the default value of vacuum_cost_page_miss (Peter Geoghegan) > > > | This new default better reflects current hardware capabilities. > > > Also say: the previous default was 10. > > > > Uh, we didn't report the new value, so why report the old one? > > Good point. > For symmetry with this one, maybe the old and new values should be included? Not sure. Those values are kind of hard to understand, so I am afraid there would be more confusion by mentioning them. > |Change checkpoint_completion_target default to 0.9 (Stephen Frost) > |The previous default was 0.5. Uh, that one is frequently modified by users, to an extent I didn't understand why we kept it at 0.5 for so long, which is why I mentioned it. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On 2021-May-20, Bruce Momjian wrote: > On Thu, May 20, 2021 at 02:55:18PM -0500, Justin Pryzby wrote: > > On Thu, May 20, 2021 at 03:44:46PM -0400, Bruce Momjian wrote: > > > > | Reduce the default value of vacuum_cost_page_miss (Peter Geoghegan) > > > > | This new default better reflects current hardware capabilities. > > > > Also say: the previous default was 10. > > > > > > Uh, we didn't report the new value, so why report the old one? > > > > Good point. > > For symmetry with this one, maybe the old and new values should be included? > > Not sure. Those values are kind of hard to understand, so I am afraid > there would be more confusion by mentioning them. > > > |Change checkpoint_completion_target default to 0.9 (Stephen Frost) > > |The previous default was 0.5. > > Uh, that one is frequently modified by users, to an extent I didn't > understand why we kept it at 0.5 for so long, which is why I mentioned > it. You also mentioned 'md5' in the entry about password_encryption, remember? I tend to agree with Justin: if it's not too much extra space to mention both values, let's just do that. "Reduce the value of X to Y from Z. The new default better reflects ..." seems OK to me. I prefer "to Y from Z" rather than "from Z to Y", because then the new value appears first, which seems a tiny improvement in readability, though the phrase is in the opposite order of traditional. Also it seems better than "change value of X to Y. The previous default was Z" because it then becomes a little more verbose than really needed. But maybe that's OK too. -- Álvaro Herrera 39°49'30"S 73°17'W "La fuerza no está en los medios físicos sino que reside en una voluntad indomable" (Gandhi)
On Thu, May 20, 2021 at 04:35:07PM -0400, Álvaro Herrera wrote:
> On 2021-May-20, Bruce Momjian wrote:
>
> > On Thu, May 20, 2021 at 02:55:18PM -0500, Justin Pryzby wrote:
> > > On Thu, May 20, 2021 at 03:44:46PM -0400, Bruce Momjian wrote:
> > > > > | Reduce the default value of vacuum_cost_page_miss (Peter Geoghegan)
> > > > > | This new default better reflects current hardware capabilities.
> > > > > Also say: the previous default was 10.
> > > >
> > > > Uh, we didn't report the new value, so why report the old one?
> > >
> > > Good point.
> > > For symmetry with this one, maybe the old and new values should be included?
> >
> > Not sure. Those values are kind of hard to understand, so I am afraid
> > there would be more confusion by mentioning them.
> >
> > > |Change checkpoint_completion_target default to 0.9 (Stephen Frost)
> > > |The previous default was 0.5.
> >
> > Uh, that one is frequently modified by users, to an extent I didn't
> > understand why we kept it at 0.5 for so long, which is why I mentioned
> > it.
>
> You also mentioned 'md5' in the entry about password_encryption,
> remember? I tend to agree with Justin: if it's not too much extra space
> to mention both values, let's just do that. "Reduce the value of X to Y
> from Z. The new default better reflects ..." seems OK to me.
>
> I prefer "to Y from Z" rather than "from Z to Y", because then the new
> value appears first, which seems a tiny improvement in readability,
> though the phrase is in the opposite order of traditional. Also it
> seems better than "change value of X to Y. The previous default was Z"
> because it then becomes a little more verbose than really needed. But
> maybe that's OK too.
I went with this text:
Reduce the default value of <varname><xref
linkend="guc-vacuum-cost-page-miss"></varname> from 10 milliseconds to 2
(Peter Geoghegan)
I think with numbers, and the fact we are saying "decrease" havint them
in the from/to order is best. If this was non-numeric, like to scram
from md5, it would make more sense to use to/from.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Thu, May 20, 2021 at 2:37 PM Bruce Momjian <bruce@momjian.us> wrote: > I went with this text: > > Reduce the default value of <varname><xref > linkend="guc-vacuum-cost-page-miss"></varname> from 10 milliseconds to 2 > (Peter Geoghegan) > > I think with numbers, and the fact we are saying "decrease" havint them > in the from/to order is best. If this was non-numeric, like to scram > from md5, it would make more sense to use to/from. The point of this change was to make the cost of dirtying pages much higher than everything else, since writes are in effect much more expensive on modern hardware. Don't know if you need to say that. -- Peter Geoghegan
On Thu, May 20, 2021 at 04:19:50PM -0700, Peter Geoghegan wrote: > On Thu, May 20, 2021 at 2:37 PM Bruce Momjian <bruce@momjian.us> wrote: > > I went with this text: > > > > Reduce the default value of <varname><xref > > linkend="guc-vacuum-cost-page-miss"></varname> from 10 milliseconds to 2 > > (Peter Geoghegan) > > > > I think with numbers, and the fact we are saying "decrease" havint them > > in the from/to order is best. If this was non-numeric, like to scram > > from md5, it would make more sense to use to/from. > > The point of this change was to make the cost of dirtying pages much > higher than everything else, since writes are in effect much more > expensive on modern hardware. Don't know if you need to say that. I think our text "This new default better reflects current hardware capabilities." is detailed enough. People can dig into the item to see what it does and how it adjusts costs. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Thu, May 20, 2021 at 4:54 PM Bruce Momjian <bruce@momjian.us> wrote: > I think our text "This new default better reflects current hardware > capabilities." is detailed enough. People can dig into the item to see > what it does and how it adjusts costs. Fair enough. I noticed something about the same item that needs to be fixed, though. The vacuum_cost_page_miss GUC does not directly represent any kind of time-based delay, but the current wording says that it uses millisecond units. In fact the vacuum_cost_page_miss GUC is based on abstract cost units, apportioned from vacuum_cost_limit. I suggested that the wording talk about relative cost differences in part because that's just how the GUC works, in general. -- Peter Geoghegan
On Sat, May 22, 2021 at 03:35:13PM -0700, Peter Geoghegan wrote:
> On Thu, May 20, 2021 at 4:54 PM Bruce Momjian <bruce@momjian.us> wrote:
> > I think our text "This new default better reflects current hardware
> > capabilities." is detailed enough. People can dig into the item to see
> > what it does and how it adjusts costs.
>
> Fair enough.
>
> I noticed something about the same item that needs to be fixed,
> though. The vacuum_cost_page_miss GUC does not directly represent any
> kind of time-based delay, but the current wording says that it uses
> millisecond units. In fact the vacuum_cost_page_miss GUC is based on
> abstract cost units, apportioned from vacuum_cost_limit. I suggested
> that the wording talk about relative cost differences in part because
> that's just how the GUC works, in general.
OK, thanks. Here is the updated text:
<listitem>
<!--
Author: Peter Geoghegan <pg@bowt.ie>
2021-01-27 [e19594c5c] Reduce the default value of
vacuum_cost_page_miss.
-->
<para>
Reduce the default value of <xref linkend="guc-vacuum-cost-page-miss"/>
to better reflects current hardware capabilities (Peter Geoghegan)
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
Greetings, * Bruce Momjian (bruce@momjian.us) wrote: > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html It occurs to me that the wording around the new default roles could probably be better. Specifically: Add predefined roles pg_read_all_data and pg_write_all_data (Stephen Frost) These non-login roles give read-only/write-only access to all objects. Might be better as: These non-login roles give read, or write, access to all tables, views, and sequences. (These roles don't actually allow, for example, a function to be redefined, so saying 'all objects' isn't quite right either.) While these roles could be used to create a 'read only' or 'write only' role, they, themselves, do not explicitly convey that on to a role because they don't do anything to prevent someone from GRANT'ing other rights to some role which has been GRANT'd these predefined roles. I don't think anyone on this list thought differently from that, but the phrasing strikes me as potentially confusing. Maybe another way would be: These non-login roles give (only) read, or write, access to all tables, views, and sequences. but I don't think saying 'only' there really adds anything and instead invites confusion. Thanks! Stephen
Attachment
On Sat, May 22, 2021 at 07:29:45PM -0400, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (bruce@momjian.us) wrote: > > I have committed the first draft of the PG 14 release notes. You can > > see the most current build of them here: > > > > https://momjian.us/pgsql_docs/release-14.html > > It occurs to me that the wording around the new default roles could > probably be better. Specifically: > > Add predefined roles pg_read_all_data and pg_write_all_data (Stephen Frost) > > These non-login roles give read-only/write-only access to all objects. > > Might be better as: > > These non-login roles give read, or write, access to all tables, views, > and sequences. > > (These roles don't actually allow, for example, a function to be > redefined, so saying 'all objects' isn't quite right either.) > > While these roles could be used to create a 'read only' or 'write only' > role, they, themselves, do not explicitly convey that on to a role > because they don't do anything to prevent someone from GRANT'ing other > rights to some role which has been GRANT'd these predefined roles. I > don't think anyone on this list thought differently from that, but the > phrasing strikes me as potentially confusing. > > Maybe another way would be: > > These non-login roles give (only) read, or write, access to all tables, > views, and sequences. > > but I don't think saying 'only' there really adds anything and instead > invites confusion. OK, I went with this text: <listitem> <!-- Author: Stephen Frost <sfrost@snowman.net> 2021-04-05 [6c3ffd697] Add pg_read_all_data and pg_write_all_data roles --> <para> Add predefined roles <link linkend="predefined-roles"><structname>pg_read_all_data</structname></link> and <structname>pg_write_all_data</structname> (Stephen Frost) </para> <para> These non-login roles can be used to give read or write permission to all tables, views, and sequences. </para> </listitem> -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
doc/src/sgml/release-14.sgml- <listitem> doc/src/sgml/release-14.sgml-<!-- doc/src/sgml/release-14.sgml-Author: Fujii Masao <fujii@postgresql.org> doc/src/sgml/release-14.sgml-2020-10-02 [8d9a93596] Add pg_stat_wal statistics view. doc/src/sgml/release-14.sgml-Author: Fujii Masao <fujii@postgresql.org> doc/src/sgml/release-14.sgml-2020-12-02 [01469241b] Track total number of WAL records, FPIs and bytes genera doc/src/sgml/release-14.sgml-Author: Fujii Masao <fujii@postgresql.org> doc/src/sgml/release-14.sgml:2021-03-09 [ff99918c6] Track total amounts of times spent writing and syncing W doc/src/sgml/release-14.sgml---> doc/src/sgml/release-14.sgml- doc/src/sgml/release-14.sgml- <para> doc/src/sgml/release-14.sgml- Add system view <link doc/src/sgml/release-14.sgml- linkend="monitoring-pg-stat-wal-view"><structname>pg_stat_wal</structname></link> doc/src/sgml/release-14.sgml- which reports <acronym>WAL</acronym> activity (Masahiro Ikeda) doc/src/sgml/release-14.sgml- </para> doc/src/sgml/release-14.sgml- </listitem> I think this should also mention the GUC: | track_wal_io_timing -- Justin
On Fri, Jun 4, 2021 at 04:31:06PM -0500, Justin Pryzby wrote:
> doc/src/sgml/release-14.sgml- <listitem>
> doc/src/sgml/release-14.sgml-<!--
> doc/src/sgml/release-14.sgml-Author: Fujii Masao <fujii@postgresql.org>
> doc/src/sgml/release-14.sgml-2020-10-02 [8d9a93596] Add pg_stat_wal statistics view.
> doc/src/sgml/release-14.sgml-Author: Fujii Masao <fujii@postgresql.org>
> doc/src/sgml/release-14.sgml-2020-12-02 [01469241b] Track total number of WAL records, FPIs and bytes genera
> doc/src/sgml/release-14.sgml-Author: Fujii Masao <fujii@postgresql.org>
> doc/src/sgml/release-14.sgml:2021-03-09 [ff99918c6] Track total amounts of times spent writing and syncing W
> doc/src/sgml/release-14.sgml--->
> doc/src/sgml/release-14.sgml-
> doc/src/sgml/release-14.sgml- <para>
> doc/src/sgml/release-14.sgml- Add system view <link
> doc/src/sgml/release-14.sgml-
linkend="monitoring-pg-stat-wal-view"><structname>pg_stat_wal</structname></link>
> doc/src/sgml/release-14.sgml- which reports <acronym>WAL</acronym> activity (Masahiro Ikeda)
> doc/src/sgml/release-14.sgml- </para>
> doc/src/sgml/release-14.sgml- </listitem>
>
> I think this should also mention the GUC:
> | track_wal_io_timing
This is part of this commit:
commit ff99918c62
Author: Fujii Masao <fujii@postgresql.org>
Date: Tue Mar 9 16:52:06 2021 +0900
Track total amounts of times spent writing and syncing WAL data to disk.
This commit adds new GUC track_wal_io_timing. When this is enabled,
the total amounts of time XLogWrite writes and issue_xlog_fsync syncs
WAL data to disk are counted in pg_stat_wal. This information would be
useful to check how much WAL write and sync affect the performance.
The feature is clearly the new system view pg_stat_wal. The question is
whether we should also document that track_wal_io_timing adds timing
control to that new system view. Normally we don't mention additional
features, like pg_dump support, for features we add in a major release.
The documentation we link to from the release notes does mention
track_wal_io_timing.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
| Add Set Server Name Indication (SNI) for SSL connection packets (Peter Eisentraut) Remove "Set" | Reduce the default value of vacuum_cost_page_miss from 10 milliseconds to 2 (Peter Geoghegan) Peter mentioned that this should not say "milliseconds" (but maybe the page I'm looking at is old). | Cause vacuum operations to be aggressive if the table is near xid or multixact wraparound (Masahiko Sawada, Peter Geoghegan) Say "become aggressive" ? | Allow the arbitrary collations of partition boundary values (Tom Lane) Remove "the" | Generate WAL invalidations message during command completion when using logical replication (Dilip Kumar, Tomas Vondra,Amit Kapila) invalidation messages | Add support for infinity and -infinity values to the numeric data type (Tom Lane) "-infinity" has markup but not "infinity" ? | Allow vacuum to deallocate space reserved by trailing unused heap line pointers (Matthias van de Meent, Peter Geoghegan) say "reclaim space" ?
On Fri, Jun 11, 2021 at 10:45:51PM -0500, Justin Pryzby wrote: > | Add Set Server Name Indication (SNI) for SSL connection packets (Peter Eisentraut) > Remove "Set" > > | Reduce the default value of vacuum_cost_page_miss from 10 milliseconds to 2 (Peter Geoghegan) > Peter mentioned that this should not say "milliseconds" (but maybe the page I'm > looking at is old). > > | Cause vacuum operations to be aggressive if the table is near xid or multixact wraparound (Masahiko Sawada, Peter Geoghegan) > Say "become aggressive" ? > > | Allow the arbitrary collations of partition boundary values (Tom Lane) > Remove "the" > > | Generate WAL invalidations message during command completion when using logical replication (Dilip Kumar, Tomas Vondra,Amit Kapila) > invalidation messages > > | Add support for infinity and -infinity values to the numeric data type (Tom Lane) > "-infinity" has markup but not "infinity" ? > > | Allow vacuum to deallocate space reserved by trailing unused heap line pointers (Matthias van de Meent, Peter Geoghegan) > say "reclaim space" ? Some more: | VACUUM now has a PROCESS_TOAST which can be set to false to disable TOAST processing, and vacuumdb has a --no-process-toastoption. has a process_toast *option | Previously, if the object already exists, EXPLAIN would fail. already existed | Function pg_stat_reset_replication_slot() resets slot statistics. *The function. But maybe it should be omitted. | New options are read-only, primary, standby, and prefer-standby. *The new options | Allow reindexdb to change the tablespace of the new index (Michael Paquier) | This is done by specifying --tablespace. I think this should be merged with the corresponding server feature, like this one: |Add ability to skip vacuuming of TOAST tables (Nathan Bossart) |VACUUM now has a PROCESS_TOAST which can be set to false to disable TOAST processing, and vacuumdb has a --no-process-toastoption. Or, the client-side option could be omitted. This is distinguished from vacuumdb --no-index-cleanup and --no-truncate, for which the server support was added in v12, and the client support was essentially an omision. | Add documentation for the factorial() function (Peter Eisentraut) | With the removal of the ! operator in this release, factorial() is the only built-in way to compute a factorial. Could be ommited or collapsed into the other item. I know Tom thinks that it's unnecesary to document changes to documentation. -- Justin
On 5/10/21 8:03 AM, Bruce Momjian wrote: > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > > I need clarification on many items, and the document still needs its > items properly ordered, and markup added. I also expect a lot of > feedback. > > I plan to work on completing this document this coming week in > preparation for beta next week. > Sorry it took me a while to look at the release notes. I have one suggestion regarding this item: Allow logical replication to stream long in-progress transactions to subscribers (Tomas Vondra, Dilip Kumar, Amit Kapila, Ajin Cherian, Nikhil Sontakke, Stas Kelvich) AFAICS the authors are generally ordered by how much they contributed to the feature. In that case I'd move myself down the list - certainly after Dilip and Amit, perhaps after Ajin. While I posted the original patch, but most of the work after that to get it committed was done by those two/three people. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Hi Bruce,
For this item:
<!--
Author: Tomas Vondra <tomas.vondra@postgresql.org>
2021-03-26 [77b88cd1b] BRIN bloom indexes
-->
<para>
Allow <acronym>BRIN</acronym> indexes to use bloom filters
(Tomas Vondra)
</para>
<para>
This allows bloom indexes to be used effectively with data that
is not physically localized in the heap.
</para>
</listitem>
Author: Tomas Vondra <tomas.vondra@postgresql.org>
2021-03-26 [77b88cd1b] BRIN bloom indexes
-->
<para>
Allow <acronym>BRIN</acronym> indexes to use bloom filters
(Tomas Vondra)
</para>
<para>
This allows bloom indexes to be used effectively with data that
is not physically localized in the heap.
</para>
</listitem>
The text implies that this affects contrib/bloom. I think it should be "This allows BRIN indexes...".
On Fri, Jun 11, 2021 at 10:45:51PM -0500, Justin Pryzby wrote:
> | Add Set Server Name Indication (SNI) for SSL connection packets (Peter Eisentraut)
> Remove "Set"
Fixed.
> | Reduce the default value of vacuum_cost_page_miss from 10 milliseconds to 2 (Peter Geoghegan)
> Peter mentioned that this should not say "milliseconds" (but maybe the page I'm
> looking at is old).
It is old. It is now:
Reduce the default value of <xref
linkend="guc-vacuum-cost-page-miss"/> to better reflects current
hardware capabilities (Peter Geoghegan)
> | Cause vacuum operations to be aggressive if the table is near xid or multixact wraparound (Masahiko Sawada, Peter
Geoghegan)
> Say "become aggressive" ?
Updated text:
Cause vacuum operations to be more aggressive if the table is near
xid or multixact wraparound (Masahiko Sawada, Peter Geoghegan)
> | Allow the arbitrary collations of partition boundary values (Tom Lane)
> Remove "the"
Agreed, removed.
> | Generate WAL invalidations message during command completion when using logical replication (Dilip Kumar, Tomas
Vondra,Amit Kapila)
> invalidation messages
Fixed.
> | Add support for infinity and -infinity values to the numeric data type (Tom Lane)
> "-infinity" has markup but not "infinity" ?
Fixed.
>
> | Allow vacuum to deallocate space reserved by trailing unused heap line pointers (Matthias van de Meent, Peter
Geoghegan)
> say "reclaim space" ?
OK, new text is:
Allow vacuum to reclaim space used by unused trailing heap
line pointers (Matthias van de Meent, Peter Geoghegan)
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Mon, Jun 14, 2021 at 11:37:58AM -0500, Justin Pryzby wrote:
> Some more:
>
> | VACUUM now has a PROCESS_TOAST which can be set to false to disable TOAST processing, and vacuumdb has a
--no-process-toastoption.
> has a process_toast *option
Agreed.
> | Previously, if the object already exists, EXPLAIN would fail.
> already existed
Fixed.
> | Function pg_stat_reset_replication_slot() resets slot statistics.
> *The function. But maybe it should be omitted.
OK, I went with "The function".
> | New options are read-only, primary, standby, and prefer-standby.
> *The new options
Agreed.
> | Allow reindexdb to change the tablespace of the new index (Michael Paquier)
> | This is done by specifying --tablespace.
> I think this should be merged with the corresponding server feature, like this one:
> |Add ability to skip vacuuming of TOAST tables (Nathan Bossart)
> |VACUUM now has a PROCESS_TOAST which can be set to false to disable TOAST processing, and vacuumdb has a
--no-process-toastoption.
>
> Or, the client-side option could be omitted. This is distinguished from
> vacuumdb --no-index-cleanup and --no-truncate, for which the server support was
> added in v12, and the client support was essentially an omision.
I am inclined to mention reindexdb because we mention the SQL command
option in the text. Here is the updated text:
<listitem>
<!--
Author: Michael Paquier <michael@paquier.xyz>
2021-02-04 [c5b286047] Add TABLESPACE option to REINDEX
Author: Michael Paquier <michael@paquier.xyz>
2021-03-03 [57e6db706] Add - -tablespace option to reindexdb
-->
<para>
Allow <link linkend="sql-reindex"><command>REINDEX</command></link>
to change the tablespace of the new index (Alexey Kondratov,
Michael Paquier, Justin Pryzby)
</para>
<para>
This is done by specifying a <literal>TABLESPACE</literal> clause.
A <option>--tablespace</option> option was also added to <link
linkend="app-reindexdb"><application>reindexdb</application></link>
to control this.
</listitem>
> | Add documentation for the factorial() function (Peter Eisentraut)
> | With the removal of the ! operator in this release, factorial() is the only built-in way to compute a factorial.
> Could be ommited or collapsed into the other item. I know Tom thinks that
> it's unnecesary to document changes to documentation.
Uh, I think we need both items. We are removing a feature and asking
people to use an existing feature that was previously undocumented. I
think having two items makes it clear that the feature existed in
previous releases. I just tried merging them into one item and there
were just too many changes for it to be clear.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Mon, Jun 14, 2021 at 06:57:41PM +0200, Tomas Vondra wrote:
> Sorry it took me a while to look at the release notes. I have one
> suggestion regarding this item:
>
>
> Allow logical replication to stream long in-progress transactions to
> subscribers (Tomas Vondra, Dilip Kumar, Amit Kapila, Ajin Cherian,
> Nikhil Sontakke, Stas Kelvich)
>
> AFAICS the authors are generally ordered by how much they contributed to
> the feature. In that case I'd move myself down the list - certainly
> after Dilip and Amit, perhaps after Ajin. While I posted the original
> patch, but most of the work after that to get it committed was done by
> those two/three people.
OK, I moved you after Ajin. Sometimes it isn't clear how much of an
original patch was modified by later authors.
FYI, the most recent PG 14 relnote doc build is at:
https://momjian.us/pgsql_docs/release-14.html
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Mon, Jun 14, 2021 at 01:29:42PM -0400, John Naylor wrote:
> Hi Bruce,
>
> For this item:
>
> <!--
> Author: Tomas Vondra <tomas.vondra@postgresql.org>
> 2021-03-26 [77b88cd1b] BRIN bloom indexes
> -->
>
> <para>
> Allow <acronym>BRIN</acronym> indexes to use bloom filters
> (Tomas Vondra)
> </para>
>
> <para>
> This allows bloom indexes to be used effectively with data that
> is not physically localized in the heap.
> </para>
> </listitem>
>
> The text implies that this affects contrib/bloom. I think it should be "This
> allows BRIN indexes...".
Ah, I see your point. Updated text is:
<listitem>
<!--
Author: Tomas Vondra <tomas.vondra@postgresql.org>
2021-03-26 [77b88cd1b] BRIN bloom indexes
-->
<para>
Allow <acronym>BRIN</acronym> indexes to use bloom filters
(Tomas Vondra)
</para>
<para>
This allows <acronym>BRIN</acronym> indexes to be used effectively
with data that is not physically localized in the heap.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Mon, May 10, 2021 at 3:03 PM Bruce Momjian <bruce@momjian.us> wrote: > > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > It might have already been discussed but I think we should mention commit bed90759f in the release note. The commit seems like a bug fix but it not back-patched to the older versions at least for now. There is a discussion[1] that we will revisit that a couple of months after 14 is released so as there is some feedback from the field with this change. Regards, [1] https://www.postgresql.org/message-id/YCsZIX2A2Ilsvfnl@paquier.xyz -- Masahiko Sawada EDB: https://www.enterprisedb.com/
On Tue, Jun 15, 2021 at 10:06:49AM +0900, Masahiko Sawada wrote: > On Mon, May 10, 2021 at 3:03 PM Bruce Momjian <bruce@momjian.us> wrote: > > > > I have committed the first draft of the PG 14 release notes. You can > > see the most current build of them here: > > > > https://momjian.us/pgsql_docs/release-14.html > > > > It might have already been discussed but I think we should mention > commit bed90759f in the release note. The commit seems like a bug fix > but it not back-patched to the older versions at least for now. There > is a discussion[1] that we will revisit that a couple of months after > 14 is released so as there is some feedback from the field with this > change. OK, but I need more information on how users will see a difference based on this commit: commit bed90759fc Author: Tom Lane <tgl@sss.pgh.pa.us> Date: Fri Oct 9 16:20:12 2020 -0400 Fix our Windows stat() emulation to handle file sizes > 4GB. Hack things so that our idea of "struct stat" is equivalent to Windows' struct __stat64, allowing it to have a wide enough st_size field. Instead of relying on native stat(), use GetFileInformationByHandle(). This avoids a number of issues with Microsoft's multiple and rather slipshod emulations of stat(). We still need to jump through hoops to deal with ERROR_DELETE_PENDING, though :-( Pull the relevant support code out of dirmod.c and put it into its own file, win32stat.c. Still TODO: do we need to do something different with lstat(), rather than treating it identically to stat()? Juan José Santamaría Flecha, reviewed by Emil Iggland; based on prior work by Michael Paquier, Sergey Zubkovsky, and others Discussion: https://postgr.es/m/1803D792815FC24D871C00D17AE95905CF5099@g01jpexmbkw24 Discussion: https://postgr.es/m/15858-9572469fd3b73263@postgresql.org -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Tue, Jun 15, 2021 at 10:36 AM Bruce Momjian <bruce@momjian.us> wrote: > > On Tue, Jun 15, 2021 at 10:06:49AM +0900, Masahiko Sawada wrote: > > On Mon, May 10, 2021 at 3:03 PM Bruce Momjian <bruce@momjian.us> wrote: > > > > > > I have committed the first draft of the PG 14 release notes. You can > > > see the most current build of them here: > > > > > > https://momjian.us/pgsql_docs/release-14.html > > > > > > > It might have already been discussed but I think we should mention > > commit bed90759f in the release note. The commit seems like a bug fix > > but it not back-patched to the older versions at least for now. There > > is a discussion[1] that we will revisit that a couple of months after > > 14 is released so as there is some feedback from the field with this > > change. > > OK, but I need more information on how users will see a difference based > on this commit: I think that since with this commit the server on Windows can handle a file over 4GB, COPY FROM loading data from an over 4GB file and pg_dump dumping a large table work now. Regards, -- Masahiko Sawada EDB: https://www.enterprisedb.com/
On Tue, Jun 15, 2021 at 11:49:21AM +0900, Masahiko Sawada wrote: > On Tue, Jun 15, 2021 at 10:36 AM Bruce Momjian <bruce@momjian.us> wrote: >> OK, but I need more information on how users will see a difference based >> on this commit: +1. That would be good to have in the release notes. > I think that since with this commit the server on Windows can handle a > file over 4GB, COPY FROM loading data from an over 4GB file and > pg_dump dumping a large table work now. Segment files or WAL files larger than 4GB also gain from that. Anything for which we may finish to do a stat() on benefits from this change if running on Windows. For pg_dump, a workaround in PG <= 13 was to use --no-sync as the stat() failure came from files with a size larger than 4GB. That's rather sad as that means sacrifying durability for more usability :( -- Michael
Attachment
On Mon, Jun 14, 2021 at 1:11 PM Bruce Momjian <bruce@momjian.us> wrote: > FYI, the most recent PG 14 relnote doc build is at: > > https://momjian.us/pgsql_docs/release-14.html I just pushed a commit that makes the existing vacuum_index_cleanup reloption and INDEX_CLEANUP VACUUM parameter support disabling the "Allow vacuum to skip index vacuuming when the number of removable index entries is insignificant" behavior. This should be mentioned in the release notes. -- Peter Geoghegan
On Tue, Jun 15, 2021 at 12:01:00PM +0900, Michael Paquier wrote: > On Tue, Jun 15, 2021 at 11:49:21AM +0900, Masahiko Sawada wrote: > > On Tue, Jun 15, 2021 at 10:36 AM Bruce Momjian <bruce@momjian.us> wrote: > >> OK, but I need more information on how users will see a difference based > >> on this commit: > > +1. That would be good to have in the release notes. > > > I think that since with this commit the server on Windows can handle a > > file over 4GB, COPY FROM loading data from an over 4GB file and > > pg_dump dumping a large table work now. > > Segment files or WAL files larger than 4GB also gain from that. > Anything for which we may finish to do a stat() on benefits from this > change if running on Windows. For pg_dump, a workaround in PG <= 13 > was to use --no-sync as the stat() failure came from files with a size > larger than 4GB. That's rather sad as that means sacrifying > durability for more usability :( OK, I went with this text and put it in the Source Code section since it applies to several layers of Postgres. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
Attachment
On Mon, Jun 21, 2021 at 12:50 PM Bruce Momjian <bruce@momjian.us> wrote:
>
> On Tue, Jun 15, 2021 at 12:01:00PM +0900, Michael Paquier wrote:
> > On Tue, Jun 15, 2021 at 11:49:21AM +0900, Masahiko Sawada wrote:
> > > On Tue, Jun 15, 2021 at 10:36 AM Bruce Momjian <bruce@momjian.us> wrote:
> > >> OK, but I need more information on how users will see a difference based
> > >> on this commit:
> >
> > +1. That would be good to have in the release notes.
> >
> > > I think that since with this commit the server on Windows can handle a
> > > file over 4GB, COPY FROM loading data from an over 4GB file and
> > > pg_dump dumping a large table work now.
> >
> > Segment files or WAL files larger than 4GB also gain from that.
> > Anything for which we may finish to do a stat() on benefits from this
> > change if running on Windows. For pg_dump, a workaround in PG <= 13
> > was to use --no-sync as the stat() failure came from files with a size
> > larger than 4GB. That's rather sad as that means sacrifying
> > durability for more usability :(
>
> OK, I went with this text and put it in the Source Code section since it
> applies to several layers of Postgres.
Thanks!
I got the parse error after applying the patch:
release-14.sgml:3562: parser error : Input is not proper UTF-8,
indicate encoding !
Bytes: 0xE9 0x20 0x53 0x61
(Juan Jos Santamara Flecha)
^
Is that a problem with my environment?
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
On Mon, Jun 21, 2021 at 01:57:32PM +0900, Masahiko Sawada wrote:
> > OK, I went with this text and put it in the Source Code section since it
> > applies to several layers of Postgres.
>
> Thanks!
>
> I got the parse error after applying the patch:
>
> release-14.sgml:3562: parser error : Input is not proper UTF-8,
> indicate encoding !
> Bytes: 0xE9 0x20 0x53 0x61
> (Juan Jos Santamara Flecha)
> ^
>
> Is that a problem with my environment?
I don't know, but it builds here and properly shows here:
https://momjian.us/pgsql_docs/release-14.html
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Fri, Jun 18, 2021 at 08:47:21PM -0700, Peter Geoghegan wrote: > On Mon, Jun 14, 2021 at 1:11 PM Bruce Momjian <bruce@momjian.us> wrote: > > FYI, the most recent PG 14 relnote doc build is at: > > > > https://momjian.us/pgsql_docs/release-14.html > > I just pushed a commit that makes the existing vacuum_index_cleanup > reloption and INDEX_CLEANUP VACUUM parameter support disabling the > "Allow vacuum to skip index vacuuming when the number of removable > index entries is insignificant" behavior. This should be mentioned in > the release notes. Agreed. I updated the PG 14 release notes to be current as of today, and adjusted your item --- patch attached. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
Attachment
On Mon, Jun 21, 2021 at 2:07 PM Bruce Momjian <bruce@momjian.us> wrote: > > On Mon, Jun 21, 2021 at 01:57:32PM +0900, Masahiko Sawada wrote: > > > OK, I went with this text and put it in the Source Code section since it > > > applies to several layers of Postgres. > > > > Thanks! > > > > I got the parse error after applying the patch: > > > > release-14.sgml:3562: parser error : Input is not proper UTF-8, > > indicate encoding ! > > Bytes: 0xE9 0x20 0x53 0x61 > > (Juan Jos Santamara Flecha) > > ^ > > > > Is that a problem with my environment? > > I don't know, but it builds here and properly shows here: > > https://momjian.us/pgsql_docs/release-14.html Maybe it's my environmental problem. Thanks anyway! Regards, -- Masahiko Sawada EDB: https://www.enterprisedb.com/
> On Mon, Jun 21, 2021 at 12:50 PM Bruce Momjian <bruce@momjian.us> wrote: >> >> On Tue, Jun 15, 2021 at 12:01:00PM +0900, Michael Paquier wrote: >> > On Tue, Jun 15, 2021 at 11:49:21AM +0900, Masahiko Sawada wrote: >> > > On Tue, Jun 15, 2021 at 10:36 AM Bruce Momjian <bruce@momjian.us> wrote: >> > >> OK, but I need more information on how users will see a difference based >> > >> on this commit: >> > >> > +1. That would be good to have in the release notes. >> > >> > > I think that since with this commit the server on Windows can handle a >> > > file over 4GB, COPY FROM loading data from an over 4GB file and >> > > pg_dump dumping a large table work now. >> > >> > Segment files or WAL files larger than 4GB also gain from that. >> > Anything for which we may finish to do a stat() on benefits from this >> > change if running on Windows. For pg_dump, a workaround in PG <= 13 >> > was to use --no-sync as the stat() failure came from files with a size >> > larger than 4GB. That's rather sad as that means sacrifying >> > durability for more usability :( >> >> OK, I went with this text and put it in the Source Code section since it >> applies to several layers of Postgres. > > Thanks! > > I got the parse error after applying the patch: > > release-14.sgml:3562: parser error : Input is not proper UTF-8, > indicate encoding ! > Bytes: 0xE9 0x20 0x53 0x61 > (Juan Jos Santamara Flecha) > ^ > > Is that a problem with my environment? Me too. I think the problem is, Bruce's patch is encoded in ISO-8859-1, not UTF-8. As far as I know PostgreSQL never encodes *.sgml files in ISO-8859-1. Anyway, attached is the Bruce's patch encoded in UTF-8. This works for me. My guess is, when Bruce attached the file, his MUA automatically changed the file encoding from UTF-8 to ISO-8859-1 (it could happen in many MUA). Also that's the reason why he does not see the problem while compiling the sgml files. In his environment release-14.sgml is encoded in UTF-8, I guess. To prevent the problem next time, it's better to change the mime type of the attached file to Application/Octet-Stream. Best regards, -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp
Attachment
On Mon, May 10, 2021 at 7:03 AM Bruce Momjian <bruce@momjian.us> wrote: > > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > > I need clarification on many items, and the document still needs its > items properly ordered, and markup added. I also expect a lot of > feedback. > > I plan to work on completing this document this coming week in > preparation for beta next week. Very good, thanks. I just noticed that these commits are missing, yet are very important new features: d9d076222f5b94a8 f9900df5f9 c98763bf51bf These are important enough to be major features of PG14. -- Simon Riggs http://www.EnterpriseDB.com/
On Wed, Jun 23, 2021 at 5:50 AM Simon Riggs <simon.riggs@enterprisedb.com> wrote: > I just noticed that these commits are missing, yet are very important > new features: > d9d076222f5b94a8 > f9900df5f9 > c98763bf51bf > > These are important enough to be major features of PG14. I certainly think that they're important enough to be mentioned. -- Peter Geoghegan
On Wed, Jun 23, 2021 at 07:45:53AM -0700, Peter Geoghegan wrote: > On Wed, Jun 23, 2021 at 5:50 AM Simon Riggs > <simon.riggs@enterprisedb.com> wrote: > > I just noticed that these commits are missing, yet are very important > > new features: > > d9d076222f5b94a8 > > f9900df5f9 > > c98763bf51bf > > > > These are important enough to be major features of PG14. > > I certainly think that they're important enough to be mentioned. OK, here is a doc patch to add a mention of this. I originally thought this was an optimization that wouldn't be of general interest. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
Attachment
> Require custom server variable names to use only character which are valid for unquoted SQL identifiers (Tom Lane)
characters plural (since 69a58bfe4)
> This is similar to how Unicode can be specified in literal string.
literal strings
> Add executor method to cache results from the inner-side of nested loop joins (David Rowley)
> This is useful if only a small percentage of rows is checked on the inner side.
I think this should mention the GUC, whether we leave it enabled by default (in
which case people may want to disable it) or disable by default (in which case
people may want to enable it).
> The postgres_fdw supports these type of scans if async_capable is set.
this type
remove "The" ?
> Prevent the containment operators (<@ and @>) for intarray from using GiST indexes (Tom Lane)
> Remove deprecated containment operators @ and ~ for built-in geometric data types and contrib modules cube, hstore,
intarray,and seg (Justin Pryzby)
> For example, disregard ^ in its expansion in \1 in (^\d+).*\1.
> Add point operators <<| and |>> to be strictly above/below geometry (Emre Hasegeli)
> Previously >^ and <^ were marked as performing this test, but non-point geometric operators used these operators for
non-strictcomparisons, leading to confusion. The old operators still exist but will be eventually removed. ACCURATE?
Should these have markup added?
> Certain discarded tokens, like underscore, caused the output of these functions to produce incorrect tsquery output,
e.g.,both websearch_to_tsquery('"pg_class pg"') and to_tsquery('pg_class <-> pg') used to output '( pg & class ) <->
pg',but now both output 'pg <-> class <-> pg'.
> Previously, quoted text that contained multiple adjacent discarded tokens were treated as multiple tokens, causing
incorrecttsquery output, e.g., websearch_to_tsquery('"aaa: bbb"') used to output 'aaa <2> bbb', but now outputs 'aaa
<->bbb'.
Missing markup?
> This is controlled by server variable ssl_crl_dir and libpq connection option sslcrldir. Previously only CRL files
couldbe specified.
> Allow pgstattuple_approx() to report on TOAST tables (Peter Eisentraut)
> Add pg_stat_statements_info system view to show pg_stat_statements activity (Katsuragi Yuta, Yuki Seino, Naoki
Nakamichi)
> Add postgres_fdw function postgres_fdw_get_connections() to report open foreign server connections (Bharath
Rupireddy)
These should have additional hyperlinks
> Add primary keys, unique constraints, and foreign keys to system catalogs (Peter Eisentraut)
Should mention and link to pg_get_catalog_foreign_keys()
> Pass doubled quote marks in Chapter 36 SQL command strings literally (Tom Lane)
"Chapter 36" looks funny?
See also: 4f7d1c309
>Previously window frame clauses like 'inf' PRECEDING AND 'inf' FOLLOWING returned incorrect results.
>Negative values produced undesirable results.
>Previously such cases returned 1.
>This previously was allowed but produced incorrect results.
>This could be accomplished previously using existing syntax.
All these details could be omitted.
>Only the target table can be referenced.
Could be omitted or folded into the preceding line.
> This was already disabled by default in previous Postgres releases, and most modern OpenSSL and TLS versions no
longersupport it.
> This was last used as the default in Postgres 7.3 (year 2002).
> By default, Postgres opens and fsyncs every data file at the start of crash recovery. This new setting,
recovery_init_sync_method=syncfs,instead syncs each filesystem used by the database cluster. This allows for faster
recoveryon systems with many database files.
> The new syntax is SUBSTRING(text SIMILAR pattern ESCAPE escapechar). The previous standard syntax was SUBSTRING(text
FROMpattern FOR escapechar), and is still supported by Postgres.
These should all say <productname>PostgreSQL</productname>
> Allow psql's \df and \do commands to specify function and operator argument types (Greg Sabino Mullane, Tom Lane)
> Add an access method column to psql's \d[i|m|t]+ output (Georgios Kokolatos)
> Allow psql's \dt and \di to show TOAST tables and their indexes (Justin Pryzby)
> Add psql command \dX to list extended statistics objects (Tatsuro Yamada)
> Fix psql's \dT to understand array syntax and backend grammar aliases, like "int" for "integer" (Greg Sabino Mullane,
TomLane)
> When editing the previous query or a file with psql's \e, or using \ef and \ev, ignore the contents if the editor
exitswithout saving (Laurenz Albe)
All these slash commands should be <literal>
> Stop pg_upgrade from creating analyze_new_cluster script (Michael Paquier)
It's called analyze_new_cluster.sh (except on window), and it's Magnus' patch.
> EXTRACT(date) now throws an error for units that are not part of the date data type.
"Date data" always seems hard to read.
Could you add markup for "<type>date</type>" ?
Or say: of type "date".
> EXTRACT(date) now throws an error for units that are not part of the date data type.
Could say "if requested to extract units that are not part of the "date" type.
--
Justin
On Fri, Jun 25, 2021 at 06:04:56PM -0500, Justin Pryzby wrote:
> > Require custom server variable names to use only character which are valid for unquoted SQL identifiers (Tom Lane)
>
> characters plural (since 69a58bfe4)
Fixed.
> > This is similar to how Unicode can be specified in literal string.
>
> literal strings
Fixed.
> > Add executor method to cache results from the inner-side of nested loop joins (David Rowley)
> > This is useful if only a small percentage of rows is checked on the inner side.
>
> I think this should mention the GUC, whether we leave it enabled by default (in
> which case people may want to disable it) or disable by default (in which case
> people may want to enable it).
OK, I changed it to:
This is useful if only a small percentage of rows is checked on
the inner side and is controlled by <xref
linkend="guc-enable-resultcache"/>.
> > The postgres_fdw supports these type of scans if async_capable is set.
> this type
> remove "The" ?
New text is:
<link
linkend="postgres-fdw"><application>postgres_fdw</application></link>
supports these type of scans if <literal>async_capable</literal>
I kept "these types" because the paragraph above says:
Allow a query referencing multiple <link
linkend="sql-createforeigntable">foreign tables</link> to perform
foreign table scans in parallel (Robert Haas, Kyotaro Horiguchi,
Thomas Munro, Etsuro Fujita)
so we are talking about scans in parallel, so I think it is plural. Wrong?
> > Prevent the containment operators (<@ and @>) for intarray from using GiST indexes (Tom Lane)
I show this markup as:
Prevent the containment operators (<@ and @>) for <xref
linkend="intarray"/> from using GiST indexes (Tom Lane)
What markup is missing?
> > Remove deprecated containment operators @ and ~ for built-in geometric data types and contrib modules cube, hstore,
intarray,and seg (Justin Pryzby)
Same. what is missing?
Remove deprecated containment operators @ and ~ for built-in
<link linkend="functions-geometry">geometric data types</link> and
contrib modules <xref linkend="cube"/>, <xref linkend="hstore"/>,
<xref linkend="intarray"/>, and <xref linkend="seg"/> (Justin Pryzby)
I could link to our "Geometric Functions and Operators" section, but my
understanding is those are operators for our built-in types, not for
extensions, which is what we are talking about here.
> > For example, disregard ^ in its expansion in \1 in (^\d+).*\1.
Fixed:
Improve handling of regular expression <link
linkend="posix-escape-sequences">back-references</link> (Tom Lane)
I didn't know we had a good section for that, but I see it now.
> > Add point operators <<| and |>> to be strictly above/below geometry (Emre Hasegeli)
> > Previously >^ and <^ were marked as performing this test, but non-point geometric operators used these operators
fornon-strict comparisons, leading to confusion. The old operators still exist but will be eventually removed.
ACCURATE?
I see markup on this:
Add <link linkend="functions-geometry">point operators</link>
<<| and |>> to be strictly above/below geometry
(Emre Hasegeli)
> Should these have markup added?
>
> > Certain discarded tokens, like underscore, caused the output of these functions to produce incorrect tsquery
output,e.g., both websearch_to_tsquery('"pg_class pg"') and to_tsquery('pg_class <-> pg') used to output '( pg & class
)<-> pg', but now both output 'pg <-> class <-> pg'.
> > Previously, quoted text that contained multiple adjacent discarded tokens were treated as multiple tokens, causing
incorrecttsquery output, e.g., websearch_to_tsquery('"aaa: bbb"') used to output 'aaa <2> bbb', but now outputs 'aaa
<->bbb'.
>
> Missing markup?
I see markup on the main text of this item:
Fix <link
linkend="functions-textsearch"><function>to_tsquery()</function></link>
and <function>websearch_to_tsquery()</function> to properly parse
query text containing discarded tokens (Alexander Korotkov)
> > This is controlled by server variable ssl_crl_dir and libpq connection option sslcrldir. Previously only CRL files
couldbe specified.
OK, added link to sslcrldir.
> > Allow pgstattuple_approx() to report on TOAST tables (Peter Eisentraut)
Fixed:
Allow <link
linkend="pgstattuple"><function>pgstattuple_approx()</function></link>
to report on <acronym>TOAST</acronym> tables (Peter Eisentraut)
> > Add pg_stat_statements_info system view to show pg_stat_statements activity (Katsuragi Yuta, Yuki Seino, Naoki
Nakamichi)
The section heading already has a link so I don't add any for individual
items.
> > Add postgres_fdw function postgres_fdw_get_connections() to report open foreign server connections (Bharath
Rupireddy)
Same, this is already in the section with a link in the heading.
> > Add primary keys, unique constraints, and foreign keys to system catalogs (Peter Eisentraut)
>
> Should mention and link to pg_get_catalog_foreign_keys()
Uh, why? I don't see the release notes as a place to explain how to use
Postgres features.
> > Pass doubled quote marks in Chapter 36 SQL command strings literally (Tom Lane)
>
> "Chapter 36" looks funny?
> See also: 4f7d1c309
Yes, fixed.
> >Previously window frame clauses like 'inf' PRECEDING AND 'inf' FOLLOWING returned incorrect results.
> >Negative values produced undesirable results.
> >Previously such cases returned 1.
> >This previously was allowed but produced incorrect results.
> >This could be accomplished previously using existing syntax.
>
> All these details could be omitted.
I think explaining the previous behavior helps people understand the new
behavior.
> >Only the target table can be referenced.
>
> Could be omitted or folded into the preceding line.
I only use a single sentence for the main text.
> > This was already disabled by default in previous Postgres releases, and most modern OpenSSL and TLS versions no
longersupport it.
> > This was last used as the default in Postgres 7.3 (year 2002).
> > By default, Postgres opens and fsyncs every data file at the start of crash recovery. This new setting,
recovery_init_sync_method=syncfs,instead syncs each filesystem used by the database cluster. This allows for faster
recoveryon systems with many database files.
> > The new syntax is SUBSTRING(text SIMILAR pattern ESCAPE escapechar). The previous standard syntax was
SUBSTRING(textFROM pattern FOR escapechar), and is still supported by Postgres.
>
> These should all say <productname>PostgreSQL</productname>
OK, fixed.
> > Allow psql's \df and \do commands to specify function and operator argument types (Greg Sabino Mullane, Tom Lane)
> > Add an access method column to psql's \d[i|m|t]+ output (Georgios Kokolatos)
I don't think this one is actually literal.
> > Allow psql's \dt and \di to show TOAST tables and their indexes (Justin Pryzby)
> > Add psql command \dX to list extended statistics objects (Tatsuro Yamada)
> > Fix psql's \dT to understand array syntax and backend grammar aliases, like "int" for "integer" (Greg Sabino
Mullane,Tom Lane)
> > When editing the previous query or a file with psql's \e, or using \ef and \ev, ignore the contents if the editor
exitswithout saving (Laurenz Albe)
>
> All these slash commands should be <literal>
Yes, fixed except for the one mentioned above.
> > Stop pg_upgrade from creating analyze_new_cluster script (Michael Paquier)
>
> It's called analyze_new_cluster.sh (except on window), and it's Magnus' patch.
Ah, yes, Magnus, fixed. I don't see the value in adding .sh since it is
not always accurate, but I could add .sh/.bat, but that seems confusing.
> > EXTRACT(date) now throws an error for units that are not part of the date data type.
>
> "Date data" always seems hard to read.
> Could you add markup for "<type>date</type>" ?
Done.
Thank you for all the help on this. Patch attached. Since we just
branched for PG 15, our official developer doc build no longer has the
PG 14 release notes, so I changed my local doc tree to build PG 14 until
PG14 is released in a few months; my URL is:
https://momjian.us/pgsql_docs/release-14.html
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
Attachment
On Mon, Jun 28, 2021 at 09:01:40PM -0400, Bruce Momjian wrote: > On Fri, Jun 25, 2021 at 06:04:56PM -0500, Justin Pryzby wrote: > > > The postgres_fdw supports these type of scans if async_capable is set. > > this type > > remove "The" ? > > New text is: > > <link > linkend="postgres-fdw"><application>postgres_fdw</application></link> > supports these type of scans if <literal>async_capable</literal> > > I kept "these types" because the paragraph above says: > > Allow a query referencing multiple <link > linkend="sql-createforeigntable">foreign tables</link> to perform > foreign table scans in parallel (Robert Haas, Kyotaro Horiguchi, > Thomas Munro, Etsuro Fujita) > > so we are talking about scans in parallel, so I think it is plural. Wrong? I think the "type" of scan being referenced is a "parallel" type, right ? So there's only one type, but multiple scans. So I think it should say "this type" of scan, but it seems like it's not only easier but generally better to say | postgres_fdw supports parallel scans if async_capable >> Prevent the containment operators (<@ and @>) for intarray from using GiST indexes (Tom Lane) >> Remove deprecated containment operators @ and ~ for built-in geometric data types and contrib modules cube, hstore, intarray,and seg (Justin Pryzby) >> For example, disregard ^ in its expansion in \1 in (^\d+).*\1. >> Add point operators <<| and |>> to be strictly above/below geometry (Emre Hasegeli) >> Previously >^ and <^ were marked as performing this test, but non-point geometric operators used these operators for non-strictcomparisons, leading to confusion. The old operators still exist but will be eventually removed. > What markup is missing? I mean markup for the operators, like <literal><@</literal> > > > Add primary keys, unique constraints, and foreign keys to system catalogs (Peter Eisentraut) > > > Should mention and link to pg_get_catalog_foreign_keys() > > Uh, why? I don't see the release notes as a place to explain how to use > Postgres features. Because the normal way to show foreign keys (\d) doesn't show them - the references are shown by the function. Thanks, -- Justin
On Fri, Jun 25, 2021 at 2:56 AM Bruce Momjian <bruce@momjian.us> wrote:
>
> On Wed, Jun 23, 2021 at 07:45:53AM -0700, Peter Geoghegan wrote:
> > On Wed, Jun 23, 2021 at 5:50 AM Simon Riggs
> > <simon.riggs@enterprisedb.com> wrote:
> > > I just noticed that these commits are missing, yet are very important
> > > new features:
> > > d9d076222f5b94a8
> > > f9900df5f9
> > > c98763bf51bf
> > >
> > > These are important enough to be major features of PG14.
> >
> > I certainly think that they're important enough to be mentioned.
>
> OK, here is a doc patch to add a mention of this. I originally thought
> this was an optimization that wouldn't be of general interest.
Perhaps we should also add this text from the commit message to ensure
the importance is understood:
"This is extremely useful in cases where CIC/RC can run for a very long
time, because that used to be a significant headache for concurrent
vacuuming of other tables."
Proposed edits:
* "during certain index operations" -> "while concurrent index
operations run on other tables"
* spell Alvaro's name correctly
* "row expiration" is a term not currently used in PG docs, so we
should probably look for something else.
There are 2 important features here, so the 2nd feature is worth
mentioning also:
Avoid spurious waits in concurrent indexing
Previously, multiple concurrent index operations could deadlock or
cause long waits.
Waits are avoided except for indexes with expressions, or WHERE predicates.
--
Simon Riggs http://www.EnterpriseDB.com/
On Tue, Jun 29, 2021 at 07:36:47PM +0100, Simon Riggs wrote: > Perhaps we should also add this text from the commit message to ensure > the importance is understood: > "This is extremely useful in cases where CIC/RC can run for a very long > time, because that used to be a significant headache for concurrent > vacuuming of other tables." Uh, hopefully this idea is captured below. > Proposed edits: > > * "during certain index operations" -> "while concurrent index > operations run on other tables" Uh, the problem here is that it isn't clear if we are talking about concurrent index operations, "concurrently"-enabled index options, or "concurrently"-enabled concurrent index operations. I went with "minimal-locking index operations". > * spell Alvaro's name correctly Uh, this is the second time Mutt's default to using iso-8859-1 for non-ASCII attachments has caused confusion. I found that using 'set send_charset="us-ascii:utf-8"' in Mutt fixed this, so the attachment should now be fine. > * "row expiration" is a term not currently used in PG docs, so we > should probably look for something else. Yeah, I changed that to "removing dead rows" which seems to be our standard text. > There are 2 important features here, so the 2nd feature is worth > mentioning also: > > Avoid spurious waits in concurrent indexing > > Previously, multiple concurrent index operations could deadlock or > cause long waits. > Waits are avoided except for indexes with expressions, or WHERE predicates. OK, I added text to the bottom to try and capture that; new patch attached, now with UTF8 encoding. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
Attachment
On Wed, Jun 30, 2021 at 11:20 PM Bruce Momjian <bruce@momjian.us> wrote: > > On Tue, Jun 29, 2021 at 07:36:47PM +0100, Simon Riggs wrote: > > Perhaps we should also add this text from the commit message to ensure > > the importance is understood: > > "This is extremely useful in cases where CIC/RC can run for a very long > > time, because that used to be a significant headache for concurrent > > vacuuming of other tables." > > Uh, hopefully this idea is captured below. > > > Proposed edits: > > > > * "during certain index operations" -> "while concurrent index > > operations run on other tables" > > Uh, the problem here is that it isn't clear if we are talking about > concurrent index operations, "concurrently"-enabled index options, or > "concurrently"-enabled concurrent index operations. I went with > "minimal-locking index operations". > > > * spell Alvaro's name correctly > > Uh, this is the second time Mutt's default to using iso-8859-1 for > non-ASCII attachments has caused confusion. I found that using 'set > send_charset="us-ascii:utf-8"' in Mutt fixed this, so the attachment > should now be fine. > > > * "row expiration" is a term not currently used in PG docs, so we > > should probably look for something else. > > Yeah, I changed that to "removing dead rows" which seems to be our > standard text. What you have now looks great for this feature, thanks. > > There are 2 important features here, so the 2nd feature is worth > > mentioning also: > > > > Avoid spurious waits in concurrent indexing > > > > Previously, multiple concurrent index operations could deadlock or > > cause long waits. > > Waits are avoided except for indexes with expressions, or WHERE predicates. > > OK, I added text to the bottom to try and capture that; new patch > attached, now with UTF8 encoding. The text from "This also avoids..." tries to explain this, but they are two separate features, each important in its own right. So regrettably, this part doesn't capture it, for me. -- Simon Riggs http://www.EnterpriseDB.com/
On Thu, Jul 1, 2021 at 03:13:30PM +0100, Simon Riggs wrote: > On Wed, Jun 30, 2021 at 11:20 PM Bruce Momjian <bruce@momjian.us> wrote: > > > * "row expiration" is a term not currently used in PG docs, so we > > > should probably look for something else. > > > > Yeah, I changed that to "removing dead rows" which seems to be our > > standard text. > > What you have now looks great for this feature, thanks. Good. > > > There are 2 important features here, so the 2nd feature is worth > > > mentioning also: > > > > > > Avoid spurious waits in concurrent indexing > > > > > > Previously, multiple concurrent index operations could deadlock or > > > cause long waits. > > > Waits are avoided except for indexes with expressions, or WHERE predicates. > > > > OK, I added text to the bottom to try and capture that; new patch > > attached, now with UTF8 encoding. > > The text from "This also avoids..." tries to explain this, but they > are two separate features, each important in its own right. > > So regrettably, this part doesn't capture it, for me. I see what you mean. This is in the VACUUM section, and this feature, though from the same commits, has nothing to do with vacuum. Attached is an updated patch. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
Attachment
On Mon, Jun 28, 2021 at 09:25:47PM -0500, Justin Pryzby wrote: > On Mon, Jun 28, 2021 at 09:01:40PM -0400, Bruce Momjian wrote: > > so we are talking about scans in parallel, so I think it is plural. Wrong? > > I think the "type" of scan being referenced is a "parallel" type, right ? > So there's only one type, but multiple scans. > So I think it should say "this type" of scan, but it seems like it's not only > easier but generally better to say > > | postgres_fdw supports parallel scans if async_capable > > >> Prevent the containment operators (<@ and @>) for intarray from using GiST indexes (Tom Lane) > >> Remove deprecated containment operators @ and ~ for built-in geometric data types and contrib modules cube, hstore,intarray, and seg (Justin Pryzby) > >> For example, disregard ^ in its expansion in \1 in (^\d+).*\1. > >> Add point operators <<| and |>> to be strictly above/below geometry (Emre Hasegeli) > >> Previously >^ and <^ were marked as performing this test, but non-point geometric operators used these operators fornon-strict comparisons, leading to confusion. The old operators still exist but will be eventually removed. > > > What markup is missing? > > I mean markup for the operators, like <literal><@</literal> > > > Uh, why? I don't see the release notes as a place to explain how to use > > Postgres features. > > Because the normal way to show foreign keys (\d) doesn't show them - the > references are shown by the function. OK, agreed. Here is an updated applied patch. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
Attachment
On Fri, Jul 2, 2021 at 12:50 AM Bruce Momjian <bruce@momjian.us> wrote: > > On Thu, Jul 1, 2021 at 03:13:30PM +0100, Simon Riggs wrote: > > On Wed, Jun 30, 2021 at 11:20 PM Bruce Momjian <bruce@momjian.us> wrote: > > > > * "row expiration" is a term not currently used in PG docs, so we > > > > should probably look for something else. > > > > > > Yeah, I changed that to "removing dead rows" which seems to be our > > > standard text. > > > > What you have now looks great for this feature, thanks. > > Good. > > > > > There are 2 important features here, so the 2nd feature is worth > > > > mentioning also: > > > > > > > > Avoid spurious waits in concurrent indexing > > > > > > > > Previously, multiple concurrent index operations could deadlock or > > > > cause long waits. > > > > Waits are avoided except for indexes with expressions, or WHERE predicates. > > > > > > OK, I added text to the bottom to try and capture that; new patch > > > attached, now with UTF8 encoding. > > > > The text from "This also avoids..." tries to explain this, but they > > are two separate features, each important in its own right. > > > > So regrettably, this part doesn't capture it, for me. > > I see what you mean. This is in the VACUUM section, and this feature, > though from the same commits, has nothing to do with vacuum. Attached > is an updated patch. Perfect, many thanks. -- Simon Riggs http://www.EnterpriseDB.com/
I think the release notes for the autovacuum item (which was first reverted and then partially un-reverted) should say something like "Partitioned tables are now included in pg_stat_all_tables": | e1efc5b465 Keep stats up to date for partitioned tables Remove some internal question/marks: ACCURATE? REMOVE? ALTER TABLE ... ALTER COLUMN ... TYPE RESETS STASTISTICS? (was never intended to have markup added?) Also, I'm not sure if this one should be included. 9a4c0e36fb Dump ALTER TABLE ... ATTACH PARTITION as a separate ArchiveEntry. See: https://www.postgresql.org/message-id/flat/20210830154249.GX26465@telsasoft.com#9ad6fd4c36e13deea1c5f92f5114320e What about this ? ce6b662aae psql: Fix name quoting on extended statistics -- Justin
Justin Pryzby <pryzby@telsasoft.com> writes: > I think the release notes for the autovacuum item (which was first reverted and > then partially un-reverted) should say something like "Partitioned tables are > now included in pg_stat_all_tables": > | e1efc5b465 Keep stats up to date for partitioned tables Hmm. If I'm reading the commit message properly, the actual change there is not that, but that analyze count and last analyze time are now tracked correctly for partitioned tables. Might be worth mentioning, not sure. > Remove some internal question/marks: > ACCURATE? > REMOVE? > ALTER TABLE ... ALTER COLUMN ... TYPE RESETS STASTISTICS? (was never intended to have markup added?) Did that in the copy-editing I just pushed. > Also, I'm not sure if this one should be included. > 9a4c0e36fb Dump ALTER TABLE ... ATTACH PARTITION as a separate ArchiveEntry. > See: https://www.postgresql.org/message-id/flat/20210830154249.GX26465@telsasoft.com#9ad6fd4c36e13deea1c5f92f5114320e Probably not worth listing --- the amount of explanation needed seems to outweigh the probability of users caring. > What about this ? > ce6b662aae psql: Fix name quoting on extended statistics Seems way too minor to bother with here. regards, tom lane
On Sat, Sep 18, 2021 at 05:15:39PM -0400, Tom Lane wrote: > Justin Pryzby <pryzby@telsasoft.com> writes: > > I think the release notes for the autovacuum item (which was first reverted and > > then partially un-reverted) should say something like "Partitioned tables are > > now included in pg_stat_all_tables": > > | e1efc5b465 Keep stats up to date for partitioned tables > > Hmm. If I'm reading the commit message properly, the actual change there > is not that, but that analyze count and last analyze time are now tracked > correctly for partitioned tables. Might be worth mentioning, not sure. The reverted patch to autoanalyze included partitioned tables in pg_stat_all_tables, and the revert specifically avoided changing that (to avoid a catbump). But last_analyzed and analyze_count were always shown as "0". So the e1 commit addresses that by tracking that information and showing correct value instead of always 0. The relevant portion starts here: https://www.postgresql.org/message-id/flat/202108161700.d4eh6a7n2lki%40alvherre.pgsql#b2e426eb19dbbddee0adf9bb1bcbbcf1 I suggest that this *should* be included in the release notes, since I specifically requested that partitioned tables be included in 2018. > In 20200418050815(dot)GE26953(at)telsasoft(dot)com I wrote: > |This patch includes partitioned tables in pg_stat_*_tables, which is great; I > |complained awhile ago that they were missing [0]. It might be useful if that > |part was split out into a separate 0001 patch (?). > | [0] https://www.postgresql.org/message-id/20180601221428.GU5164%40telsasoft.com Also, I've patched my analyze script to use that field (same as for nonpartitioned tables) rather than needing to do a subquery involving max(last_analyzed) of the partitions. Since it's still needed to manually analyze parent tables. -- Justin
On Mon, 10 May 2021 02:03:08 -0400 Bruce Momjian <bruce@momjian.us> wrote: > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > > I need clarification on many items, and the document still needs its > items properly ordered, and markup added. I also expect a lot of > feedback. I found that there isn't an item that referring the following commit: commit 547f04e7348b6ed992bd4a197d39661fe7c25097 Author: Thomas Munro <tmunro@postgresql.org> Date: Wed Mar 10 16:09:50 2021 +1300 pgbench: Improve time logic. Instead of instr_time (struct timespec) and the INSTR_XXX macros, introduce pg_time_usec_t and use integer arithmetic. Don't include the connection time in TPS unless using -C mode, but report it separately. This commit changed the output results that are visible for users, that is, tps with "including connection establishing" and "excluding ..." are no longer reported, so I wonder we should let users know it. Another user-visible change on pgbench that is not listed in the release notes is the support for pipeline mode. There is a mention about libpq supporting pipeline, so I think we can also mention that of pgbench. There are other minor improvement on pgbench, but pgbench is just a benchnark tool and its changes do not directly affect to user application, so maybe we don't have to describe all in the release notes. Regards, Yugo Nagata -- Yugo NAGATA <nagata@sraoss.co.jp>