Thread: First draft of the PG 15 release notes
I have completed the first draft of the PG 15 release notes and you can
see the results here:
https://momjian.us/pgsql_docs/release-15.html
The feature count is similar to recent major releases:
release-10 195
release-11 185
release-12 198
release-13 183
release-14 229
--> release-15 186
I assume there will be major adjustments in the next few weeks based on
feedback.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
I'm guessing this should be "trailing", not training? > Prevent numeric literals from having non-numeric training characters (Peter Eisentraut)
On Tue, May 10, 2022 at 04:58:50PM +0100, Geoff Winkless wrote: > I'm guessing this should be "trailing", not training? > > > Prevent numeric literals from having non-numeric training characters (Peter Eisentraut) Thanks, fixed. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Op 10-05-2022 om 17:44 schreef Bruce Momjian: > I have completed the first draft of the PG 15 release notes and you can > see the results here: > > https://momjian.us/pgsql_docs/release-15.html typos: 'accept empty array' should be 'to accept empty array' 'super users' should be 'superusers' (several times) 'The new options causes the column names' 'The new options cause the column names' 'were always effected.' 'were always affected.' (I think...) 'Previous the actual schema' 'Previously the actual schema' 'inforcement' should be 'enforcement' (surely?) 'server slide' should be 'server side' 'Add extensions to define their own' should be 'Allow extensions to define their own' And one strangely unfinished sentence: 'They also can only be' Erik Rijkers
Thanks for writting this. Comments from my first read: | This mode could cause server startup failure if the database server stopped abruptly while in this mode. This sentence both begins and ends with "this mode". | This allows query hash operations to use double the amount of work_mem memory as other operations. Should it say "node" ? | Allow tsvector_delete_arr() and tsvector_setweight_by_filter() accept empty array elements (Jean-Christophe Arnu) TO accept I don't think this should be in the "compatibility" section? | This accepts numeric formats like ".1" and "1.", and disallow trailing junk after numeric literals, like "1.type()". disallows with an ess | This will cause setseed() followed by random() to return a different value on older servers. *than* older servers | The Postgres default has always been to treat NULL indexed values as distinct, but this can now be changed by creatingconstraints and indexes using UNIQUE NULLS NOT DISTINCT. should not say default, since it wasn't previously configurable. "The previous behavior was ..." | Have extended statistics track statistics for a table's children separately (Tomas Vondra, Justin Pryzby) | Regular statistics already tracked child and non-child statistics separately. "separately" seems vague. Currently in v10-v13, extended stats are collected for partitioned tables, and collected for the "SELECT FROM ONLY"/parent case for inheritance parents. This changes to also collect stats for the "SELECT FROM tbl*" case. See also: 20220204214103.GT23027@telsasoft.com. | Allow members of the pg_checkpointer predefined role to run the CHECKPOINT command (Jeff Davis) | Previously these views could only be run by super users. | Allow members of the pg_read_all_stats predefined role to access the views pg_backend_memory_contexts and pg_shmem_allocations(Bharath Rupireddy) | Previously these views could only be run by super users. checkpoint is not a view (and views cannot be run) | Previously runtime-computed values data_checksums, wal_segment_size, and data_directory_mode would report values that wouldnot be accurate on the running server. They also can only be be what ? | Add server variable allow_in_place_tablespaces for tablespace testing (Thomas Munro) This is a developer setting, so doesn't need to be mentioned ? | Add function pg_settings_get_flags() to get the flags of server-side variables (Justin Pryzby) IMO this is the same, but I think maybe Michael things about it differently... | Allow WAL full page writes to use LZ4 and ZSTD compression (Andrey Borodin, Justin Pryzby) | Add support for LZ4 and ZSTD compression of server-side base backups (Jeevan Ladhe, Robert Haas) | Allow pg_basebackup to decompress LZ4 and ZSTD compressed server-side base backups, and LZ4 and ZSTD compress output files(Dipesh Pandit, Jeevan Ladhe) | Add configure option --with-zstd to enable ZSTD build (Jeevan Ladhe, Robert Haas, Michael Paquier) Maybe these should say "Zstandard" ? See 586955dddecc95e0003262a3954ae83b68ce0372. | The new options causes the column names to be output, and optionally verified on input. option | Previous the actual schema name was used. Previously | When EXPLAIN references the temporary object schema, refer to it as "pg_temp" (Amul Sul) | When specifying fractional interval values in units greater than months, round to the nearest month (Bruce Momjian) | Limit support of psql to servers running PostgreSQL 9.2 and later (Tom Lane) | Limit support of pg_dump and pg_dumpall to servers running PostgreSQL 9.2 and later (Tom Lane) | Limit support of pg_upgrade to old servers running PostgreSQL 9.2 and later (Tom Lane) | Remove server support for old BASE_BACKUP command syntax and base backup protocol (Robert Haas) Do these need to be in the "compatibility" section ? | Fix inforcement of PL/pgSQL variable CONSTANT markings (Tom Lane) enforcement | Allow IP address matching against a server's certificate Subject Alternative Name (Jacob Champion) Should say "server certificate's" ? | Allow libpq's SSL private to be owned by the root user (David Steele) private *key* | Have psql output all output if multiple queries are passed to the server at once (Fabien Coelho) all *results* ? | This can be disabled setting SHOW_ALL_RESULTS. disabled *by* setting | Allow pg_basebackup's --compress option to control the compression method (Michael Paquier, Robert Haas) Should probably say "compression method and options" | Allow pg_basebackup to decompress LZ4 and ZSTD compressed server-side base backups, and LZ4 and ZSTD compress output files(Dipesh Pandit, Jeevan Ladhe) | Allow pg_basebackup to compress on the server slide and decompress on the client side before storage (Dipesh Pandit) Maybe these should be combined into one entry ? | Add the LZ4 compression method to pg_receivewal (Georgios Kokolatos) | This is enabled via --compression-method=lz4 and requires binaries to be built using --with-lz4. | Redesign pg_receivewal's compression options (Georgios Kokolatos) | The new --compression-method option controls the type of compression, rather than just relying on --compress. It's --compress since 042a923ad. | Previously, pg_receivewal would start based on the WAL file stored in the local archive directory, or at the sending server'scurrent WAL flush location. With this change, if the sending server is running Postgres 15 or later, the local archivedirectory is empty, and a replication slot is specified, the replication slots restart point will be used. slot's restart point (with a >>'<<) | Add dump/restore option --no-table-access-method to force restore to use only the default table access method (Justin Pryzby) remove "only" ? | This is for portability in restoring from systems using non-default table access methods. I would remove part about "portability". The use-case I see for this is restoring something to a different table AM (not just heapam), in the same way as is possible for tablespaces: PGOPTIONS='-c default-table-access-method=foo' pg_restore --no-table-am ./dump | Previously only the first invalid connection setting database was reported. "only the first database with an invalid connection setting..." | Add new protocol message TARGET to specific a new COPY method to be for base backups (Robert Haas) specify | Automatically export server variables using PGDLLIMPORT on Windows (Robert Haas) I don't think it's "automatic" ? | Allow informational escape sequences to be used in postgres_fdw's application name (Hayato Kuroda, Fujii Masao) I don't think this should be a separate entry | This is enabled with the "parallel_commit" postgres_fdw option. It's an option to the SQL "SERVER" command.
On Tue, May 10, 2022 at 08:07:45PM +0200, Erik Rijkers wrote: > And one strangely unfinished sentence: > 'They also can only be' I suspect this meant to highlight that "postgres -C" with runtime-computed GUCs doesn't work if the server is running. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com
Agreed on all of these, and URL contents updated. Thanks. --------------------------------------------------------------------------- On Tue, May 10, 2022 at 08:07:45PM +0200, Erik Rijkers wrote: > Op 10-05-2022 om 17:44 schreef Bruce Momjian: > > I have completed the first draft of the PG 15 release notes and you can > > see the results here: > > > > https://momjian.us/pgsql_docs/release-15.html > > typos: > > 'accept empty array' should be > 'to accept empty array' > > 'super users' should be > 'superusers' > (several times) > > 'The new options causes the column names' > 'The new options cause the column names' > > 'were always effected.' > 'were always affected.' > (I think...) > > 'Previous the actual schema' > 'Previously the actual schema' > > 'inforcement' should be > 'enforcement' > (surely?) > > 'server slide' should be > 'server side' > > 'Add extensions to define their own' should be > 'Allow extensions to define their own' > > > And one strangely unfinished sentence: > 'They also can only be' > > > Erik Rijkers > > -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On 5/10/22 11:44 AM, Bruce Momjian wrote: > I have completed the first draft of the PG 15 release notes and you can > see the results here: > > https://momjian.us/pgsql_docs/release-15.html Thanks for pulling this together. + Allow logical replication to transfer sequence changes I believe this was reverted in 2c7ea57e5, unless some other parts of this work made it in. Thanks, Jonathan
Attachment
On Tue, May 10, 2022 at 01:09:35PM -0500, Justin Pryzby wrote:
> Thanks for writting this.
>
> Comments from my first read:
>
> | This mode could cause server startup failure if the database server stopped abruptly while in this mode.
>
> This sentence both begins and ends with "this mode".
Agreed, I rewrote this.
> | This allows query hash operations to use double the amount of work_mem memory as other operations.
>
> Should it say "node" ?
Uh, I think users think of things like operations, e.g. sort operation
vs sort node.
> | Allow tsvector_delete_arr() and tsvector_setweight_by_filter() accept empty array elements (Jean-Christophe Arnu)
>
> TO accept
> I don't think this should be in the "compatibility" section?
Yes, moved to Function.
> | This accepts numeric formats like ".1" and "1.", and disallow trailing junk after numeric literals, like
"1.type()".
>
> disallows with an ess
Fixed.
> | This will cause setseed() followed by random() to return a different value on older servers.
>
> *than* older servers
Fixed.
> | The Postgres default has always been to treat NULL indexed values as distinct, but this can now be changed by
creatingconstraints and indexes using UNIQUE NULLS NOT DISTINCT.
>
> should not say default, since it wasn't previously configurable.
> "The previous behavior was ..."
Agreed, reworded.
> | Have extended statistics track statistics for a table's children separately (Tomas Vondra, Justin Pryzby)
> | Regular statistics already tracked child and non-child statistics separately.
>
> "separately" seems vague. Currently in v10-v13, extended stats are collected
> for partitioned tables, and collected for the "SELECT FROM ONLY"/parent case
> for inheritance parents. This changes to also collect stats for the "SELECT
> FROM tbl*" case. See also: 20220204214103.GT23027@telsasoft.com.
Agreed, reworded. Can you check you like my new wording?
> | Allow members of the pg_checkpointer predefined role to run the CHECKPOINT command (Jeff Davis)
> | Previously these views could only be run by super users.
> | Allow members of the pg_read_all_stats predefined role to access the views pg_backend_memory_contexts and
pg_shmem_allocations(Bharath Rupireddy)
> | Previously these views could only be run by super users.
>
> checkpoint is not a view (and views cannot be run)
Fixed, was copy/paste error.
> | Previously runtime-computed values data_checksums, wal_segment_size, and data_directory_mode would report values
thatwould not be accurate on the running server. They also can only be
>
> be what ?
Removed.
> | Add server variable allow_in_place_tablespaces for tablespace testing (Thomas Munro)
>
> This is a developer setting, so doesn't need to be mentioned ?
Moved to Source Code.
> | Add function pg_settings_get_flags() to get the flags of server-side variables (Justin Pryzby)
>
> IMO this is the same, but I think maybe Michael things about it differently...
Uh, I thought it might hvae user value as well as developer.
> | Allow WAL full page writes to use LZ4 and ZSTD compression (Andrey Borodin, Justin Pryzby)
> | Add support for LZ4 and ZSTD compression of server-side base backups (Jeevan Ladhe, Robert Haas)
> | Allow pg_basebackup to decompress LZ4 and ZSTD compressed server-side base backups, and LZ4 and ZSTD compress
outputfiles (Dipesh Pandit, Jeevan Ladhe)
> | Add configure option --with-zstd to enable ZSTD build (Jeevan Ladhe, Robert Haas, Michael Paquier)
>
> Maybe these should say "Zstandard" ? See 586955dddecc95e0003262a3954ae83b68ce0372.
I wasn't aware that ZSTD stood for that, so updated.
> | The new options causes the column names to be output, and optionally verified on input.
>
> option
Fixed.
>
> | Previous the actual schema name was used.
>
> Previously
Fixed.
> | When EXPLAIN references the temporary object schema, refer to it as "pg_temp" (Amul Sul)
> | When specifying fractional interval values in units greater than months, round to the nearest month (Bruce
Momjian)
> | Limit support of psql to servers running PostgreSQL 9.2 and later (Tom Lane)
> | Limit support of pg_dump and pg_dumpall to servers running PostgreSQL 9.2 and later (Tom Lane)
> | Limit support of pg_upgrade to old servers running PostgreSQL 9.2 and later (Tom Lane)
> | Remove server support for old BASE_BACKUP command syntax and base backup protocol (Robert Haas)
>
> Do these need to be in the "compatibility" section ?
Uh, I think of compatibility as breakage, while removing support for
something doesn't seem like breakage. The protocol removal of
BASE_BACKUP only relates to people writing tools, I thought, so no
breakage for non-internals users. I didn't think the fractional
interval change would be a breakage, though maybe it is. I didn't think
EXPLAIN changes were user-parsed, so no breakage?
> | Fix inforcement of PL/pgSQL variable CONSTANT markings (Tom Lane)
>
> enforcement
Fixed.
> | Allow IP address matching against a server's certificate Subject Alternative Name (Jacob Champion)
>
> Should say "server certificate's" ?
Agreed.
> | Allow libpq's SSL private to be owned by the root user (David Steele)
>
> private *key*
I changed it to "private key file".
> | Have psql output all output if multiple queries are passed to the server at once (Fabien Coelho)
>
> all *results* ?
Yes, fixed.
> | This can be disabled setting SHOW_ALL_RESULTS.
>
> disabled *by* setting
Agreed, fixed.
> | Allow pg_basebackup's --compress option to control the compression method (Michael Paquier, Robert Haas)
>
> Should probably say "compression method and options"
Good point, that feature moved around during the development cycle.
> | Allow pg_basebackup to decompress LZ4 and ZSTD compressed server-side base backups, and LZ4 and ZSTD compress
outputfiles (Dipesh Pandit, Jeevan Ladhe)
> | Allow pg_basebackup to compress on the server slide and decompress on the client side before storage (Dipesh
Pandit)
>
> Maybe these should be combined into one entry ?
Uh, I think it applies to gzip as well so they can't be combined, and
they seem to do different things.
> | Add the LZ4 compression method to pg_receivewal (Georgios Kokolatos)
> | This is enabled via --compression-method=lz4 and requires binaries to be built using --with-lz4.
> | Redesign pg_receivewal's compression options (Georgios Kokolatos)
> | The new --compression-method option controls the type of compression, rather than just relying on --compress.
>
> It's --compress since 042a923ad.
Yep, fixed.
> | Previously, pg_receivewal would start based on the WAL file stored in the local archive directory, or at the
sendingserver's current WAL flush location. With this change, if the sending server is running Postgres 15 or later,
thelocal archive directory is empty, and a replication slot is specified, the replication slots restart point will be
used.
>
> slot's restart point (with a >>'<<)
Fixed.
> | Add dump/restore option --no-table-access-method to force restore to use only the default table access method
(JustinPryzby)
>
> remove "only" ?
I changed it to "to only use the default" since I think that is the
point --- it doesn't use anything but the default.
> | This is for portability in restoring from systems using non-default table access methods.
>
> I would remove part about "portability". The use-case I see for this is
> restoring something to a different table AM (not just heapam), in the same way
> as is possible for tablespaces:
> PGOPTIONS='-c default-table-access-method=foo' pg_restore --no-table-am ./dump
I removed the portability sentence.
> | Previously only the first invalid connection setting database was reported.
>
> "only the first database with an invalid connection setting..."
Yes, reworded.
> | Add new protocol message TARGET to specific a new COPY method to be for base backups (Robert Haas)
>
> specify
Fixed.
> | Automatically export server variables using PGDLLIMPORT on Windows (Robert Haas)
>
> I don't think it's "automatic" ?
Yes, reworded.
> | Allow informational escape sequences to be used in postgres_fdw's application name (Hayato Kuroda, Fujii Masao)
>
> I don't think this should be a separate entry
Uh, the entry above is about per-connection application name, while this
is about escapes --- seems different to me, and hard to combine.
> | This is enabled with the "parallel_commit" postgres_fdw option.
>
> It's an option to the SQL "SERVER" command.
Yes, reworded. URL contents updated:
https://momjian.us/pgsql_docs/release-15.html
Can you verify you like the new contents please? Thanks.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
On Tue, May 10, 2022 at 12:44:56PM -0700, Nathan Bossart wrote: > On Tue, May 10, 2022 at 08:07:45PM +0200, Erik Rijkers wrote: > > And one strangely unfinished sentence: > > 'They also can only be' > > I suspect this meant to highlight that "postgres -C" with runtime-computed > GUCs doesn't work if the server is running. Yes, you are correct --- wording updated. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Tue, May 10, 2022 at 04:17:59PM -0400, Jonathan Katz wrote: > On 5/10/22 11:44 AM, Bruce Momjian wrote: > > I have completed the first draft of the PG 15 release notes and you can > > see the results here: > > > > https://momjian.us/pgsql_docs/release-15.html > > Thanks for pulling this together. > > + Allow logical replication to transfer sequence changes > > I believe this was reverted in 2c7ea57e5, unless some other parts of this > work made it in. Yes, sorry, I missed that. Oddly, the unlogged sequence patch was retained, even though there is no value for it on the primary. I removed the sentence that mentioned that benefit from the release notes since it doesn't apply to PG 15 anymore. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Tue, May 10, 2022 at 04:18:10PM -0400, Bruce Momjian wrote: > > "separately" seems vague. Currently in v10-v13, extended stats are collected > > for partitioned tables, and collected for the "SELECT FROM ONLY"/parent case > > for inheritance parents. This changes to also collect stats for the "SELECT > > FROM tbl*" case. See also: 20220204214103.GT23027@telsasoft.com. > > Agreed, reworded. Can you check you like my new wording? Now it says: | Allow extended statistics to record statistics for a parent with all it children (Tomas Vondra, Justin Pryzby) it should say "its" children. > > | Add function pg_settings_get_flags() to get the flags of server-side variables (Justin Pryzby) > > > > IMO this is the same, but I think maybe Michael things about it differently... > > Uh, I thought it might hvae user value as well as developer. The list of flags it includes is defined as "the flags we needed to deprecate ./check_guc", but it could conceivably be useful to someone else... But it seems more like allow_in_place_tablespaces; it could go into the "source code" section, or be removed. > > | When EXPLAIN references the temporary object schema, refer to it as "pg_temp" (Amul Sul) > > | When specifying fractional interval values in units greater than months, round to the nearest month (Bruce Momjian) > > | Limit support of psql to servers running PostgreSQL 9.2 and later (Tom Lane) > > | Limit support of pg_dump and pg_dumpall to servers running PostgreSQL 9.2 and later (Tom Lane) > > | Limit support of pg_upgrade to old servers running PostgreSQL 9.2 and later (Tom Lane) > > | Remove server support for old BASE_BACKUP command syntax and base backup protocol (Robert Haas) > > > > Do these need to be in the "compatibility" section ? > > Uh, I think of compatibility as breakage, while removing support for > something doesn't seem like breakage. I think removing support which breaks a user-facing behavior is presumptively a compatibility issue. > I didn't think EXPLAIN changes were user-parsed, so no breakage? Why would we have explain(format json/xml) if it wasn't meant to be parsed ? At one point I was parsing its xml. I'll let other's comment about the rest of the list. > > | Automatically export server variables using PGDLLIMPORT on Windows (Robert Haas) > > > > I don't think it's "automatic" ? > > Yes, reworded. Maybe it's a tiny bit better to say: | Export all server variables on Windows using PGDLLIMPORT (Robert Haas) (Otherwise, "all server variables using PGDLLIMPORT" could sound like only those "server variables [which were] using PGDLLIMPORT" were exported). > > | Allow informational escape sequences to be used in postgres_fdw's application name (Hayato Kuroda, Fujii Masao) > > > > I don't think this should be a separate entry > > Uh, the entry above is about per-connection application name, while this > is about escapes --- seems different to me, and hard to combine. 449ab635052 postgres_fdw: Allow application_name of remote connection to be set via GUC. 6e0cb3dec10 postgres_fdw: Allow postgres_fdw.application_name to include escape sequences. 94c49d53402 postgres_fdw: Make postgres_fdw.application_name support more escape sequences. You have one entry for 449a, and one entry where you've combined 6e0c and 94c4. My point is that the 2nd two commits changed the behavior of the first commit, and I don't see why an end-user would want to know about the intermediate behavior from the middle of the development cycle when escape sequences weren't expanded. So I don't know why they'd be listed separately. -- Justin
> I have completed the first draft of the PG 15 release notes and you can > see the results here: > > https://momjian.us/pgsql_docs/release-15.html > Allow pgbench to retry after serialization and deadlock failures (Yugo Nagata, Marina Polyakova) This is in the "Additional Modules" section. I think this should be in the "Client Applications" section because pgbench lives in bin directory, not in contrib directory. Actually, pgbench was in the "Client Applications" section in the PG 14 release note. Best reagards, -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp
On Tue, May 10, 2022 at 04:02:35PM -0500, Justin Pryzby wrote:
> On Tue, May 10, 2022 at 04:18:10PM -0400, Bruce Momjian wrote:
> > > "separately" seems vague. Currently in v10-v13, extended stats are collected
> > > for partitioned tables, and collected for the "SELECT FROM ONLY"/parent case
> > > for inheritance parents. This changes to also collect stats for the "SELECT
> > > FROM tbl*" case. See also: 20220204214103.GT23027@telsasoft.com.
> >
> > Agreed, reworded. Can you check you like my new wording?
>
> Now it says:
> | Allow extended statistics to record statistics for a parent with all it children (Tomas Vondra, Justin Pryzby)
>
> it should say "its" children.
Fixed.
> > > | Add function pg_settings_get_flags() to get the flags of server-side variables (Justin Pryzby)
> > >
> > > IMO this is the same, but I think maybe Michael things about it differently...
> >
> > Uh, I thought it might hvae user value as well as developer.
>
> The list of flags it includes is defined as "the flags we needed to deprecate
> ./check_guc", but it could conceivably be useful to someone else... But it
> seems more like allow_in_place_tablespaces; it could go into the "source code"
> section, or be removed.
Okay, I moved into source code.
> > > | When EXPLAIN references the temporary object schema, refer to it as "pg_temp" (Amul Sul)
> > > | When specifying fractional interval values in units greater than months, round to the nearest month (Bruce
Momjian)
> > > | Limit support of psql to servers running PostgreSQL 9.2 and later (Tom Lane)
> > > | Limit support of pg_dump and pg_dumpall to servers running PostgreSQL 9.2 and later (Tom Lane)
> > > | Limit support of pg_upgrade to old servers running PostgreSQL 9.2 and later (Tom Lane)
> > > | Remove server support for old BASE_BACKUP command syntax and base backup protocol (Robert Haas)
> > >
> > > Do these need to be in the "compatibility" section ?
> >
> > Uh, I think of compatibility as breakage, while removing support for
> > something doesn't seem like breakage.
>
> I think removing support which breaks a user-facing behavior is presumptively a
> compatibility issue.
I moved the EXPLAIN and fractional interval items to the compatibility
section. I didn't change the psql and pg_dump items since they are
already in their own sections and are clearly support removal rather
than direct changes in behavior that would be expected. However, if
someone else feels they should be moved, I will move them, so someone
please reply if you feel that way.
> > I didn't think EXPLAIN changes were user-parsed, so no breakage?
>
> Why would we have explain(format json/xml) if it wasn't meant to be parsed ?
> At one point I was parsing its xml.
>
> I'll let other's comment about the rest of the list.
Good point on the formatted EXPLAIN output being affected, which is why
moving it does make sense.
> > > | Automatically export server variables using PGDLLIMPORT on Windows (Robert Haas)
> > >
> > > I don't think it's "automatic" ?
> >
> > Yes, reworded.
>
> Maybe it's a tiny bit better to say:
> | Export all server variables on Windows using PGDLLIMPORT (Robert Haas)
>
> (Otherwise, "all server variables using PGDLLIMPORT" could sound like only
> those "server variables [which were] using PGDLLIMPORT" were exported).
Ah, yes, I see the improvement, done.
> > > | Allow informational escape sequences to be used in postgres_fdw's application name (Hayato Kuroda, Fujii
Masao)
> > >
> > > I don't think this should be a separate entry
> >
> > Uh, the entry above is about per-connection application name, while this
> > is about escapes --- seems different to me, and hard to combine.
>
> 449ab635052 postgres_fdw: Allow application_name of remote connection to be set via GUC.
> 6e0cb3dec10 postgres_fdw: Allow postgres_fdw.application_name to include escape sequences.
> 94c49d53402 postgres_fdw: Make postgres_fdw.application_name support more escape sequences.
>
> You have one entry for 449a, and one entry where you've combined 6e0c and 94c4.
>
> My point is that the 2nd two commits changed the behavior of the first commit,
> and I don't see why an end-user would want to know about the intermediate
> behavior from the middle of the development cycle when escape sequences weren't
> expanded. So I don't know why they'd be listed separately.
I see your point --- postgres_fdw.application_name supports escapes that
the normal application_name does not. I combined all three items now.
Thanks. The new entry is:
<!--
Author: Fujii Masao <fujii@postgresql.org>
2021-09-07 [449ab6350] postgres_fdw: Allow application_name of remote connectio
Author: Fujii Masao <fujii@postgresql.org>
2021-12-24 [6e0cb3dec] postgres_fdw: Allow postgres_fdw.application_name to inc
Author: Fujii Masao <fujii@postgresql.org>
2022-02-18 [94c49d534] postgres_fdw: Make postgres_fdw.application_name support
-->
<listitem>
<para>
Add server variable postgres_fdw.application_name to control the
application name of postgres_fdw connections (Hayato Kuroda)
</para>
<para>
Previously the remote application_name could only be set on the
remote server or via postgres_fdw connection specification.
postgres_fdw.application_name also supports escape sequences
for customization.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
>> I have completed the first draft of the PG 15 release notes and you can >> see the results here: >> >> https://momjian.us/pgsql_docs/release-15.html > >> Allow pgbench to retry after serialization and deadlock failures (Yugo Nagata, Marina Polyakova) > > This is in the "Additional Modules" section. I think this should be in > the "Client Applications" section because pgbench lives in bin > directory, not in contrib directory. Actually, pgbench was in the > "Client Applications" section in the PG 14 release note. I think you missed this: commit 06ba4a63b85e5aa47b325c3235c16c05a0b58b96 Use COPY FREEZE in pgbench for faster benchmark table population. Best reagards, -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp
On Wed, May 11, 2022 at 06:47:13AM +0900, Tatsuo Ishii wrote: > >> I have completed the first draft of the PG 15 release notes and you can > >> see the results here: > >> > >> https://momjian.us/pgsql_docs/release-15.html > > > >> Allow pgbench to retry after serialization and deadlock failures (Yugo Nagata, Marina Polyakova) > > > > This is in the "Additional Modules" section. I think this should be in > > the "Client Applications" section because pgbench lives in bin > > directory, not in contrib directory. Actually, pgbench was in the > > "Client Applications" section in the PG 14 release note. > > I think you missed this: > > commit 06ba4a63b85e5aa47b325c3235c16c05a0b58b96 > Use COPY FREEZE in pgbench for faster benchmark table population. I didn't mention it since it is automatic and I didn't think pgbench load time was significant enough to mention. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Tue, May 10, 2022 at 05:45:01PM -0400, Bruce Momjian wrote:
> I see your point --- postgres_fdw.application_name supports escapes that
> the normal application_name does not. I combined all three items now.
> Thanks. The new entry is:
The URL is updated with the current commits:
https://momjian.us/pgsql_docs/release-15.html
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
> On May 10, 2022, at 8:44 AM, Bruce Momjian <bruce@momjian.us> wrote:
>
> I have completed the first draft of the PG 15 release notes and you can
> see the results here
Thanks, Bruce! This release note:
• Prevent logical replication into tables where the subscription owner is subject to the table's row-level security
policies(Mark Dilger)
... should mention, independent of any RLS considerations, subscriptions are now applied under the privilege of the
subscriptionowner. I don't think we can fit it in the release note, but the basic idea is that:
CREATE SUBSCRIPTION ... CONNECTION '...' PUBLICATION ... WITH (enabled = false);
ALTER SUBSCRIPTION ... OWNER TO nonsuperuser_whoever;
ALTER SUBSCRIPTION ... ENABLE;
can be used to replicate a subscription without sync or apply workers operating as superuser. That's the main
advantage. Previously, subscriptions always ran with superuser privilege, which creates security concerns if the
publisheris malicious (or foolish). Avoiding any unintentional bypassing of RLS was just a necessary detail to close
thesecurity loophole, not the main point of the security enhancement.
—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On 5/10/22 4:18 PM, Bruce Momjian wrote: > On Tue, May 10, 2022 at 01:09:35PM -0500, Justin Pryzby wrote: > >> | Allow libpq's SSL private to be owned by the root user (David Steele) >> >> private *key* > > I changed it to "private key file". This was backpatched to all supported versions[1]. While I'm a huge fan of this behavior change for a plethora of reasons, I'm not sure if this should be included as part of the PG15 release notes, given it's in the 14.3 et al. Thanks, Jonathan [1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=2a1f84636dc335a3edf53a8361ae44bb2ae00093
Attachment
On Wed, 11 May 2022 at 03:44, Bruce Momjian <bruce@momjian.us> wrote: > > I have completed the first draft of the PG 15 release notes and you can > see the results here: > > https://momjian.us/pgsql_docs/release-15.html Thanks for doing that tedious work. I think the sort improvements done in v15 are worth a mention under General Performance. The commits for this were 91e9e89dc, 40af10b57 and 697492434. I've been running a few benchmarks between v14 and v15 over the past few days and a fairly average case speedup is about 25%. but there are cases where I've seen up to 400%. I think the increase is to an extent that we maybe should have considered making tweaks in cost_tuplesort(). I saw some plans that ran in about 60% of the time by disabling Hash Agg and allowing Sort / Group Agg to do the work. David
"Jonathan S. Katz" <jkatz@postgresql.org> writes:
> On 5/10/22 4:18 PM, Bruce Momjian wrote:
> | Allow libpq's SSL private to be owned by the root user (David Steele)
> This was backpatched to all supported versions[1]. While I'm a huge fan
> of this behavior change for a plethora of reasons, I'm not sure if this
> should be included as part of the PG15 release notes, given it's in the
> 14.3 et al.
It should not. However, the backpatch happened later than the commit
to HEAD, and our git_changelog tool is not smart enough to match them
up, so Bruce didn't see that there were followup commits. That's a
generic hazard for major-release notes; if you spot any other cases
please do mention them.
regards, tom lane
On Tue, May 10, 2022 at 04:18:10PM -0400, Bruce Momjian wrote: > > | Add the LZ4 compression method to pg_receivewal (Georgios Kokolatos) > > | This is enabled via --compression-method=lz4 and requires binaries to be built using --with-lz4. > > | Redesign pg_receivewal's compression options (Georgios Kokolatos) > > | The new --compression-method option controls the type of compression, rather than just relying on --compress. > > > > It's --compress since 042a923ad. > > Yep, fixed. It now says: | The new --compression option controls the type of compression, rather than just relying on --compress. But the option is --compress, and not --compression.
| Remove incorrect duplicate partition tables in system view pg_publication_tables (Hou Zhijie) should say "partitions" ? "Do not show partitions whose parents are also published" (is that accurate?) | Allow system and TOAST B-tree indexes to efficiently store duplicates (Peter Geoghegan) | Previously de-duplication was disabled for these types of indexes. I think the user-facing change here is that (in addition to being "allowed"), it's now enabled by default for catalog indexes. "Enable de-duplication of system indexes by default". | Prevent changes to columns only indexed by BRIN indexes from preventing HOT updates (Josef Simanek) says "prevent" twice. "Allow HOT updates when changed columns are only indexed by BRIN indexes" (or "avoid precluding...") | Improve the performance of window functions that use row_number(), rank(), and count() (David Rowley) The essential feature is a new kind of "prosupport", which is implemented for those core functions. I suggest to add another sentence about how prosupport can also be added to user-defined/non-core functions. | Store server-level statistics in shared memory (Kyotaro Horiguchi, Andres Freund, Melanie Plageman) Should this be called "cumulative" statistics? As in b3abca68106d518ce5d3c0d9a1e0ec02a647ceda. | Allows view access to be controlled by privileges of the view user (Christoph Heiss) Allow | New function "The new function .." (a few times) | Improve the parallel pg_dump performance of TOAST tables (Tom Lane) I don't think this needs to be mentioned, unless maybe folded into an entry like "improve performance when dumping with many objects or relations with large toast tables". | Allow pg_basebackup to decompress LZ4 and Zstandard compressed server-side base backups, and LZ4 and Zstandard compressoutput files (Dipesh Pandit, Jeevan Ladhe) maybe: "... and to compress output files with LZ4 and Zstandard." | Add direct I/O support to macOS (Thomas Munro) | This only works if max_wal_senders=0 and wal_level=minimal. I think this should mention that it's only for WAL. | Remove status reporting during pg_upgrade operation if the output is not a terminal (Andres Freund) Maybe: "By default, do not output status information unless the output is a terminal" | Add new protocol message COMPRESSION and COMPRESSION_DETAIL to specify the compression method and level (Robert Haas) s/level/options/ ? | Prevent DROP DATABASE, DROP TABLESPACE, and ALTER DATABASE SET TABLESPACE from occasionally failing during concurrent useon Windows (Thomas Munro) Maybe this doesn't need to be mentioned ? | Fix pg_statio_all_tables to sum values for the rare case of TOAST tables with multiple indexes (Andrei Zubkov) | Previously such cases would have one row for each index. Doesn't need to be mentioned ? It doesn't seem like a "compatibility" issue anyway. Should this be included? 6b94e7a6da2 Consider fractional paths in generate_orderedappend_paths Should any of these be listed as incompatible changes (some of these I asked before, but the others are from another list). 95ab1e0a9db interval: round values when spilling to months 9cd28c2e5f1 Remove server support for old BASE_BACKUP command syntax. 0d4513b6138 Remove server support for the previous base backup protocol. ccd10a9bfa5 Fix enforcement of PL/pgSQL variable CONSTANT markings (Tom Lane) 38bfae36526 pg_upgrade: Move all the files generated internally to a subdirectory 376ce3e404b Prefer $HOME when looking up the current user's home directory. 7844c9918a4 psql: Show all query results by default 17a856d08be Change aggregated log format of pgbench. ? 73508475d69 Remove pg_atoi() ? aa64f23b029 Remove MaxBackends variable in favor of GetMaxBackends() function. ? d816f366bc4 psql: Make SSL info display more compact ? 27b02e070fd pg_upgrade: Don't print progress status when output is not a tty. ? ab4fd4f868e Remove 'datlastsysoid'.
On Tue, May 10, 2022 at 03:12:18PM -0700, Mark Dilger wrote:
>
>
> > On May 10, 2022, at 8:44 AM, Bruce Momjian <bruce@momjian.us> wrote:
> >
> > I have completed the first draft of the PG 15 release notes and you can
> > see the results here
>
>
> Thanks, Bruce! This release note:
>
> • Prevent logical replication into tables where the subscription owner is subject to the table's row-level
securitypolicies (Mark Dilger)
>
> ... should mention, independent of any RLS considerations, subscriptions are now applied under the privilege of the
subscriptionowner. I don't think we can fit it in the release note, but the basic idea is that:
>
> CREATE SUBSCRIPTION ... CONNECTION '...' PUBLICATION ... WITH (enabled = false);
> ALTER SUBSCRIPTION ... OWNER TO nonsuperuser_whoever;
> ALTER SUBSCRIPTION ... ENABLE;
>
> can be used to replicate a subscription without sync or apply workers operating as superuser. That's the main
advantage. Previously, subscriptions always ran with superuser privilege, which creates security concerns if the
publisheris malicious (or foolish). Avoiding any unintentional bypassing of RLS was just a necessary detail to close
thesecurity loophole, not the main point of the security enhancement.
Oh, interesting. New text:
<!--
Author: Jeff Davis <jdavis@postgresql.org>
2022-01-07 [a2ab9c06e] Respect permissions within logical replication.
-->
<listitem>
<para>
Allow logical replication to run as the owner of the publication (Mark Dilger)
</para>
<para>
Because row-level security policies are not checked, only
superusers, roles with bypassrls, and table owners can replicate
into tables with row-level security policies.
</para>
</listitem>
How is this?
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
On Tue, May 10, 2022 at 08:14:28PM -0400, Jonathan Katz wrote: > On 5/10/22 4:18 PM, Bruce Momjian wrote: > > On Tue, May 10, 2022 at 01:09:35PM -0500, Justin Pryzby wrote: > > > > > > | Allow libpq's SSL private to be owned by the root user (David Steele) > > > > > > private *key* > > > > I changed it to "private key file". > > This was backpatched to all supported versions[1]. While I'm a huge fan of > this behavior change for a plethora of reasons, I'm not sure if this should > be included as part of the PG15 release notes, given it's in the 14.3 et al. Right, is should be removed, and I have done so. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Tue, May 10, 2022 at 09:16:33PM -0400, Tom Lane wrote: > "Jonathan S. Katz" <jkatz@postgresql.org> writes: > > On 5/10/22 4:18 PM, Bruce Momjian wrote: > > | Allow libpq's SSL private to be owned by the root user (David Steele) > > > This was backpatched to all supported versions[1]. While I'm a huge fan > > of this behavior change for a plethora of reasons, I'm not sure if this > > should be included as part of the PG15 release notes, given it's in the > > 14.3 et al. > > It should not. However, the backpatch happened later than the commit > to HEAD, and our git_changelog tool is not smart enough to match them > up, so Bruce didn't see that there were followup commits. That's a > generic hazard for major-release notes; if you spot any other cases > please do mention them. Yes, known problem. :-( -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
> On May 10, 2022, at 6:46 PM, Bruce Momjian <bruce@momjian.us> wrote: > > Allow logical replication to run as the owner of the publication Make that "owner of the subscription". This change operates on the subscriber-side. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Tue, May 10, 2022 at 08:28:54PM -0500, Justin Pryzby wrote: > On Tue, May 10, 2022 at 04:18:10PM -0400, Bruce Momjian wrote: > > > | Add the LZ4 compression method to pg_receivewal (Georgios Kokolatos) > > > | This is enabled via --compression-method=lz4 and requires binaries to be built using --with-lz4. > > > | Redesign pg_receivewal's compression options (Georgios Kokolatos) > > > | The new --compression-method option controls the type of compression, rather than just relying on --compress. > > > > > > It's --compress since 042a923ad. > > > > Yep, fixed. > > It now says: > > | The new --compression option controls the type of compression, rather than just relying on --compress. > > But the option is --compress, and not --compression. Agreed, fixed. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Tue, May 10, 2022 at 06:49:48PM -0700, Mark Dilger wrote: > > > > On May 10, 2022, at 6:46 PM, Bruce Momjian <bruce@momjian.us> wrote: > > > > Allow logical replication to run as the owner of the publication > > Make that "owner of the subscription". This change operates on the subscriber-side. Thanks, done. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Wed, May 11, 2022 at 12:39:41PM +1200, David Rowley wrote: > On Wed, 11 May 2022 at 03:44, Bruce Momjian <bruce@momjian.us> wrote: > > > > I have completed the first draft of the PG 15 release notes and you can > > see the results here: > > > > https://momjian.us/pgsql_docs/release-15.html > > Thanks for doing that tedious work. > > I think the sort improvements done in v15 are worth a mention under > General Performance. The commits for this were 91e9e89dc, 40af10b57 > and 697492434. I've been running a few benchmarks between v14 and v15 > over the past few days and a fairly average case speedup is about 25%. > but there are cases where I've seen up to 400%. I think the increase > is to an extent that we maybe should have considered making tweaks in > cost_tuplesort(). I saw some plans that ran in about 60% of the time > by disabling Hash Agg and allowing Sort / Group Agg to do the work. Good point. Do you have any suggested text? I can't really see it clearly based on the commits, except "sorting is faster". -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Wed, 11 May 2022 at 14:02, Bruce Momjian <bruce@momjian.us> wrote: > > On Wed, May 11, 2022 at 12:39:41PM +1200, David Rowley wrote: > > I think the sort improvements done in v15 are worth a mention under > > General Performance. The commits for this were 91e9e89dc, 40af10b57 > > and 697492434. I've been running a few benchmarks between v14 and v15 > > over the past few days and a fairly average case speedup is about 25%. > > but there are cases where I've seen up to 400%. I think the increase > > is to an extent that we maybe should have considered making tweaks in > > cost_tuplesort(). I saw some plans that ran in about 60% of the time > > by disabling Hash Agg and allowing Sort / Group Agg to do the work. > > Good point. Do you have any suggested text? I can't really see it > clearly based on the commits, except "sorting is faster". If we're going to lump those into a single line then maybe something along the lines of: * Reduce memory consumption and improve performance of sorting tuples in memory I think one line is fine from a user's perspective, but it's slightly harder to know the order of the names in the credits given the 3 independent commits. David
On Wed, May 11, 2022 at 12:39:41PM +1200, David Rowley wrote: > I think the sort improvements done in v15 are worth a mention under > General Performance. The commits for this were 91e9e89dc, 40af10b57 > and 697492434. I've been running a few benchmarks between v14 and v15 > over the past few days and a fairly average case speedup is about 25%. > but there are cases where I've seen up to 400%. I think the increase > is to an extent that we maybe should have considered making tweaks in > cost_tuplesort(). I saw some plans that ran in about 60% of the time > by disabling Hash Agg and allowing Sort / Group Agg to do the work. Is there any reason not to consider it now ? Either for v15 or v15+1. I wonder if this is also relevant. 65014000b35 Replace polyphase merge algorithm with a simple balanced k-way merge. -- Justin
On 5/10/22 9:48 PM, Bruce Momjian wrote: > On Tue, May 10, 2022 at 09:16:33PM -0400, Tom Lane wrote: >> "Jonathan S. Katz" <jkatz@postgresql.org> writes: >>> On 5/10/22 4:18 PM, Bruce Momjian wrote: >>> | Allow libpq's SSL private to be owned by the root user (David Steele) >> >>> This was backpatched to all supported versions[1]. While I'm a huge fan >>> of this behavior change for a plethora of reasons, I'm not sure if this >>> should be included as part of the PG15 release notes, given it's in the >>> 14.3 et al. >> >> It should not. However, the backpatch happened later than the commit >> to HEAD, and our git_changelog tool is not smart enough to match them >> up, so Bruce didn't see that there were followup commits. That's a >> generic hazard for major-release notes; if you spot any other cases >> please do mention them. > > Yes, known problem. :-( Got it. I did a scan of the release notes and diff'd with the 14.[1-3] and did not find any additional overlap. Jonathan
Attachment
On 5/10/22 10:31 PM, David Rowley wrote: > On Wed, 11 May 2022 at 14:02, Bruce Momjian <bruce@momjian.us> wrote: >> >> On Wed, May 11, 2022 at 12:39:41PM +1200, David Rowley wrote: >>> I think the sort improvements done in v15 are worth a mention under >>> General Performance. The commits for this were 91e9e89dc, 40af10b57 >>> and 697492434. I've been running a few benchmarks between v14 and v15 >>> over the past few days and a fairly average case speedup is about 25%. >>> but there are cases where I've seen up to 400%. I think the increase >>> is to an extent that we maybe should have considered making tweaks in >>> cost_tuplesort(). I saw some plans that ran in about 60% of the time >>> by disabling Hash Agg and allowing Sort / Group Agg to do the work. >> >> Good point. Do you have any suggested text? I can't really see it >> clearly based on the commits, except "sorting is faster". > > If we're going to lump those into a single line then maybe something > along the lines of: > > * Reduce memory consumption and improve performance of sorting tuples in memory > > I think one line is fine from a user's perspective, but it's slightly > harder to know the order of the names in the credits given the 3 > independent commits. I think a brief description following the one-liner would be useful for the release notes. If you can share a few more details about the benchmarks, we can expand on the one-liner in the release announcement (as this sounds like one of those cool, buzz-y things :) Jonathan
Attachment
On Wed, 11 May 2022 at 14:38, Justin Pryzby <pryzby@telsasoft.com> wrote: > > On Wed, May 11, 2022 at 12:39:41PM +1200, David Rowley wrote: > > I think the sort improvements done in v15 are worth a mention under > > General Performance. The commits for this were 91e9e89dc, 40af10b57 > > and 697492434. I've been running a few benchmarks between v14 and v15 > > over the past few days and a fairly average case speedup is about 25%. > > but there are cases where I've seen up to 400%. I think the increase > > is to an extent that we maybe should have considered making tweaks in > > cost_tuplesort(). I saw some plans that ran in about 60% of the time > > by disabling Hash Agg and allowing Sort / Group Agg to do the work. > > Is there any reason not to consider it now ? Either for v15 or v15+1. If the changes done had resulted in a change to the number of expected operations as far as big-O notation goes, then I think we might be able to do something. However, nothing changed in the number of operations. We only sped up the constant factors. If it were possible to adjust those constant factors based on some performance benchmarks results that were spat out by some single machine somewhere, then maybe we could do some tweaks. The problem is that to know that we're actually making some meaningful improvements to the costs, we'd want to get the opinion of >1 machine and likely >1 CPU architecture. That feels like something that would be much better to do during a release cycle rather than at this very late hour. The majority of my benchmarks were on AMD zen2 hardware. That's likely not going to reflect well on what the average hardware is that runs PostgreSQL. Also, I've no idea at this stage what we'd even do to cost_tuplesort(). The nruns calculation is a bit fuzzy and never really took the power-of-2 wastage that 40af10b57 reduces. Maybe there's some argument for adjusting the 2.0 constant in compute_cpu_sort_cost() based on what's done in 697492434. But there's plenty of datatypes that don't use the new sort specialization functions. Would we really want to add extra code to the planner to get it to try and figure that out? David
On Tue, May 10, 2022 at 08:31:17PM -0500, Justin Pryzby wrote:
> | Remove incorrect duplicate partition tables in system view pg_publication_tables (Hou Zhijie)
>
> should say "partitions" ?
> "Do not show partitions whose parents are also published" (is that accurate?)
I went with:
Remove incorrect duplicate partitions in system view
pg_publication_tables (Hou Zhijie)
> | Allow system and TOAST B-tree indexes to efficiently store duplicates (Peter Geoghegan)
> | Previously de-duplication was disabled for these types of indexes.
>
> I think the user-facing change here is that (in addition to being "allowed"),
> it's now enabled by default for catalog indexes. "Enable de-duplication of
> system indexes by default".
I went with:
Enable system and TOAST B-tree indexes to efficiently store duplicates
(Peter Geoghegan)
> | Prevent changes to columns only indexed by BRIN indexes from preventing HOT updates (Josef Simanek)
>
> says "prevent" twice.
> "Allow HOT updates when changed columns are only indexed by BRIN indexes"
> (or "avoid precluding...")
I went with:
Prevent changes to columns only indexed by BRIN indexes from
disabling HOT updates (Josef Simanek)
> | Improve the performance of window functions that use row_number(), rank(), and count() (David Rowley)
>
> The essential feature is a new kind of "prosupport", which is implemented for
> those core functions. I suggest to add another sentence about how prosupport
> can also be added to user-defined/non-core functions.
Uh, I don't see how "prosupport" would be relevant for users to know
about.
> | Store server-level statistics in shared memory (Kyotaro Horiguchi, Andres Freund, Melanie Plageman)
>
> Should this be called "cumulative" statistics? As in b3abca68106d518ce5d3c0d9a1e0ec02a647ceda.
Uh, they are counters, which I guess is cummulative, but that doesn't
seem very descriptive. The documentation call it the statistics
collector, but I am not sure we even have that anymore with an in-memory
implementation. I am kind of not sure what to call it.
> | Allows view access to be controlled by privileges of the view user (Christoph Heiss)
>
> Allow
Fixed.
> | New function
>
> "The new function .." (a few times)
Uh, I only see it once.
> | Improve the parallel pg_dump performance of TOAST tables (Tom Lane)
>
> I don't think this needs to be mentioned, unless maybe folded into an entry
> like "improve performance when dumping with many objects or relations with
> large toast tables".
I mentioned it because I thought users who tried parallelism might find
it is faster now so they should re-test it, no?
> | Allow pg_basebackup to decompress LZ4 and Zstandard compressed server-side base backups, and LZ4 and Zstandard
compressoutput files (Dipesh Pandit, Jeevan Ladhe)
>
> maybe: "... and to compress output files with LZ4 and Zstandard."
Yes, I like that better, done.
> | Add direct I/O support to macOS (Thomas Munro)
> | This only works if max_wal_senders=0 and wal_level=minimal.
>
> I think this should mention that it's only for WAL.
Agreed, done.
> | Remove status reporting during pg_upgrade operation if the output is not a terminal (Andres Freund)
>
> Maybe: "By default, do not output status information unless the output is a terminal"
I went with:
Disable default status reporting during pg_upgrade operation if
the output is not a terminal (Andres Freund)
> | Add new protocol message COMPRESSION and COMPRESSION_DETAIL to specify the compression method and level (Robert
Haas)
>
> s/level/options/ ?
Ah, yes, this changed to be more generic than level, done.
> | Prevent DROP DATABASE, DROP TABLESPACE, and ALTER DATABASE SET TABLESPACE from occasionally failing during
concurrentuse on Windows (Thomas Munro)
>
> Maybe this doesn't need to be mentioned ?
Uh, the previous behavior seems pretty bad so I wanted to mention it
will not happen anymore.
> | Fix pg_statio_all_tables to sum values for the rare case of TOAST tables with multiple indexes (Andrei Zubkov)
> | Previously such cases would have one row for each index.
>
> Doesn't need to be mentioned ?
> It doesn't seem like a "compatibility" issue anyway.
Uh, there were certain cases where multiple indexes happened and I think
we need to tell people it is no longer a problem to work around, no?
> Should this be included?
> 6b94e7a6da2 Consider fractional paths in generate_orderedappend_paths
I looked at that but didn't see how it would be relevent for users. Do
you have a suggestion for text?
> Should any of these be listed as incompatible changes (some of these I asked
> before, but the others are from another list).
>
> 95ab1e0a9db interval: round values when spilling to months
Yes, moved already.
> 9cd28c2e5f1 Remove server support for old BASE_BACKUP command syntax.
Seems internal-only so moved to Source Code.
> 0d4513b6138 Remove server support for the previous base backup protocol.
Same.
> ccd10a9bfa5 Fix enforcement of PL/pgSQL variable CONSTANT markings (Tom Lane)
I didn't see not enforcing constant as an incompatibility, but rather a
bug.
> 38bfae36526 pg_upgrade: Move all the files generated internally to a subdirectory
I think since we have a pg_upgrade section, it seems better there.
> 376ce3e404b Prefer $HOME when looking up the current user's home directory.
Uh, I didn't think so.
> 7844c9918a4 psql: Show all query results by default
Same.
> 17a856d08be Change aggregated log format of pgbench.
We have a pgbench section and I can't see it. I am trying to keep
incompatiblities as things related to in-production problems or
surprises.
> ? 73508475d69 Remove pg_atoi()
I don't see who would care except for internals folks.
> ? aa64f23b029 Remove MaxBackends variable in favor of GetMaxBackends() function.
Same.
> ? d816f366bc4 psql: Make SSL info display more compact
I did look at that but considered that this wouldn't be something that
would break anything.
> ? 27b02e070fd pg_upgrade: Don't print progress status when output is not a tty.
Same.
> ? ab4fd4f868e Remove 'datlastsysoid'.
Seemed too internal.
Thanks for all these ideas!
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
On Wed, 11 May 2022 at 15:41, Bruce Momjian <bruce@momjian.us> wrote: > > On Tue, May 10, 2022 at 08:31:17PM -0500, Justin Pryzby wrote: > > | Improve the performance of window functions that use row_number(), rank(), and count() (David Rowley) > > > > The essential feature is a new kind of "prosupport", which is implemented for > > those core functions. I suggest to add another sentence about how prosupport > > can also be added to user-defined/non-core functions. > > Uh, I don't see how "prosupport" would be relevant for users to know > about. I'd say if it's not mentioned in our documentation then we shouldn't put it in the release notes. Currently, it's only documented in the source code. If someone felt strongly that something should be written in the documents about this then I'd say only then should we consider mentioning it in the release notes. David
On Wed, May 11, 2022 at 12:44 AM Bruce Momjian <bruce@momjian.us> wrote: > I have completed the first draft of the PG 15 release notes Thanks. Regarding: <!-- Author: Alvaro Herrera <alvherre@alvh.no-ip.org> 2022-03-20 [ba9a7e392] Enforce foreign key correctly during cross-partition upd --> <listitem> <para> Improve the trigger behavior of updates on partitioned tables that move rows between partitions (Amit Langote) </para> <para> Previously, such updates fired delete triggers on the source partition and fired insert triggers on the target partition. PostgreSQL will now fire an update trigger on the partition root. This makes foreign key behavior more consistent. ALL TRIGGERS? </para> </listitem> The commit is intended to only change the behavior of RI triggers, while leaving user-defined triggers firing as before. I think it might be a good idea to be specific by wording this, maybe as follows? Improve the firing of foreign key triggers during cross-partition updates of partitioned tables (Amit Langote) Previously, such updates fired delete triggers on the source partition and insert triggers on the target partition, whereas PostgreSQL will now fire update triggers on the partitioned table mentioned in the query, which makes the behavior of foreign keys pointing into that table more consistent. Note that other user-defined triggers are fired as they were before. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com
On Tue, 10 May 2022 at 17:44, Bruce Momjian <bruce@momjian.us> wrote: > > I have completed the first draft of the PG 15 release notes and you can > see the results here: > > https://momjian.us/pgsql_docs/release-15.html > > The feature count is similar to recent major releases: > > release-10 195 > release-11 185 > release-12 198 > release-13 183 > release-14 229 > --> release-15 186 > > I assume there will be major adjustments in the next few weeks based on > feedback. With 10a8d138 (an extension on the work of PG14's 3c3b8a4b, which was listed in the release notes for PG14) we now truncate the LP array in more cases, which means that overall a workload with mainly HOT updates might require VACUUM to run less often to keep the relation size stable. I will admit that I am tooting my own horn, but I think it is worth mentioning for DBAs, as it reduces the bloating behaviour of certain classes of bloat-generating workloads (and thus they might need to re-tune any vacuum-related settings). -Matthias
I think this item is pointless: Remove unused function parameter in get_qual_from_partbound() (Hou Zhijie) it's just a C-level code, and we don't document such API changes. If we were to document them all, it'd be a very very long document. Here: Improve the algorithm used to compute random() (Fabien Coelho) This will cause setseed() followed by random() to return a different value than on older servers. Maybe it's clearer as "This will cause random() sequences to differ from what was emitted by prior versions for the same seed values." If I don't know anything about the random() API, I understand this as saying that setseed() returns a value, and we only changed that value when random() is called afterwards. Here: Fix ALTER TRIGGER RENAME on partitioned tables to rename partitions (Arne Roland, Álvaro Herrera) Also prohibit cloned triggers from being renamed. "... to rename the corresponding triggers on partitions. Also prohibit such triggers on partitions from being renamed." (It's not the *partitions* that are renamed but the triggers, obviously.) Here: Add server variable recursive_worktable_factor to allow the user to specify the expected recursive query worktable size (Simon Riggs) WHAT IS A WORKTABLE? NOT DEFINED. Do we need to explain in the relnotes that this is relevant to planning of WITH RECURSIVE queries? Generate periodic log message during slow server starts (Nitin Jadhav, Robert Haas, Álvaro Herrera) Please credit only Nitin and Robert, not me. I only edited the docs. Allow members of the pg_checkpointer predefined role to run the CHECKPOINT command (Jeff Davis) The 11-era entry said that we *added* new roles for the tasks, and I think we should do likewise here: Add predefined role pg_checkpointer that enables to run CHECKPOINT Otherwise it sounds like pg_checkpointer already existed and we gave it this new responsibility. Here: Create unlogged sequences and allow them to be skipped in logical replication (Peter Eisentraut) This is not specific to logical replication, actually; it's a generic new feature of sequences. So I don't think it belongs in the logical replication section. But it's not clear to me where to put it. Here: Add SQL MERGE command to adjust one table to match another (Pavan Deolasee, Álvaro Herrera, Amit Langote, Simon Riggs) I'm not sure this accurately describes the purpose of the command. Maybe "Add SQL MERGE command that allows to run INSERT, UPDATE, DELETE subcommands based on another table or the output of a query." Also, it doesn't belong in the Utilities section. Maybe it should be in the SELECT,INSERT section, and rename the section to something like "SQL Queries", and put the whole JSON subsection inside that section (rather than inside the Functions section). I think Simon should appear as first author here. Add new protocol message TARGET to specify a new COPY method to be for base backups (Robert Haas) I think this one should be in some other section, maybe "Streaming Replication and Recovery". Add server variable archive_library to specify the library to be called for archiving (Nathan Bossart) Maybe "Allow site-specific WAL archiving, which may no longer use shell commands." or something to that effect? The reference to a library is a bit obscure. -- Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/ "In fact, the basic problem with Perl 5's subroutines is that they're not crufty enough, so the cruft leaks out into user-defined code instead, by the Conservation of Cruft Principle." (Larry Wall, Apocalypse 6)
On Wed, May 11, 2022 at 02:31:08PM +1200, David Rowley wrote:
> On Wed, 11 May 2022 at 14:02, Bruce Momjian <bruce@momjian.us> wrote:
> >
> > On Wed, May 11, 2022 at 12:39:41PM +1200, David Rowley wrote:
> > > I think the sort improvements done in v15 are worth a mention under
> > > General Performance. The commits for this were 91e9e89dc, 40af10b57
> > > and 697492434. I've been running a few benchmarks between v14 and v15
> > > over the past few days and a fairly average case speedup is about 25%.
> > > but there are cases where I've seen up to 400%. I think the increase
> > > is to an extent that we maybe should have considered making tweaks in
> > > cost_tuplesort(). I saw some plans that ran in about 60% of the time
> > > by disabling Hash Agg and allowing Sort / Group Agg to do the work.
> >
> > Good point. Do you have any suggested text? I can't really see it
> > clearly based on the commits, except "sorting is faster".
>
> If we're going to lump those into a single line then maybe something
> along the lines of:
>
> * Reduce memory consumption and improve performance of sorting tuples in memory
>
> I think one line is fine from a user's perspective, but it's slightly
> harder to know the order of the names in the credits given the 3
> independent commits.
Okay, I went with this:
<!--
Author: David Rowley <drowley@postgresql.org>
2021-07-22 [91e9e89dc] Make nodeSort.c use Datum sorts for single column sorts
Author: David Rowley <drowley@postgresql.org>
2022-04-04 [40af10b57] Use Generation memory contexts to store tuples in sorts
Author: John Naylor <john.naylor@postgresql.org>
2022-04-02 [697492434] Specialize tuplesort routines for different kinds of abb
-->
<listitem>
<para>
Improve performance and reduce memory consumption of in-memory
sorts (Ronan Dunklau, David Rowley, Thomas Munro)
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
On Wed, May 11, 2022 at 04:02:31PM +0900, Amit Langote wrote:
> On Wed, May 11, 2022 at 12:44 AM Bruce Momjian <bruce@momjian.us> wrote:
> > I have completed the first draft of the PG 15 release notes
> The commit is intended to only change the behavior of RI triggers,
> while leaving user-defined triggers firing as before. I think it
> might be a good idea to be specific by wording this, maybe as follows?
>
> Improve the firing of foreign key triggers during cross-partition
> updates of partitioned tables (Amit Langote)
>
> Previously, such updates fired delete triggers on the source partition
> and insert triggers on the target partition, whereas PostgreSQL will
> now fire update triggers on the partitioned table mentioned in the
> query, which makes the behavior of foreign keys pointing into that
> table more consistent. Note that other user-defined triggers are
> fired as they were before.
Yes, this is what I needed to know. The updated text is:
<!--
Author: Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-03-20 [ba9a7e392] Enforce foreign key correctly during cross-partition upd
-->
<listitem>
<para>
Improve foreign key behavior of updates on partitioned tables
that move rows between partitions (Amit Langote)
</para>
<para>
Previously, such updates ran delete actions on the source partition
and insert actions on the target partition. PostgreSQL will now
run update actions on the partition root.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
On Wed, May 11, 2022 at 02:19:20PM +0200, Matthias van de Meent wrote: > On Tue, 10 May 2022 at 17:44, Bruce Momjian <bruce@momjian.us> wrote: > > > > I have completed the first draft of the PG 15 release notes and you can > > see the results here: > > > > https://momjian.us/pgsql_docs/release-15.html > > > > The feature count is similar to recent major releases: > > > > release-10 195 > > release-11 185 > > release-12 198 > > release-13 183 > > release-14 229 > > --> release-15 186 > > > > I assume there will be major adjustments in the next few weeks based on > > feedback. > > With 10a8d138 (an extension on the work of PG14's 3c3b8a4b, which was > listed in the release notes for PG14) we now truncate the LP array in > more cases, which means that overall a workload with mainly HOT > updates might require VACUUM to run less often to keep the relation > size stable. > > I will admit that I am tooting my own horn, but I think it is worth > mentioning for DBAs, as it reduces the bloating behaviour of certain > classes of bloat-generating workloads (and thus they might need to > re-tune any vacuum-related settings). We barely document HOT updates, and this change seems too marginal to mention. I think we already have enough VACUUM changes that people will need to reevaluate their vacuum settings anyway. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Wed, May 11, 2022 at 04:12:23PM +0200, Álvaro Herrera wrote:
> I think this item is pointless:
>
> Remove unused function parameter in get_qual_from_partbound() (Hou Zhijie)
>
> it's just a C-level code, and we don't document such API changes. If we
> were to document them all, it'd be a very very long document.
Okay, removed. I had added it because of this commit text:
This is an external function that extensions could use, so this is
potentially a breaking change. No external callers are known, however,
and this will make it simpler to write such callers in the future.
> Here:
> Improve the algorithm used to compute random() (Fabien Coelho)
>
> This will cause setseed() followed by random() to return a different
> value than on older servers.
> Maybe it's clearer as "This will cause random() sequences to differ from
> what was emitted by prior versions for the same seed values." If I
> don't know anything about the random() API, I understand this as saying
> that setseed() returns a value, and we only changed that value when
> random() is called afterwards.
Yes, better, thanks.
> Here:
> Fix ALTER TRIGGER RENAME on partitioned tables to rename partitions
> (Arne Roland, Álvaro Herrera)
>
> Also prohibit cloned triggers from being renamed.
>
> "... to rename the corresponding triggers on partitions.
>
> Also prohibit such triggers on partitions from being renamed."
> (It's not the *partitions* that are renamed but the triggers,
> obviously.)
Okay, new text:
Fix ALTER TRIGGER RENAME on partitioned tables to properly rename
triggers an all partitions (Arne Roland, Álvaro Herrera)
> Here:
> Add server variable recursive_worktable_factor to allow the user to
> specify the expected recursive query worktable size (Simon Riggs)
>
> WHAT IS A WORKTABLE? NOT DEFINED.
> Do we need to explain in the relnotes that this is relevant to planning
> of WITH RECURSIVE queries?
You mean the syntax? I figured "recursive query" was enough, but the
item clearly needs help.
> Generate periodic log message during slow server starts (Nitin Jadhav,
> Robert Haas, Álvaro Herrera)
> Please credit only Nitin and Robert, not me. I only edited the docs.
Okay, done.
> Allow members of the pg_checkpointer predefined role to run the
> CHECKPOINT command (Jeff Davis)
>
> The 11-era entry said that we *added* new roles for the tasks, and I
> think we should do likewise here:
> Add predefined role pg_checkpointer that enables to run CHECKPOINT
> Otherwise it sounds like pg_checkpointer already existed and we gave it
> this new responsibility.
Agreed, much better. New text:
Add predefined role pg_checkpointer that allows members to run
CHECKPOINT (Jeff Davis)
> Here:
> Create unlogged sequences and allow them to be skipped in logical
> replication (Peter Eisentraut)
> This is not specific to logical replication, actually; it's a generic
> new feature of sequences. So I don't think it belongs in the logical
> replication section. But it's not clear to me where to put it.
Oh, yeah, I had it there because that was its value, but now that we
don't replication sequences, it needs to moved. I put it in the "Data
Types" section.
> Here:
> Add SQL MERGE command to adjust one table to match another (Pavan
> Deolasee, Álvaro Herrera, Amit Langote, Simon Riggs)
> I'm not sure this accurately describes the purpose of the command.
> Maybe "Add SQL MERGE command that allows to run INSERT, UPDATE, DELETE
> subcommands based on another table or the output of a query."
Uh, that sounds odd to me, though I realize it is accurate.
> Also, it doesn't belong in the Utilities section. Maybe it should be in
> the SELECT,INSERT section, and rename the section to something like
> "SQL Queries", and put the whole JSON subsection inside that section
Uh, SQL queries seems very vague --- isn't SELECT the only actual query,
and if not, aren't all commands queries.
> (rather than inside the Functions section).
> I think Simon should appear as first author here.
Done.
> Add new protocol message TARGET to specify a new COPY method to be for
> base backups (Robert Haas)
> I think this one should be in some other section, maybe "Streaming
> Replication and Recovery".
I didn't think anyone cared about the protocol so I put it in Source
Code.
>
> Add server variable archive_library to specify the library to be called
> for archiving (Nathan Bossart)
> Maybe "Allow site-specific WAL archiving, which may no longer use shell
> commands." or something to that effect? The reference to a library is
> a bit obscure.
I added this sentence below it:
Previously only shell commands could be called to perform archiving.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
On Wed, 11 May 2022 at 14:38, Justin Pryzby <pryzby@telsasoft.com> wrote: > I wonder if this is also relevant. > > 65014000b35 Replace polyphase merge algorithm with a simple balanced k-way merge. Thanks for highlighting that. It very much is relevant. In fact, it seems to account for most of the 25% I mentioned. That particular test was sorting 10 million tuples with 4MB of work_mem. I think that "Improve sorting performance (Heikki Linnakangas)" should be moved out from "E.1.3.1.2. Indexes" and put below "E.1.3.1.4. General Performance" The text likely should include the words "disk-based" so that it's clear that it's not the same as the other line about "in-memory sorts". I'd also be open to just having a single line too. I'd vote to put Heikki's name first if we did that. Maybe: * Improve performance of sorting tuples (Heikki Linnakangas, Ronan Dunklau, David Rowley, Thomas Munro) This improves the merging performance of individual on-disk sort batches, reduces memory consumption for in-memory sorts and reduces CPU overheads for certain in-memory sorts. David
On Thu, May 12, 2022 at 10:38:42AM +1200, David Rowley wrote:
> On Wed, 11 May 2022 at 14:38, Justin Pryzby <pryzby@telsasoft.com> wrote:
> > I wonder if this is also relevant.
> >
> > 65014000b35 Replace polyphase merge algorithm with a simple balanced k-way merge.
>
> Thanks for highlighting that. It very much is relevant. In fact, it
> seems to account for most of the 25% I mentioned. That particular
> test was sorting 10 million tuples with 4MB of work_mem.
>
> I think that "Improve sorting performance (Heikki Linnakangas)" should
> be moved out from "E.1.3.1.2. Indexes" and put below "E.1.3.1.4.
> General Performance"
Yes, good point, moved.
> The text likely should include the words "disk-based" so that it's
> clear that it's not the same as the other line about "in-memory
> sorts". I'd also be open to just having a single line too. I'd vote
> to put Heikki's name first if we did that.
>
> Maybe:
>
> * Improve performance of sorting tuples (Heikki Linnakangas, Ronan
> Dunklau, David Rowley, Thomas Munro)
>
> This improves the merging performance of individual on-disk sort
> batches, reduces memory consumption for in-memory sorts and reduces
> CPU overheads for certain in-memory sorts.
I kept separate entries:
<!--
Author: Heikki Linnakangas <heikki.linnakangas@iki.fi>
2021-10-18 [65014000b] Replace polyphase merge algorithm with a simple balanced
Author: Heikki Linnakangas <heikki.linnakangas@iki.fi>
2021-10-25 [166f94377] Clarify the logic in a few places in the new balanced me
-->
<listitem>
<para>
Improve performance for sorts that exceed work_mem (Heikki Linnakangas)
</para>
<para>
Specifically, switch to a batch sorting algorithm that uses more
output streams internally.
</para>
</listitem>
<!--
Author: David Rowley <drowley@postgresql.org>
2021-07-22 [91e9e89dc] Make nodeSort.c use Datum sorts for single column sorts
Author: David Rowley <drowley@postgresql.org>
2022-04-04 [40af10b57] Use Generation memory contexts to store tuples in sorts
Author: John Naylor <john.naylor@postgresql.org>
2022-04-02 [697492434] Specialize tuplesort routines for different kinds of abb
-->
<listitem>
<para>
Improve performance and reduce memory consumption of in-memory
sorts (Ronan Dunklau, David Rowley, Thomas Munro)
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
2022年5月11日(水) 0:44 Bruce Momjian <bruce@momjian.us>: > > I have completed the first draft of the PG 15 release notes and you can > see the results here: > > https://momjian.us/pgsql_docs/release-15.html > > The feature count is similar to recent major releases: > > release-10 195 > release-11 185 > release-12 198 > release-13 183 > release-14 229 > --> release-15 186 > > I assume there will be major adjustments in the next few weeks based on > feedback. Re this: >> Remove exclusive backup mode (David Steele, Nathan Bossart) >> >> If the database server stops abruptly while in this mode, the server could fail to start. >> The non-exclusive backup mode requires a continuous database connection during the backup. It'd be useful to mention exclusive backup mode has been deprecated since 9.6, lest the impression arise that an important-sounding feature has been torn out suddenly. Also not sure why we need to mention that non-exclusive backup requires a continuous database connection, AFAIR that was also the case with exclusive backups. The patch also removed 4 and added 2 new functions, a change which anyone maintaining backup utilities would need to be aware of. Patch attached with suggested changes. Regards Ian Barwick -- EnterpriseDB: https://www.enterprisedb.com
Attachment
On Thu, May 12, 2022 at 12:53 PM Bruce Momjian <bruce@momjian.us> wrote: > <listitem> > <para> > Improve performance and reduce memory consumption of in-memory > sorts (Ronan Dunklau, David Rowley, Thomas Munro) > </para> > </listitem> I'd also add John Naylor here, as he did a lot of work to validate and polish the specialisation stuff.
On Thu, May 12, 2022 at 10:37:56AM +0900, Ian Lawrence Barwick wrote: > >> Remove exclusive backup mode (David Steele, Nathan Bossart) > >> > >> If the database server stops abruptly while in this mode, the server could fail to start. > >> The non-exclusive backup mode requires a continuous database connection during the backup. > > It'd be useful to mention exclusive backup mode has been deprecated since 9.6, > lest the impression arise that an important-sounding feature has been torn out > suddenly. Also not sure why we need to mention that non-exclusive backup Well, the documentation was clear about it being deprecated, so I don't see a need to mention it in the release notes. > requires a continuous database connection, AFAIR that was also the case with > exclusive backups. Uh, you could do pg_backup_start, disconnect, then pg_backup_stop, no? I thought the non-exclusive mode required a continuous connection because it aborts if you disconnect. > The patch also removed 4 and added 2 new functions, a change which anyone > maintaining backup utilities would need to be aware of. > > Patch attached with suggested changes. Oh, good points, I had not noticed those renames and removals. URL updated with new text. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Thu, May 12, 2022 at 01:59:36PM +1200, Thomas Munro wrote: > On Thu, May 12, 2022 at 12:53 PM Bruce Momjian <bruce@momjian.us> wrote: > > <listitem> > > <para> > > Improve performance and reduce memory consumption of in-memory > > sorts (Ronan Dunklau, David Rowley, Thomas Munro) > > </para> > > </listitem> > > I'd also add John Naylor here, as he did a lot of work to validate and > polish the specialisation stuff. Sure, done. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
2022年5月12日(木) 11:01 Bruce Momjian <bruce@momjian.us>: > > On Thu, May 12, 2022 at 10:37:56AM +0900, Ian Lawrence Barwick wrote: > > >> Remove exclusive backup mode (David Steele, Nathan Bossart) > > >> > > >> If the database server stops abruptly while in this mode, the server could fail to start. > > >> The non-exclusive backup mode requires a continuous database connection during the backup. > > > > It'd be useful to mention exclusive backup mode has been deprecated since 9.6, > > lest the impression arise that an important-sounding feature has been torn out > > suddenly. > > Well, the documentation was clear about it being deprecated, so I don't > see a need to mention it in the release notes. Looking at the release notes from the point of view of someone who has maybe not been following the long-running debate on removing exclusive backups: "Important-sounding backup thing is suddenly gone! What was that again? Hmm, can't find anything in the now-current Pg 15 docs [*], do I need to worry about this!?" [*] the backup section has removed all mention of the word "exclusive" https://www.postgresql.org/docs/devel/continuous-archiving.html#BACKUP-LOWLEVEL-BASE-BACKUP versus: "Long-deprecated thing is finally gone, ah OK whatever". I am thinking back here to a point in my working life where the release notes were reviewed (by a team including non-Pg specialists) for potential issues when considering a major upgrade - from experience the more clarity with this kind of change the better so as not to unnecessarily raise alarm bells. > > Also not sure why we need to mention that non-exclusive backup > > requires a continuous database connection, AFAIR that was also the case with > > exclusive backups. > > Uh, you could do pg_backup_start, disconnect, then pg_backup_stop, no? > I thought the non-exclusive mode required a continuous connection > because it aborts if you disconnect. Aha, you are right, I was conflating server shutdown with disconnection. > > The patch also removed 4 and added 2 new functions, a change which anyone > > maintaining backup utilities would need to be aware of. > > > > Patch attached with suggested changes. > > Oh, good points, I had not noticed those renames and removals. URL > updated with new text. Thanks! Regards Ian Barwick -- EnterpriseDB: https://www.enterprisedb.com
Bruce Momjian <bruce@momjian.us> writes:
> On Thu, May 12, 2022 at 10:37:56AM +0900, Ian Lawrence Barwick wrote:
>>>> Remove exclusive backup mode (David Steele, Nathan Bossart)
>> It'd be useful to mention exclusive backup mode has been deprecated since 9.6,
>> lest the impression arise that an important-sounding feature has been torn out
>> suddenly.
> Well, the documentation was clear about it being deprecated, so I don't
> see a need to mention it in the release notes.
Yeah, but somebody reading these notes doesn't necessarily have that
old documentation at hand.
I think writing "Remove the long-deprecated exclusive backup mode"
would do nicely to make this point without many extra words.
regards, tom lane
On Thu, May 12, 2022 at 11:40:17AM +0900, Ian Lawrence Barwick wrote: > Looking at the release notes from the point of view of someone who has maybe > not been following the long-running debate on removing exclusive backups: > > "Important-sounding backup thing is suddenly gone! What was that > again? Hmm, can't > find anything in the now-current Pg 15 docs [*], do I need to worry > about this!?" > > [*] the backup section has removed all mention of the word "exclusive" > https://www.postgresql.org/docs/devel/continuous-archiving.html#BACKUP-LOWLEVEL-BASE-BACKUP > > versus: > > "Long-deprecated thing is finally gone, ah OK whatever". > > I am thinking back here to a point in my working life where the > release notes were reviewed > (by a team including non-Pg specialists) for potential issues when > considering a major > upgrade - from experience the more clarity with this kind of change > the better so > as not to unnecessarily raise alarm bells. Ah, you are right. I thought I had "deprecated" in the text, but I now see I did not, and we do have cases where we mention the deprecated status in previous release notes, so the new text is: Remove long-deprecated exclusive backup mode (David Steele, Nathan Bossart) -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
2022年5月12日(木) 11:46 Bruce Momjian <bruce@momjian.us>: > > On Thu, May 12, 2022 at 11:40:17AM +0900, Ian Lawrence Barwick wrote: > > Looking at the release notes from the point of view of someone who has maybe > > not been following the long-running debate on removing exclusive backups: > > > > "Important-sounding backup thing is suddenly gone! What was that > > again? Hmm, can't > > find anything in the now-current Pg 15 docs [*], do I need to worry > > about this!?" > > > > [*] the backup section has removed all mention of the word "exclusive" > > https://www.postgresql.org/docs/devel/continuous-archiving.html#BACKUP-LOWLEVEL-BASE-BACKUP > > > > versus: > > > > "Long-deprecated thing is finally gone, ah OK whatever". > > > > I am thinking back here to a point in my working life where the > > release notes were reviewed > > (by a team including non-Pg specialists) for potential issues when > > considering a major > > upgrade - from experience the more clarity with this kind of change > > the better so > > as not to unnecessarily raise alarm bells. > > Ah, you are right. I thought I had "deprecated" in the text, but I now > see I did not, and we do have cases where we mention the deprecated > status in previous release notes, so the new text is: > > Remove long-deprecated exclusive backup mode (David Steele, Nathan > Bossart) That works, thanks! Regards Ian Barwick -- EnterpriseDB: https://www.enterprisedb.com
On Wed, May 11, 2022 at 11:41 PM Bruce Momjian <bruce@momjian.us> wrote: > On Wed, May 11, 2022 at 04:02:31PM +0900, Amit Langote wrote: > > On Wed, May 11, 2022 at 12:44 AM Bruce Momjian <bruce@momjian.us> wrote: > > > I have completed the first draft of the PG 15 release notes > > The commit is intended to only change the behavior of RI triggers, > > while leaving user-defined triggers firing as before. I think it > > might be a good idea to be specific by wording this, maybe as follows? > > > > Improve the firing of foreign key triggers during cross-partition > > updates of partitioned tables (Amit Langote) > > > > Previously, such updates fired delete triggers on the source partition > > and insert triggers on the target partition, whereas PostgreSQL will > > now fire update triggers on the partitioned table mentioned in the > > query, which makes the behavior of foreign keys pointing into that > > table more consistent. Note that other user-defined triggers are > > fired as they were before. > > Yes, this is what I needed to know. The updated text is: > > <!-- > Author: Alvaro Herrera <alvherre@alvh.no-ip.org> > 2022-03-20 [ba9a7e392] Enforce foreign key correctly during cross-partition upd > --> > > <listitem> > <para> > Improve foreign key behavior of updates on partitioned tables > that move rows between partitions (Amit Langote) > </para> > > <para> > Previously, such updates ran delete actions on the source partition > and insert actions on the target partition. PostgreSQL will now > run update actions on the partition root. > </para> > </listitem> Looks fine to me. Though I think maybe we should write the last sentence as "PostgreSQL will now run update actions on the partition root mentioned in the query" to be less ambiguous about which "root", because it can also mean the actual root table in the partition tree. A user may be updating only a particular subtree by mentioning that subtree's root in the query, for example. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com
On Wed, May 11, 2022 at 2:02 AM Bruce Momjian <bruce@momjian.us> wrote: > > On Tue, May 10, 2022 at 04:17:59PM -0400, Jonathan Katz wrote: > > On 5/10/22 11:44 AM, Bruce Momjian wrote: > > > I have completed the first draft of the PG 15 release notes and you can > > > see the results here: > > > > > > https://momjian.us/pgsql_docs/release-15.html > > > > Thanks for pulling this together. > > > > + Allow logical replication to transfer sequence changes > > > > I believe this was reverted in 2c7ea57e5, unless some other parts of this > > work made it in. > > Yes, sorry, I missed that. Oddly, the unlogged sequence patch was > retained, even though there is no value for it on the primary. I > removed the sentence that mentioned that benefit from the release notes > since it doesn't apply to PG 15 anymore. > + Create unlogged sequences and allow them to be skipped in logical replication Is it right to say the second part of the sentence: "allow them to be skipped in logical replication" when we are not replicating them in the first place? -- With Regards, Amit Kapila.
On Thu, May 12, 2022 at 2:25 PM Amit Kapila <amit.kapila16@gmail.com> wrote: > > On Wed, May 11, 2022 at 2:02 AM Bruce Momjian <bruce@momjian.us> wrote: > > > > On Tue, May 10, 2022 at 04:17:59PM -0400, Jonathan Katz wrote: > > > On 5/10/22 11:44 AM, Bruce Momjian wrote: > > > > I have completed the first draft of the PG 15 release notes and you can > > > > see the results here: > > > > > > > > https://momjian.us/pgsql_docs/release-15.html > > > > > > Thanks for pulling this together. > > > > > > + Allow logical replication to transfer sequence changes > > > > > > I believe this was reverted in 2c7ea57e5, unless some other parts of this > > > work made it in. > > > > Yes, sorry, I missed that. Oddly, the unlogged sequence patch was > > retained, even though there is no value for it on the primary. I > > removed the sentence that mentioned that benefit from the release notes > > since it doesn't apply to PG 15 anymore. > > > > + Create unlogged sequences and allow them to be skipped in logical replication > > Is it right to say the second part of the sentence: "allow them to be > skipped in logical replication" when we are not replicating them in > the first place? > One more point related to logical replication features: > Add SQL functions to monitor the directory contents of replication slots (Bharath Rupireddy) Specifically, the functions are pg_ls_logicalsnapdir(), pg_ls_logicalmapdir(), and pg_ls_replslotdir(). They can be run by members of the predefined pg_monitor role. > This feature is currently under the section "Streaming Replication and Recovery". Shouldn't it be under "Logical Replication"? The function names themselves seem to indicate that they are used for logical replication contents. I think the replication slot-related function would fall under both categories but overall it seems to belong to the "Logical Replication" section. -- With Regards, Amit Kapila.
On Wed, May 11, 2022 at 1:44 AM Bruce Momjian <bruce@momjian.us> wrote: > > I have completed the first draft of the PG 15 release notes and you can > see the results here: > > https://momjian.us/pgsql_docs/release-15.html > > The feature count is similar to recent major releases: > > release-10 195 > release-11 185 > release-12 198 > release-13 183 > release-14 229 > --> release-15 186 > > I assume there will be major adjustments in the next few weeks based on > feedback. > I wonder if this is worth mentioning: Skip empty transactions for logical replication. commit d5a9d86d8ffcadc52ff3729cd00fbd83bc38643c https://github.com/postgres/postgres/commit/d5a9d86d8ffcadc52ff3729cd00fbd83bc38643c regards, Ajin Cherian Fujitsu Australia
On 5/11/22 10:50 PM, Ian Lawrence Barwick wrote: > 2022年5月12日(木) 11:46 Bruce Momjian <bruce@momjian.us>: >> >> On Thu, May 12, 2022 at 11:40:17AM +0900, Ian Lawrence Barwick wrote: >>> Looking at the release notes from the point of view of someone who has maybe >>> not been following the long-running debate on removing exclusive backups: >>> >>> "Important-sounding backup thing is suddenly gone! What was that >>> again? Hmm, can't >>> find anything in the now-current Pg 15 docs [*], do I need to worry >>> about this!?" >>> >>> [*] the backup section has removed all mention of the word "exclusive" >>> https://www.postgresql.org/docs/devel/continuous-archiving.html#BACKUP-LOWLEVEL-BASE-BACKUP >>> >>> versus: >>> >>> "Long-deprecated thing is finally gone, ah OK whatever". >>> >>> I am thinking back here to a point in my working life where the >>> release notes were reviewed >>> (by a team including non-Pg specialists) for potential issues when >>> considering a major >>> upgrade - from experience the more clarity with this kind of change >>> the better so >>> as not to unnecessarily raise alarm bells. >> >> Ah, you are right. I thought I had "deprecated" in the text, but I now >> see I did not, and we do have cases where we mention the deprecated >> status in previous release notes, so the new text is: >> >> Remove long-deprecated exclusive backup mode (David Steele, Nathan >> Bossart) > > > That works, thanks! A bit late to this conversation, but +1 from me. -- -David david@pgmasters.net
On Wednesday, May 11, 2022 12:44 AM Bruce Momjian <bruce@momjian.us> wrote: > I have completed the first draft of the PG 15 release notes and you can see the > results here: > > https://momjian.us/pgsql_docs/release-15.html > > The feature count is similar to recent major releases: > > release-10 195 > release-11 185 > release-12 198 > release-13 183 > release-14 229 > --> release-15 186 > > I assume there will be major adjustments in the next few weeks based on > feedback. Hi, I'd like to suggest that we mention a new option for subscription 'disable_on_error'. https://github.com/postgres/postgres/commit/705e20f8550c0e8e47c0b6b20b5f5ffd6ffd9e33 Best Regards, Takamichi Osumi
On Thu, May 12, 2022 at 02:27:26PM +0900, Amit Langote wrote:
> > Yes, this is what I needed to know. The updated text is:
> >
> > <!--
> > Author: Alvaro Herrera <alvherre@alvh.no-ip.org>
> > 2022-03-20 [ba9a7e392] Enforce foreign key correctly during cross-partition upd
> > -->
> >
> > <listitem>
> > <para>
> > Improve foreign key behavior of updates on partitioned tables
> > that move rows between partitions (Amit Langote)
> > </para>
> >
> > <para>
> > Previously, such updates ran delete actions on the source partition
> > and insert actions on the target partition. PostgreSQL will now
> > run update actions on the partition root.
> > </para>
> > </listitem>
>
> Looks fine to me. Though I think maybe we should write the last
> sentence as "PostgreSQL will now run update actions on the partition
> root mentioned in the query" to be less ambiguous about which "root",
> because it can also mean the actual root table in the partition tree.
> A user may be updating only a particular subtree by mentioning that
> subtree's root in the query, for example.
Okay, I went with:
Previously, such updates ran delete actions on the source
partition and insert actions on the target partition. PostgreSQL will
now run update actions on the referenced partition root.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
On Thu, May 12, 2022 at 02:25:36PM +0530, Amit Kapila wrote:
> On Wed, May 11, 2022 at 2:02 AM Bruce Momjian <bruce@momjian.us> wrote:
> > Yes, sorry, I missed that. Oddly, the unlogged sequence patch was
> > retained, even though there is no value for it on the primary. I
> > removed the sentence that mentioned that benefit from the release notes
> > since it doesn't apply to PG 15 anymore.
> >
>
> + Create unlogged sequences and allow them to be skipped in logical replication
>
> Is it right to say the second part of the sentence: "allow them to be
> skipped in logical replication" when we are not replicating them in
> the first place?
Oops, yeah, that second part was reverted; new text:
Allow the creation of unlogged sequences (Peter Eisentraut)
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
On Thu, May 12, 2022 at 04:40:49PM +0530, Amit Kapila wrote:
> One more point related to logical replication features:
>
> >
> Add SQL functions to monitor the directory contents of replication
> slots (Bharath Rupireddy)
>
> Specifically, the functions are pg_ls_logicalsnapdir(),
> pg_ls_logicalmapdir(), and pg_ls_replslotdir(). They can be run by
> members of the predefined pg_monitor role.
> >
>
> This feature is currently under the section "Streaming Replication and
> Recovery". Shouldn't it be under "Logical Replication"? The function
> names themselves seem to indicate that they are used for logical
> replication contents. I think the replication slot-related function
> would fall under both categories but overall it seems to belong to the
> "Logical Replication" section.
Oh, very good point! I missed that this is logical-slot-only
monitoring, so I moved the item to logical replication and changed the
description to:
Add SQL functions to monitor the directory contents of logical
replication slots (Bharath Rupireddy)
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
On Thu, May 12, 2022 at 09:32:48PM +1000, Ajin Cherian wrote: > On Wed, May 11, 2022 at 1:44 AM Bruce Momjian <bruce@momjian.us> wrote: > > > > I have completed the first draft of the PG 15 release notes and you can > > see the results here: > > > > https://momjian.us/pgsql_docs/release-15.html > > > > The feature count is similar to recent major releases: > > > > release-10 195 > > release-11 185 > > release-12 198 > > release-13 183 > > release-14 229 > > --> release-15 186 > > > > I assume there will be major adjustments in the next few weeks based on > > feedback. > > > > I wonder if this is worth mentioning: > > Skip empty transactions for logical replication. > commit d5a9d86d8ffcadc52ff3729cd00fbd83bc38643c > > https://github.com/postgres/postgres/commit/d5a9d86d8ffcadc52ff3729cd00fbd83bc38643c I looked at that but thought that everyone would already assume we skipped replication of empty transactions, and I didn't see much impact for the user, so I didn't include it. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Thu, May 12, 2022 at 01:35:39PM +0000, osumi.takamichi@fujitsu.com wrote: > I'd like to suggest that we mention a new option for subscription 'disable_on_error'. > > > https://github.com/postgres/postgres/commit/705e20f8550c0e8e47c0b6b20b5f5ffd6ffd9e33 Yes, I missed that one, added: <!-- Author: Amit Kapila <akapila@postgresql.org> 2022-03-14 [705e20f85] Optionally disable subscriptions on error. --> <listitem> <para> Allow subscribers to stop logical replication application on error (Osumi Takamichi, Mark Dilger) </para> <para> This is enabled with the subscriber option "disable_on_error" and avoids possible infinite loops during stream application. </para> </listitem> -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Thu, May 12, 2022, at 11:03 AM, Bruce Momjian wrote:
I looked at that but thought that everyone would already assume weskipped replication of empty transactions, and I didn't see much impactfor the user, so I didn't include it.
It certainly has an impact on heavy workloads that replicate tables with few
modifications. It receives a high traffic of 'begin' and 'commit' messages that
the previous Postgres versions have to handle (discard). I would classify it as
a performance improvement for logical replication. Don't have a strong opinion
if it should be mentioned or not.
On Thu, May 12, 2022 at 11:12:54AM -0300, Euler Taveira wrote: OB> On Thu, May 12, 2022, at 11:03 AM, Bruce Momjian wrote: > > I looked at that but thought that everyone would already assume we > skipped replication of empty transactions, and I didn't see much impact > for the user, so I didn't include it. > > It certainly has an impact on heavy workloads that replicate tables with few > modifications. It receives a high traffic of 'begin' and 'commit' messages that > the previous Postgres versions have to handle (discard). I would classify it as > a performance improvement for logical replication. Don't have a strong opinion > if it should be mentioned or not. Oh, so your point is that a transaction that only has SELECT would previously send an empty transaction? I thought this was only for apps that create literal empty transactions, which seem rare. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
I wonder if this is worth mentioning: Raise a WARNING for missing publications. commit 8f2e2bbf145384784bad07a96d461c6bbd91f597 https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=8f2e2bbf145384784bad07a96d461c6bbd91f597 Regards, Vignesh On Thu, May 12, 2022 at 7:52 PM Bruce Momjian <bruce@momjian.us> wrote: > > On Thu, May 12, 2022 at 11:12:54AM -0300, Euler Taveira wrote: > OB> On Thu, May 12, 2022, at 11:03 AM, Bruce Momjian wrote: > > > > I looked at that but thought that everyone would already assume we > > skipped replication of empty transactions, and I didn't see much impact > > for the user, so I didn't include it. > > > > It certainly has an impact on heavy workloads that replicate tables with few > > modifications. It receives a high traffic of 'begin' and 'commit' messages that > > the previous Postgres versions have to handle (discard). I would classify it as > > a performance improvement for logical replication. Don't have a strong opinion > > if it should be mentioned or not. > > Oh, so your point is that a transaction that only has SELECT would > previously send an empty transaction? I thought this was only for apps > that create literal empty transactions, which seem rare. > > -- > Bruce Momjian <bruce@momjian.us> https://momjian.us > EDB https://enterprisedb.com > > Indecision is a decision. Inaction is an action. Mark Batterson > > >
On Thu, May 12, 2022 at 08:05:40PM +0530, vignesh C wrote: > I wonder if this is worth mentioning: > > Raise a WARNING for missing publications. > commit 8f2e2bbf145384784bad07a96d461c6bbd91f597 > > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=8f2e2bbf145384784bad07a96d461c6bbd91f597 Reading the commit message, it looked like only a warning was being added, which was more of a helpful change rather than something we need to mention. However, if this means you could can now create a subscription for a missing publication that you couldn't do before, it should be added --- I couldn't tell from the patch. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Thu, May 12, 2022, at 11:22 AM, Bruce Momjian wrote:
On Thu, May 12, 2022 at 11:12:54AM -0300, Euler Taveira wrote:OB> On Thu, May 12, 2022, at 11:03 AM, Bruce Momjian wrote:>> I looked at that but thought that everyone would already assume we> skipped replication of empty transactions, and I didn't see much impact> for the user, so I didn't include it.>> It certainly has an impact on heavy workloads that replicate tables with few> modifications. It receives a high traffic of 'begin' and 'commit' messages that> the previous Postgres versions have to handle (discard). I would classify it as> a performance improvement for logical replication. Don't have a strong opinion> if it should be mentioned or not.Oh, so your point is that a transaction that only has SELECT wouldpreviously send an empty transaction? I thought this was only for appsthat create literal empty transactions, which seem rare.
No. It should be a write transaction. If you have a replication setup that
publish only table foo (that isn't modified often) and most of your
workload does not contain table foo, Postgres sends 'begin' and 'commit'
messages to subscriber even if there is no change to replicate.
Let me show you an example:
postgres=# CREATE TABLE foo (a integer primary key, b text);
CREATE TABLE
postgres=# CREATE TABLE bar (c integer primary key, d text);
CREATE TABLE
postgres=# CREATE TABLE baz (e integer primary key, f text);
CREATE TABLE
postgres=# CREATE PUBLICATION pubfoo FOR TABLE foo;
CREATE PUBLICATION
postgres=# SELECT pg_create_logical_replication_slot('slotfoo', 'pgoutput');
pg_create_logical_replication_slot
------------------------------------
(slotfoo,0/E709AC50)
(1 row)
Let's create a transaction without table foo:
postgres=# BEGIN;
BEGIN
postgres=*# INSERT INTO bar (c, d) VALUES(1, 'blah');
INSERT 0 1
postgres=*# INSERT INTO baz (e, f) VALUES(2, 'xpto');
INSERT 0 1
postgres=*# COMMIT;
COMMIT
As you can see, the replication slot contains messages for that transaction.
Although, table bar and baz are NOT published, the begin (B) and commit (C)
messages that refers to this transaction are sent to subscriber.
postgres=# SELECT chr(get_byte(data, 0)) FROM
pg_logical_slot_peek_binary_changes('slotfoo', NULL, NULL,
'proto_version', '1', 'publication_names', 'pubfoo');
chr
-----
B
C
(2 rows)
If you execute another transaction without table foo, there will be another B/C
pair.
postgres=# DELETE FROM baz WHERE e = 2;
DELETE 1
postgres=# SELECT chr(get_byte(data, 0)) FROM
pg_logical_slot_peek_binary_changes('slotfoo', NULL, NULL,
'proto_version', '1', 'publication_names', 'pubfoo');
chr
-----
B
C
B
C
(4 rows)
Let's create a transaction that uses table foo but also table bar:
postgres=# BEGIN;
BEGIN
postgres=*# INSERT INTO foo (a, b) VALUES(100, 'asdf');
INSERT 0 1
postgres=*# INSERT INTO bar (c, d) VALUES(200, 'qwert');
INSERT 0 1
postgres=*# COMMIT;
COMMIT
In this case, there will be other messages since the publication pubfoo
publishes table foo. ('I' means there is an INSERT for table foo).
postgres=# SELECT chr(get_byte(data, 0)), length(data) FROM
pg_logical_slot_peek_binary_changes('slotfoo', NULL, NULL,
'proto_version', '1', 'publication_names', 'pubfoo');
chr | length
-----+--------
B | 21
C | 26
B | 21
C | 26
B | 21
R | 41
I | 25
C | 26
(8 rows)
In summary, a logical replication setup sends 47 bytes per skipped transaction.
v15 won't send the first 2 B/C pairs. Discussion started here [1].
On Thursday, May 12, 2022 11:10 PM 'Bruce Momjian' <bruce@momjian.us> wrote: > On Thu, May 12, 2022 at 01:35:39PM +0000, osumi.takamichi@fujitsu.com > wrote: > > I'd like to suggest that we mention a new option for subscription > 'disable_on_error'. > > > > > > > https://github.com/postgres/postgres/commit/705e20f8550c0e8e47c0b6b20 > b5f5ffd6ffd9e33 > > Yes, I missed that one, added: > > <!-- > Author: Amit Kapila <akapila@postgresql.org> > 2022-03-14 [705e20f85] Optionally disable subscriptions on error. > --> > > <listitem> > <para> > Allow subscribers to stop logical replication application on error > (Osumi Takamichi, Mark Dilger) > </para> > > <para> > This is enabled with the subscriber option "disable_on_error" > and avoids possible infinite loops during stream application. > </para> > </listitem> Thank you ! In this last paragraph, how about replacing "infinite loops" with "infinite error loops" ? I think it makes the situation somewhat clear for readers. Best Regards, Takamichi Osumi
On Thu, May 12, 2022 at 10:52 PM Bruce Momjian <bruce@momjian.us> wrote: > Okay, I went with: > > Previously, such updates ran delete actions on the source > partition and insert actions on the target partition. PostgreSQL will > now run update actions on the referenced partition root. WFM, thanks. Btw, perhaps the following should be listed under E.1.3.2.1. Logical Replication, not E.1.3.1.1. Partitioning? <!-- Author: Amit Kapila <akapila@postgresql.org> 2021-12-08 [a61bff2bf] De-duplicate the result of pg_publication_tables view. --> <listitem> <para> Remove incorrect duplicate partitions in system view pg_publication_tables (Hou Zhijie) </para> </listitem> Attached a patch to do so. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com
On Fri, May 13, 2022 at 7:19 AM Amit Langote <amitlangote09@gmail.com> wrote: > > On Thu, May 12, 2022 at 10:52 PM Bruce Momjian <bruce@momjian.us> wrote: > > Okay, I went with: > > > > Previously, such updates ran delete actions on the source > > partition and insert actions on the target partition. PostgreSQL will > > now run update actions on the referenced partition root. > > WFM, thanks. > > Btw, perhaps the following should be listed under E.1.3.2.1. Logical > Replication, not E.1.3.1.1. Partitioning? > Right. > <!-- > Author: Amit Kapila <akapila@postgresql.org> > 2021-12-08 [a61bff2bf] De-duplicate the result of pg_publication_tables view. > --> > > <listitem> > <para> > Remove incorrect duplicate partitions in system view > pg_publication_tables (Hou Zhijie) > </para> > </listitem> > > Attached a patch to do so. > I don't see any attachment. -- With Regards, Amit Kapila.
On Fri, May 13, 2022 at 11:44 AM Amit Kapila <amit.kapila16@gmail.com> wrote: > On Fri, May 13, 2022 at 7:19 AM Amit Langote <amitlangote09@gmail.com> wrote: > > > > On Thu, May 12, 2022 at 10:52 PM Bruce Momjian <bruce@momjian.us> wrote: > > > Okay, I went with: > > > > > > Previously, such updates ran delete actions on the source > > > partition and insert actions on the target partition. PostgreSQL will > > > now run update actions on the referenced partition root. > > > > WFM, thanks. > > > > Btw, perhaps the following should be listed under E.1.3.2.1. Logical > > Replication, not E.1.3.1.1. Partitioning? > > > > Right. > > > <!-- > > Author: Amit Kapila <akapila@postgresql.org> > > 2021-12-08 [a61bff2bf] De-duplicate the result of pg_publication_tables view. > > --> > > > > <listitem> > > <para> > > Remove incorrect duplicate partitions in system view > > pg_publication_tables (Hou Zhijie) > > </para> > > </listitem> > > > > Attached a patch to do so. > > > > I don't see any attachment. Oops, attached this time. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com
Attachment
On Fri, May 13, 2022 at 6:02 AM Euler Taveira <euler@eulerto.com> wrote: > > On Thu, May 12, 2022, at 11:22 AM, Bruce Momjian wrote: > > On Thu, May 12, 2022 at 11:12:54AM -0300, Euler Taveira wrote: > OB> On Thu, May 12, 2022, at 11:03 AM, Bruce Momjian wrote: > > > > I looked at that but thought that everyone would already assume we > > skipped replication of empty transactions, and I didn't see much impact > > for the user, so I didn't include it. > > > > It certainly has an impact on heavy workloads that replicate tables with few > > modifications. It receives a high traffic of 'begin' and 'commit' messages that > > the previous Postgres versions have to handle (discard). I would classify it as > > a performance improvement for logical replication. Don't have a strong opinion > > if it should be mentioned or not. > > Oh, so your point is that a transaction that only has SELECT would > previously send an empty transaction? I thought this was only for apps > that create literal empty transactions, which seem rare. > > No. It should be a write transaction. If you have a replication setup that > publish only table foo (that isn't modified often) and most of your > workload does not contain table foo, Postgres sends 'begin' and 'commit' > messages to subscriber even if there is no change to replicate. > It reduces network traffic and improves performance by 3-14% on simple tests [1] like the one shown by Euler. I see a value in adding this as for the workloads where it hits, it seems more than 99% of network traffic [2] is due to these empty messages. [1] - https://www.postgresql.org/message-id/OSZPR01MB63105A71CFAA46F5BD7C9D7CFD1E9%40OSZPR01MB6310.jpnprd01.prod.outlook.com [2] - https://www.postgresql.org/message-id/CAMkU=1yohp9-dv48FLoSPrMqYEyyS5ZWkaZGD41RJr10xiNo_Q@mail.gmail.com -- With Regards, Amit Kapila.
On Fri, May 13, 2022 at 01:36:04AM +0000, osumi.takamichi@fujitsu.com wrote: > > <para> > > This is enabled with the subscriber option "disable_on_error" > > and avoids possible infinite loops during stream application. > > </para> > > </listitem> > Thank you ! > > In this last paragraph, how about replacing "infinite loops" > with "infinite error loops" ? I think it makes the situation somewhat > clear for readers. Agreed, done. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Fri, May 13, 2022 at 10:48:41AM +0900, Amit Langote wrote: > On Thu, May 12, 2022 at 10:52 PM Bruce Momjian <bruce@momjian.us> wrote: > > Okay, I went with: > > > > Previously, such updates ran delete actions on the source > > partition and insert actions on the target partition. PostgreSQL will > > now run update actions on the referenced partition root. > > WFM, thanks. > > Btw, perhaps the following should be listed under E.1.3.2.1. Logical > Replication, not E.1.3.1.1. Partitioning? > > <!-- > Author: Amit Kapila <akapila@postgresql.org> > 2021-12-08 [a61bff2bf] De-duplicate the result of pg_publication_tables view. > --> > > <listitem> > <para> > Remove incorrect duplicate partitions in system view > pg_publication_tables (Hou Zhijie) > </para> > </listitem> > > Attached a patch to do so. Agreed, done. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Fri, May 13, 2022 at 08:24:53AM +0530, Amit Kapila wrote:
> On Fri, May 13, 2022 at 6:02 AM Euler Taveira <euler@eulerto.com> wrote:
> >
> > On Thu, May 12, 2022, at 11:22 AM, Bruce Momjian wrote:
> >
> > On Thu, May 12, 2022 at 11:12:54AM -0300, Euler Taveira wrote:
> > OB> On Thu, May 12, 2022, at 11:03 AM, Bruce Momjian wrote:
> > >
> > > I looked at that but thought that everyone would already assume we
> > > skipped replication of empty transactions, and I didn't see much impact
> > > for the user, so I didn't include it.
> > >
> > > It certainly has an impact on heavy workloads that replicate tables with few
> > > modifications. It receives a high traffic of 'begin' and 'commit' messages that
> > > the previous Postgres versions have to handle (discard). I would classify it as
> > > a performance improvement for logical replication. Don't have a strong opinion
> > > if it should be mentioned or not.
> >
> > Oh, so your point is that a transaction that only has SELECT would
> > previously send an empty transaction? I thought this was only for apps
> > that create literal empty transactions, which seem rare.
> >
> > No. It should be a write transaction. If you have a replication setup that
> > publish only table foo (that isn't modified often) and most of your
> > workload does not contain table foo, Postgres sends 'begin' and 'commit'
> > messages to subscriber even if there is no change to replicate.
> >
>
> It reduces network traffic and improves performance by 3-14% on simple
> tests [1] like the one shown by Euler. I see a value in adding this as
> for the workloads where it hits, it seems more than 99% of network
> traffic [2] is due to these empty messages.
I see the point now --- new item:
<!--
Author: Amit Kapila <akapila@postgresql.org>
2022-03-30 [d5a9d86d8] Skip empty transactions for logical replication.
-->
<listitem>
<para>
Prevent logical replication of empty transactions (Ajin Cherian,
Hou Zhijie, Euler Taveira)
</para>
<para>
Previously, write transactions would send empty transactions to
subscribers if subscribed tables were not modified.
</para>
</listitem>
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
On Fri, May 13, 2022 at 9:18 PM Bruce Momjian <bruce@momjian.us> wrote: > > On Fri, May 13, 2022 at 08:24:53AM +0530, Amit Kapila wrote: > > On Fri, May 13, 2022 at 6:02 AM Euler Taveira <euler@eulerto.com> wrote: > > > > > > On Thu, May 12, 2022, at 11:22 AM, Bruce Momjian wrote: > > > > > > On Thu, May 12, 2022 at 11:12:54AM -0300, Euler Taveira wrote: > > > OB> On Thu, May 12, 2022, at 11:03 AM, Bruce Momjian wrote: > > > > > > > > I looked at that but thought that everyone would already assume we > > > > skipped replication of empty transactions, and I didn't see much impact > > > > for the user, so I didn't include it. > > > > > > > > It certainly has an impact on heavy workloads that replicate tables with few > > > > modifications. It receives a high traffic of 'begin' and 'commit' messages that > > > > the previous Postgres versions have to handle (discard). I would classify it as > > > > a performance improvement for logical replication. Don't have a strong opinion > > > > if it should be mentioned or not. > > > > > > Oh, so your point is that a transaction that only has SELECT would > > > previously send an empty transaction? I thought this was only for apps > > > that create literal empty transactions, which seem rare. > > > > > > No. It should be a write transaction. If you have a replication setup that > > > publish only table foo (that isn't modified often) and most of your > > > workload does not contain table foo, Postgres sends 'begin' and 'commit' > > > messages to subscriber even if there is no change to replicate. > > > > > > > It reduces network traffic and improves performance by 3-14% on simple > > tests [1] like the one shown by Euler. I see a value in adding this as > > for the workloads where it hits, it seems more than 99% of network > > traffic [2] is due to these empty messages. > > I see the point now --- new item: > > <!-- > Author: Amit Kapila <akapila@postgresql.org> > 2022-03-30 [d5a9d86d8] Skip empty transactions for logical replication. > --> > > <listitem> > <para> > Prevent logical replication of empty transactions (Ajin Cherian, > Hou Zhijie, Euler Taveira) > </para> > > <para> > Previously, write transactions would send empty transactions to > subscribers if subscribed tables were not modified. > </para> > </listitem> > Thanks! -- With Regards, Amit Kapila.
On Sat, May 14, 2022 at 10:22:10AM +0530, Amit Kapila wrote: > > I see the point now --- new item: > > > > <!-- > > Author: Amit Kapila <akapila@postgresql.org> > > 2022-03-30 [d5a9d86d8] Skip empty transactions for logical replication. > > --> > > > > <listitem> > > <para> > > Prevent logical replication of empty transactions (Ajin Cherian, > > Hou Zhijie, Euler Taveira) > > </para> > > > > <para> > > Previously, write transactions would send empty transactions to > > subscribers if subscribed tables were not modified. > > </para> > > </listitem> > > > > Thanks! I will admit I had a little trouble with the wording of this item. :-) -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Saturday, May 14, 2022 12:07 AM 'Bruce Momjian' <bruce@momjian.us> wrote:
> On Fri, May 13, 2022 at 01:36:04AM +0000, osumi.takamichi@fujitsu.com wrote:
> > > <para>
> > > This is enabled with the subscriber option "disable_on_error"
> > > and avoids possible infinite loops during stream application.
> > > </para>
> > > </listitem>
> > Thank you !
> >
> > In this last paragraph, how about replacing "infinite loops"
> > with "infinite error loops" ? I think it makes the situation somewhat
> > clear for readers.
>
> Agreed, done.
Thanks !
Best Regards,
Takamichi Osumi
Hi Bruce, "Improve validation of ASCII and UTF-8 text by processing 16 bytes at a time (John Naylor)" The reader might assume here that ASCII is optimized regardless of encoding, but it is only optimized in the context of UTF-8. So I would just mention UTF-8. Thanks! -- John Naylor EDB: http://www.enterprisedb.com
On Mon, May 16, 2022 at 01:21:22PM +0700, John Naylor wrote:
> Hi Bruce,
>
> "Improve validation of ASCII and UTF-8 text by processing 16 bytes at
> a time (John Naylor)"
>
> The reader might assume here that ASCII is optimized regardless of
> encoding, but it is only optimized in the context of UTF-8. So I would
> just mention UTF-8.
I struggled with this item because it seemed to me that even if the
UTF-8 text was only ASCII, it would benefit, so I just rewrote it to:
Improve validation of UTF-8 text (even ASCII-only) by processing 16
bytes at a time (John Naylor)
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
On Mon, May 16, 2022 at 10:09:18AM -0400, Bruce Momjian wrote:
> On Mon, May 16, 2022 at 01:21:22PM +0700, John Naylor wrote:
> > Hi Bruce,
> >
> > "Improve validation of ASCII and UTF-8 text by processing 16 bytes at
> > a time (John Naylor)"
> >
> > The reader might assume here that ASCII is optimized regardless of
> > encoding, but it is only optimized in the context of UTF-8. So I would
> > just mention UTF-8.
>
> I struggled with this item because it seemed to me that even if the
> UTF-8 text was only ASCII, it would benefit, so I just rewrote it to:
>
> Improve validation of UTF-8 text (even ASCII-only) by processing 16
> bytes at a time (John Naylor)
Newer wording:
Improve validation of UTF-8 text (even if only ASCII) by processing
16 bytes at a time (John Naylor)
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
On 2022-May-10, Bruce Momjian wrote: > I have completed the first draft of the PG 15 release notes and you can > see the results here: > > https://momjian.us/pgsql_docs/release-15.html Just to be clear -- 15beta1 will be released with the "new features and enhancements" list as empty, is that right? -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
On Mon, May 16, 2022 at 05:41:08PM +0200, Alvaro Herrera wrote: > On 2022-May-10, Bruce Momjian wrote: > > > I have completed the first draft of the PG 15 release notes and you can > > see the results here: > > > > https://momjian.us/pgsql_docs/release-15.html > > Just to be clear -- 15beta1 will be released with the "new features and > enhancements" list as empty, is that right? I assume so. Last year, it was empty until *after* 14rc1. https://www.postgresql.org/message-id/flat/CAH2-WzntQzn_jJSUZYZMPSTgc0C98_mZXYBU0TxzHLvkieTGUA@mail.gmail.com -- Justin
On Mon, May 16, 2022 at 11:02:39AM -0500, Justin Pryzby wrote: > On Mon, May 16, 2022 at 05:41:08PM +0200, Alvaro Herrera wrote: > > On 2022-May-10, Bruce Momjian wrote: > > > > > I have completed the first draft of the PG 15 release notes and you can > > > see the results here: > > > > > > https://momjian.us/pgsql_docs/release-15.html > > > > Just to be clear -- 15beta1 will be released with the "new features and > > enhancements" list as empty, is that right? > > I assume so. Last year, it was empty until *after* 14rc1. > > https://www.postgresql.org/message-id/flat/CAH2-WzntQzn_jJSUZYZMPSTgc0C98_mZXYBU0TxzHLvkieTGUA@mail.gmail.com Yeah, Jonathan does that, and it is pulled usually from the press release. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Sat, May 14, 2022 at 12:42 AM Bruce Momjian <bruce@momjian.us> wrote: > On Fri, May 13, 2022 at 10:48:41AM +0900, Amit Langote wrote: > > Btw, perhaps the following should be listed under E.1.3.2.1. Logical > > Replication, not E.1.3.1.1. Partitioning? > > > > <!-- > > Author: Amit Kapila <akapila@postgresql.org> > > 2021-12-08 [a61bff2bf] De-duplicate the result of pg_publication_tables view. > > --> > > > > <listitem> > > <para> > > Remove incorrect duplicate partitions in system view > > pg_publication_tables (Hou Zhijie) > > </para> > > </listitem> > > > > Attached a patch to do so. > > Agreed, done. Thank you. Though a bit late given beta is now wrapped, I have another partition item wording improvement suggestion: -Previously, a partitioned table with any LIST partition containing multiple values could not be used for ordered partition scans. Now only non-pruned LIST partitions are checked. This also helps with -partitioned tables with DEFAULT partitions. +Previously, an ordered partition scan would not be considered for a LIST-partitioned table with any partition containing multiple values, nor for partitioned tables with DEFAULT partition. I think the "Now only non-pruned LIST partitions are checked" bit in the original wording is really an implementation detail of the actual improvement that ordered partition scans are now possible in more cases -- it simply became easier for the code that implements this optimization to refer to non-pruned partitions, using a bitmapset rather than having to trawl through the whole array of partition rels, which is what I think the commit message of this item mentions. David can correct me if I got that wrong. Attached a patch. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com
Attachment
On Thu, 19 May 2022 at 14:41, Amit Langote <amitlangote09@gmail.com> wrote: > Though a bit late given beta is now wrapped, I have another partition > item wording improvement suggestion: > > -Previously, a partitioned table with any LIST partition containing > multiple values could not be used for ordered partition scans. Now > only non-pruned LIST partitions are checked. This also helps with > -partitioned tables with DEFAULT partitions. > > +Previously, an ordered partition scan would not be considered for a > LIST-partitioned table with any partition containing multiple values, > nor for partitioned tables with DEFAULT partition. I think your proposed wording does not really improve things. The "Now only non-pruned LIST partitions are checked" is important and I think Bruce did the right thing to mention that. Prior to this change, ordered scans were not possible if there was a DEFAULT or if any LIST partition allowed >1 value. Now, if the default partition is pruned and there are no non-pruned partitions that allow Datum values that are inter-mixed with ones from another non-pruned partition, then an ordered scan can be performed. For example, non-pruned partition a allows IN(1,3), and non-pruned partition b allows IN(2,4), we cannot do the ordered scan. With IN(1,2), IN(3,4), we can. David
On Thu, May 19, 2022 at 2:56 PM David Rowley <dgrowleyml@gmail.com> wrote: > On Thu, 19 May 2022 at 14:41, Amit Langote <amitlangote09@gmail.com> wrote: > > Though a bit late given beta is now wrapped, I have another partition > > item wording improvement suggestion: > > > > -Previously, a partitioned table with any LIST partition containing > > multiple values could not be used for ordered partition scans. Now > > only non-pruned LIST partitions are checked. This also helps with > > -partitioned tables with DEFAULT partitions. > > > > +Previously, an ordered partition scan would not be considered for a > > LIST-partitioned table with any partition containing multiple values, > > nor for partitioned tables with DEFAULT partition. > > I think your proposed wording does not really improve things. The > "Now only non-pruned LIST partitions are checked" is important and I > think Bruce did the right thing to mention that. Prior to this change, > ordered scans were not possible if there was a DEFAULT or if any LIST > partition allowed >1 value. Now, if the default partition is pruned > and there are no non-pruned partitions that allow Datum values that > are inter-mixed with ones from another non-pruned partition, then an > ordered scan can be performed. > > For example, non-pruned partition a allows IN(1,3), and non-pruned > partition b allows IN(2,4), we cannot do the ordered scan. With > IN(1,2), IN(3,4), we can. I think that's what I understood this change to be about. Before this change, partitions_are_ordered() only returned true if *all* partitions of a parent are known to be ordered, which they're not in the presence of the default partition and of a list partition containing out-of-order values. It didn't matter to partitions_are_ordered() that the caller might not care about those partitions being present in the PartitionDesc because of having been pruned by the query, but that information was not readily available . So, you added PartitionBoundInfo.interleaved_parts to record indexes of partitions containing out-of-order values and RelOptInfo.live_parts to record non-pruned partitions, which made it more feasible for partitions_are_ordered() to address those cases. I suppose you think it's better to be verbose by mentioning that partitions_are_ordered() now considers only non-pruned partitions which allows supporting more cases, but I see that as mentioning implementation details unnecessarily. Or maybe we could mention that but use a wording that doesn't make it sound like an implementation detail, like: +Previously, an ordered partition scan could not be used for a LIST-partitioned table with any partition containing multiple values, nor for partitioned tables with DEFAULT partition. Now it can be used in those cases at least for queries in which such partitions are pruned. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com
Attachment
On Mon, May 16, 2022 at 9:18 PM Bruce Momjian <bruce@momjian.us> wrote: > > Newer wording: > > Improve validation of UTF-8 text (even if only ASCII) by processing > 16 bytes at a time (John Naylor) Thanks! I also think Heikki should be mentioned as a coauthor here -- the ASCII coding was his work in large part. -- John Naylor EDB: http://www.enterprisedb.com
On Tue, May 24, 2022 at 12:16:10PM +0700, John Naylor wrote: > On Mon, May 16, 2022 at 9:18 PM Bruce Momjian <bruce@momjian.us> wrote: > > > > Newer wording: > > > > Improve validation of UTF-8 text (even if only ASCII) by processing > > 16 bytes at a time (John Naylor) > > Thanks! I also think Heikki should be mentioned as a coauthor here -- > the ASCII coding was his work in large part. Sure, done. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Thu, May 19, 2022 at 06:13:28PM +0900, Amit Langote wrote: > Or maybe we could mention that but use a wording that doesn't make it > sound like an implementation detail, like: > > +Previously, an ordered partition scan could not be used for a > LIST-partitioned table with any partition containing multiple values, > nor for partitioned tables with DEFAULT partition. Now it can be used > in those cases at least for queries in which such partitions are > pruned. Sorry, I just don't see this as an improvement because it starts with a complex term "an ordered partition scan" rather than simply "a partitioned table". -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Wed, May 25, 2022 at 8:36 AM Bruce Momjian <bruce@momjian.us> wrote: > On Thu, May 19, 2022 at 06:13:28PM +0900, Amit Langote wrote: > > Or maybe we could mention that but use a wording that doesn't make it > > sound like an implementation detail, like: > > > > +Previously, an ordered partition scan could not be used for a > > LIST-partitioned table with any partition containing multiple values, > > nor for partitioned tables with DEFAULT partition. Now it can be used > > in those cases at least for queries in which such partitions are > > pruned. > > Sorry, I just don't see this as an improvement because it starts with a > complex term "an ordered partition scan" rather than simply "a > partitioned table". The headline says "Allow ordered scans of partitions to avoid sorting in more cases", so I proposed starting the description too with "an ordered scan". Also, not sure about going with: "previously, <table-with-limiting-properties> could not be used for <scan-method>, but now it can be provided <conditions>" instead of: "previously, <scan-method> could not be used for <table-with-limiting-properties>, but now it can be provided <conditions>" as in my proposed wording, but maybe that's just me. Anyway, I still think it would be better to fix the description such that the cases in which ordered scans will continue to not be usable are clear. The existing text doesn't make clear, for example, that a DEFAULT partition if present must have been pruned for an ordered scan to be used. So I propose: +Previously, a partitioned table with DEFAULT partition or a LIST partition containing multiple values could not be used for ordered partition scans. Now it can be used at least in the cases where such partitions are pruned. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com
Attachment
On Wed, 25 May 2022 at 15:01, Amit Langote <amitlangote09@gmail.com> wrote: > +Previously, a partitioned table with DEFAULT partition or a LIST > partition containing multiple values could not be used for ordered > partition scans. Now it can be used at least in the cases where such > partitions are pruned. I think this one is an improvement. I'd drop "at least". David
On Wed, May 25, 2022 at 12:44 PM David Rowley <dgrowleyml@gmail.com> wrote: > On Wed, 25 May 2022 at 15:01, Amit Langote <amitlangote09@gmail.com> wrote: > > +Previously, a partitioned table with DEFAULT partition or a LIST > > partition containing multiple values could not be used for ordered > > partition scans. Now it can be used at least in the cases where such > > partitions are pruned. > > I think this one is an improvement. I'd drop "at least". Okay, I can agree that "at least" sounds a bit extraneous, so removed. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com
Attachment
On Wed, May 25, 2022 at 1:04 PM Amit Langote <amitlangote09@gmail.com> wrote: > On Wed, May 25, 2022 at 12:44 PM David Rowley <dgrowleyml@gmail.com> wrote: > > On Wed, 25 May 2022 at 15:01, Amit Langote <amitlangote09@gmail.com> wrote: > > > +Previously, a partitioned table with DEFAULT partition or a LIST > > > partition containing multiple values could not be used for ordered > > > partition scans. Now it can be used at least in the cases where such > > > partitions are pruned. > > > > I think this one is an improvement. I'd drop "at least". > > Okay, I can agree that "at least" sounds a bit extraneous, so removed. * I think it's better to s/...or a LIST partition/...or with a LIST partition * The capitalization of DEFAULT and LIST seems unnecessary. Updated the patch on those points. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com
Attachment
On Thu, May 26, 2022 at 10:31:14AM +0900, Amit Langote wrote:
> * I think it's better to s/...or a LIST partition/...or with a LIST partition
>
> * The capitalization of DEFAULT and LIST seems unnecessary.
>
> Updated the patch on those points.
I went with this text:
Previously, a partitioned table with a DEFAULT partition or a LIST
partition containing multiple values could not be used for ordered
partition scans. Now they can be used if these partitions are pruned.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
On Thu, May 26, 2022 at 11:17 AM Bruce Momjian <bruce@momjian.us> wrote: > On Thu, May 26, 2022 at 10:31:14AM +0900, Amit Langote wrote: > > * I think it's better to s/...or a LIST partition/...or with a LIST partition > > > > * The capitalization of DEFAULT and LIST seems unnecessary. > > > > Updated the patch on those points. > > I went with this text: > > Previously, a partitioned table with a DEFAULT partition or a LIST > partition containing multiple values could not be used for ordered > partition scans. Now they can be used if these partitions are pruned. Good enough for me, thanks. -- Thanks, Amit Langote EDB: http://www.enterprisedb.com
On Tue, May 10, 2022 at 11:44:15AM -0400, Bruce Momjian wrote: > I have completed the first draft of the PG 15 release notes > <!-- > Author: Noah Misch <noah@leadboat.com> > 2021-09-09 [b073c3ccd] Revoke PUBLIC CREATE from public schema, now owned by pg > --> > > <listitem> > <para> > Remove <literal>PUBLIC</literal> creation permission on the <link > linkend="ddl-schemas-public"><literal>public</literal> schema</link> > (Noah Misch) > </para> > > <para> > This is a change in the default for newly-created databases in > existing clusters and for new clusters; <literal>USAGE</literal> If you dump/reload an unmodified v14 template1 (as pg_dumpall and pg_upgrade do), your v15 template1 will have a v14 ACL on its public schema. At that point, the fate of "newly-created databases in existing clusters" depends on whether you clone template1 or template0. Does any of that detail belong here, or does the existing text suffice? > permissions on the <literal>public</literal> schema has not > been changed. Databases restored from previous Postgres releases > will be restored with their current permissions. Users wishing > to have the old permissions on new objects will need to grant The phrase "old permissions on new objects" doesn't sound right to me, but I'm not sure why. I think you're aiming for the fact that this is just a default; one can still change the ACL to anything, including to the old default. If these notes are going to mention the old default like they do so far, I think they should also urge readers to understand https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS before returning to the old default. What do you think? > <literal>CREATE</literal> permission for <literal>PUBLIC</literal> > on the <literal>public</literal> schema; this change can be made > on <literal>template1</literal> to cause all new databases > to have these permissions. <literal>template1</literal> > permissions for <application>pg_dumpall</application> and > <application>pg_upgrade</application>? pg_dumpall will change template1. I think pg_upgrade will too, and neither program will change template0. > </para> > </listitem> > > <!-- > Author: Noah Misch <noah@leadboat.com> > 2021-09-09 [b073c3ccd] Revoke PUBLIC CREATE from public schema, now owned by pg > --> > > <listitem> > <para> > Change the owner of the <literal>public</literal> schema to > <literal>pg_database_owner</literal> (Noah Misch) > </para> > > <para> > Previously it was the literal user name of the database owner. It was the bootstrap superuser. > Databases restored from previous Postgres releases will be restored > with their current owner specification. > </para> > </listitem>
On Mon, Jun 27, 2022 at 11:37:19PM -0700, Noah Misch wrote: > On Tue, May 10, 2022 at 11:44:15AM -0400, Bruce Momjian wrote: > > I have completed the first draft of the PG 15 release notes > > > <!-- > > Author: Noah Misch <noah@leadboat.com> > > 2021-09-09 [b073c3ccd] Revoke PUBLIC CREATE from public schema, now owned by pg > > --> > > > > <listitem> > > <para> > > Remove <literal>PUBLIC</literal> creation permission on the <link > > linkend="ddl-schemas-public"><literal>public</literal> schema</link> > > (Noah Misch) > > </para> > > > > <para> > > This is a change in the default for newly-created databases in > > existing clusters and for new clusters; <literal>USAGE</literal> > > If you dump/reload an unmodified v14 template1 (as pg_dumpall and pg_upgrade > do), your v15 template1 will have a v14 ACL on its public schema. At that > point, the fate of "newly-created databases in existing clusters" depends on > whether you clone template1 or template0. Does any of that detail belong > here, or does the existing text suffice? I think it is very confusing to have template0 have one value and template1 have a different one, but as I understand it template0 will only be used for pg_dump comparison, and that will keep template1 with the same permissions, so I guess it is okay. > > permissions on the <literal>public</literal> schema has not > > been changed. Databases restored from previous Postgres releases > > will be restored with their current permissions. Users wishing > > to have the old permissions on new objects will need to grant > > The phrase "old permissions on new objects" doesn't sound right to me, but I'm > not sure why. I think you're aiming for the fact that this is just a default; > one can still change the ACL to anything, including to the old default. If > these notes are going to mention the old default like they do so far, I think > they should also urge readers to understand > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > before returning to the old default. What do you think? Agreed, the new text is: Users wishing to have the former permissions will need to grant <literal>CREATE</literal> permission for <literal>PUBLIC</literal> on the <literal>public</literal> schema; this change can be made on <literal>template1</literal> to cause all new databases to have these permissions. > > > <literal>CREATE</literal> permission for <literal>PUBLIC</literal> > > on the <literal>public</literal> schema; this change can be made > > on <literal>template1</literal> to cause all new databases > > to have these permissions. <literal>template1</literal> > > permissions for <application>pg_dumpall</application> and > > <application>pg_upgrade</application>? > > pg_dumpall will change template1. I think pg_upgrade will too, and neither > program will change template0. Okay, I will remove that question mark sentence. > > </para> > > </listitem> > > > > <!-- > > Author: Noah Misch <noah@leadboat.com> > > 2021-09-09 [b073c3ccd] Revoke PUBLIC CREATE from public schema, now owned by pg > > --> > > > > <listitem> > > <para> > > Change the owner of the <literal>public</literal> schema to > > <literal>pg_database_owner</literal> (Noah Misch) > > </para> > > > > <para> > > Previously it was the literal user name of the database owner. > > It was the bootstrap superuser. Okay, text updated, thanks. Applied patch attached. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Attachment
On Tue, Jun 28, 2022 at 1:35 PM Bruce Momjian <bruce@momjian.us> wrote: > Okay, text updated, thanks. Applied patch attached. I have some notes on these items: 1. "Allow vacuum to be more aggressive in setting the oldest frozenxid (Peter Geoghegan)" 2. "Add additional information to VACUUM VERBOSE and autovacuum logging messages (Peter Geoghegan)" The main enhancement to VACUUM for Postgres 15 was item 1, which taught VACUUM to dynamically track the oldest remaining XID (and the oldest remaining MXID) that will remain in the table at the end of the same VACUUM operation. These final/oldest XID/MXID values are what we now use to set relfrozenxid and relminmxid in pg_class. Previously we just set relfrozenxid/relminmxid to whatever XID/MXID value was used to determine which XIDs/MXIDs needed to be frozen. These values often indicated more about VACUUM implementation details (like the vacuum_freeze_min_age GUc's value) than the actual true contents of the table at the end of the most recent VACUUM. It might be worth explaining the shift directly in the release notes. The new approach is simpler and makes a lot more sense -- why should the relfrozenxid be closely tied to freezing? We don't necessarily have to freeze any tuple to advance relfrozenxid right up to the removable cutoff/OldestXmin used by VACUUM. For example, anti-wraparound VACUUMs that run against static tables now set relfrozenxid/relminmxid to VACUUM's removable cutoff/OldestXmin directly, without freezing anything (after the first time). Same with tables that happen to have every row deleted -- only the actual unfrozen XIDs/MXIDs left in the table matter, and if there happen to be none at all then we can use the same relfrozenxid as we would for a CREATE TABLE. All depends on what the workload allows. There will also be a real practical benefit for users that allocate a lot of MultiXactIds: We'll now have pg_class.relminmxid values that are much more reliable indicators of what is really going on in the table, MultiXactId-wise. I expect that this will make it much less likely that anti-wraparound VACUUMs will run needlessly against the largest tables, where there probably wasn't ever one single MultiXactId. In other words, the implementation will have more accurate information at the level of each table, and won't . I think that very uneven consumption of MultiXactIds at the table level is probably common in real databases. Plus VACUUM can usually remove a non-running MultiXact from a tuple's xmax, regardless of whether or not the mxid happens to be before the vacuum_multixact_freeze_min_age-based MXID cutoff -- VACUUM has always just set xmax to InvalidXid in passing when it's possible to do so easily. MultiXacts are inherently pretty short-lived information about row lockers at a point in time. We don't really need to keep them around for very long. We may now be able to truncate the two MultiXact related SLRUs much more frequently with some workloads. Finally, note that the new VACUUM VERBOSE output (which is now pretty much the same as the autovacuum log output) shows when and how relfrozenxid/relminmxid have advanced. This should make it relatively easy to observe these effects where they exist. Thanks -- Peter Geoghegan
On Tue, Jun 28, 2022 at 04:35:45PM -0400, Bruce Momjian wrote: > On Mon, Jun 27, 2022 at 11:37:19PM -0700, Noah Misch wrote: > > On Tue, May 10, 2022 at 11:44:15AM -0400, Bruce Momjian wrote: > > > I have completed the first draft of the PG 15 release notes > > > > > <!-- > > > Author: Noah Misch <noah@leadboat.com> > > > 2021-09-09 [b073c3ccd] Revoke PUBLIC CREATE from public schema, now owned by pg > > > --> > > > > > > <listitem> > > > <para> > > > Remove <literal>PUBLIC</literal> creation permission on the <link > > > linkend="ddl-schemas-public"><literal>public</literal> schema</link> > > > (Noah Misch) > > > </para> > > > > > > <para> > > > This is a change in the default for newly-created databases in > > > existing clusters and for new clusters; <literal>USAGE</literal> > > > > If you dump/reload an unmodified v14 template1 (as pg_dumpall and pg_upgrade > > do), your v15 template1 will have a v14 ACL on its public schema. At that > > point, the fate of "newly-created databases in existing clusters" depends on > > whether you clone template1 or template0. Does any of that detail belong > > here, or does the existing text suffice? > > I think it is very confusing to have template0 have one value and > template1 have a different one, but as I understand it template0 will > only be used for pg_dump comparison, and that will keep template1 with > the same permissions, so I guess it is okay. It's an emergent property of two decisions. In the interest of backward compatibility, I decided to have v15 pg_dump emit GRANT for the public schema even when the source is an unmodified v14- database. When that combines with the ancient decision that a pg_dumpall or pg_upgrade covers template1 but not template0, one gets the above consequences. I don't see a way to improve on this outcome. > > > permissions on the <literal>public</literal> schema has not > > > been changed. Databases restored from previous Postgres releases > > > will be restored with their current permissions. Users wishing > > > to have the old permissions on new objects will need to grant > > > > The phrase "old permissions on new objects" doesn't sound right to me, but I'm > > not sure why. I think you're aiming for the fact that this is just a default; > > one can still change the ACL to anything, including to the old default. If > > these notes are going to mention the old default like they do so far, I think > > they should also urge readers to understand > > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > > before returning to the old default. What do you think? > > Agreed, the new text is: > > Users wishing to have the former permissions will need to grant > <literal>CREATE</literal> permission for <literal>PUBLIC</literal> on > the <literal>public</literal> schema; this change can be made on > <literal>template1</literal> to cause all new databases to have these > permissions. What do you think about the "should also urge readers ..." part of my message?
On Tue, Jun 28, 2022 at 05:32:26PM -0700, Peter Geoghegan wrote: > The main enhancement to VACUUM for Postgres 15 was item 1, which > taught VACUUM to dynamically track the oldest remaining XID (and the > oldest remaining MXID) that will remain in the table at the end of the > same VACUUM operation. These final/oldest XID/MXID values are what we > now use to set relfrozenxid and relminmxid in pg_class. Previously we > just set relfrozenxid/relminmxid to whatever XID/MXID value was used > to determine which XIDs/MXIDs needed to be frozen. These values often > indicated more about VACUUM implementation details (like the > vacuum_freeze_min_age GUc's value) than the actual true contents of the > table at the end of the most recent VACUUM. > > It might be worth explaining the shift directly in the release notes. > The new approach is simpler and makes a lot more sense -- why should > the relfrozenxid be closely tied to freezing? We don't necessarily I don't think this is an appropriate detail for the release notes. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Fri, Jul 1, 2022 at 9:41 AM Bruce Momjian <bruce@momjian.us> wrote: > > It might be worth explaining the shift directly in the release notes. > > The new approach is simpler and makes a lot more sense -- why should > > the relfrozenxid be closely tied to freezing? We don't necessarily > > I don't think this is an appropriate detail for the release notes. Okay. What about saying something about relminmxid advancement where the database consumes lots of multixacts? -- Peter Geoghegan
On Wed, Jun 29, 2022 at 10:08:08PM -0700, Noah Misch wrote: > On Tue, Jun 28, 2022 at 04:35:45PM -0400, Bruce Momjian wrote: > > > If you dump/reload an unmodified v14 template1 (as pg_dumpall and pg_upgrade > > > do), your v15 template1 will have a v14 ACL on its public schema. At that > > > point, the fate of "newly-created databases in existing clusters" depends on > > > whether you clone template1 or template0. Does any of that detail belong > > > here, or does the existing text suffice? > > > > I think it is very confusing to have template0 have one value and > > template1 have a different one, but as I understand it template0 will > > only be used for pg_dump comparison, and that will keep template1 with > > the same permissions, so I guess it is okay. > > It's an emergent property of two decisions. In the interest of backward > compatibility, I decided to have v15 pg_dump emit GRANT for the public schema > even when the source is an unmodified v14- database. When that combines with > the ancient decision that a pg_dumpall or pg_upgrade covers template1 but not > template0, one gets the above consequences. I don't see a way to improve on > this outcome. Thanks for the summary. > > > > permissions on the <literal>public</literal> schema has not > > > > been changed. Databases restored from previous Postgres releases > > > > will be restored with their current permissions. Users wishing > > > > to have the old permissions on new objects will need to grant > > > > > > The phrase "old permissions on new objects" doesn't sound right to me, but I'm > > > not sure why. I think you're aiming for the fact that this is just a default; > > > one can still change the ACL to anything, including to the old default. If > > > these notes are going to mention the old default like they do so far, I think > > > they should also urge readers to understand > > > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > > > before returning to the old default. What do you think? > > > > Agreed, the new text is: > > > > Users wishing to have the former permissions will need to grant > > <literal>CREATE</literal> permission for <literal>PUBLIC</literal> on > > the <literal>public</literal> schema; this change can be made on > > <literal>template1</literal> to cause all new databases to have these > > permissions. > > What do you think about the "should also urge readers ..." part of my message? I see your point, that there is no indication of why you might not want to restore the old permissions. I created the attached patch which makes two additions to clarify this. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Attachment
On Fri, Jul 01, 2022 at 02:08:00PM -0400, Bruce Momjian wrote: > On Wed, Jun 29, 2022 at 10:08:08PM -0700, Noah Misch wrote: > > On Tue, Jun 28, 2022 at 04:35:45PM -0400, Bruce Momjian wrote: > > > > > permissions on the <literal>public</literal> schema has not > > > > > been changed. Databases restored from previous Postgres releases > > > > > will be restored with their current permissions. Users wishing > > > > > to have the old permissions on new objects will need to grant > > > > > > > > The phrase "old permissions on new objects" doesn't sound right to me, but I'm > > > > not sure why. I think you're aiming for the fact that this is just a default; > > > > one can still change the ACL to anything, including to the old default. If > > > > these notes are going to mention the old default like they do so far, I think > > > > they should also urge readers to understand > > > > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > > > > before returning to the old default. What do you think? > > > > > > Agreed, the new text is: > > > > > > Users wishing to have the former permissions will need to grant > > > <literal>CREATE</literal> permission for <literal>PUBLIC</literal> on > > > the <literal>public</literal> schema; this change can be made on > > > <literal>template1</literal> to cause all new databases to have these > > > permissions. > > > > What do you think about the "should also urge readers ..." part of my message? > > I see your point, that there is no indication of why you might not want > to restore the old permissions. I created the attached patch which > makes two additions to clarify this. > --- a/doc/src/sgml/release-15.sgml > +++ b/doc/src/sgml/release-15.sgml > @@ -63,12 +63,11 @@ Author: Noah Misch <noah@leadboat.com> > permissions on the <literal>public</literal> schema has not > been changed. Databases restored from previous Postgres releases > will be restored with their current permissions. Users wishing > - to have the former more-open permissions will need to grant > + to have the former permissions will need to grant > <literal>CREATE</literal> permission for <literal>PUBLIC</literal> > on the <literal>public</literal> schema; this change can be made > on <literal>template1</literal> to cause all new databases > - to have these permissions. This change was made to increase > - security. > + to have these permissions. > </para> > </listitem> Here's what I've been trying to ask: what do you think of linking to https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS here? The release note text is still vague, and the docs have extensive coverage of the topic. The notes can just link to that extensive coverage.
On Fri, Jul 1, 2022 at 09:56:17AM -0700, Peter Geoghegan wrote: > On Fri, Jul 1, 2022 at 9:41 AM Bruce Momjian <bruce@momjian.us> wrote: > > > It might be worth explaining the shift directly in the release notes. > > > The new approach is simpler and makes a lot more sense -- why should > > > the relfrozenxid be closely tied to freezing? We don't necessarily > > > > I don't think this is an appropriate detail for the release notes. > > Okay. What about saying something about relminmxid advancement where > the database consumes lots of multixacts? No. same issue. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Okay, thanks Bruce.
Peter Geoghegan
On Sat, Jul 2, 2022 at 08:13:41PM -0400, Bruce Momjian wrote: > On Fri, Jul 1, 2022 at 09:56:17AM -0700, Peter Geoghegan wrote: > > On Fri, Jul 1, 2022 at 9:41 AM Bruce Momjian <bruce@momjian.us> wrote: > > > > It might be worth explaining the shift directly in the release notes. > > > > The new approach is simpler and makes a lot more sense -- why should > > > > the relfrozenxid be closely tied to freezing? We don't necessarily > > > > > > I don't think this is an appropriate detail for the release notes. > > > > Okay. What about saying something about relminmxid advancement where > > the database consumes lots of multixacts? > > No. same issue. Actually, I was wrong. I thought that we only mentioned that we computed a more agressive xid, but now see I was mentioning the _frozen_ xid. Reading the commit, we do compute the multi-xid and store that too so I have updated the PG 15 release notes with the attached patch. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Attachment
On Fri, Jul 1, 2022 at 06:21:28PM -0700, Noah Misch wrote: > Here's what I've been trying to ask: what do you think of linking to > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > here? The release note text is still vague, and the docs have extensive > coverage of the topic. The notes can just link to that extensive coverage. Sure. how is this patch? -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Attachment
On Tue, Jul 5, 2022 at 11:09 AM Bruce Momjian <bruce@momjian.us> wrote: > > Actually, I was wrong. I thought that we only mentioned that we > computed a more agressive xid, but now see I was mentioning the _frozen_ > xid. Reading the commit, we do compute the multi-xid and store that too > so I have updated the PG 15 release notes with the attached patch. It might be worth using the "symbol names" directly, since they appear in the documentation already (under "Routine Vacuuming"). These are <structfield>relfrozenxid</structfield> and <structfield>relminmxid</structfield>. These are implementation details, but they're documented in detail (though admittedly the documentation has *lots* of problems). Here is what I would like this item to hint at, to advanced users with tricky requirements: The new approach to setting relminmxid will improve the behavior of VACUUM in databases that already happen to use lots of MultiXacts. These users will notice that autovacuum now works off of relminmxid values that actually tell us something about each table's consumption of MultiXacts over time. Most individual tables naturally consume *zero* MultiXacts, even in databases that consume many MultiXacts -- due to naturally occuring workload characteristics. The old approach failed to recognize this, leading to very uniform relminmxid values across tables that were in fact very different, MultiXact-wise. The way that we handle relfrozenxid is probably much less likely to make life much easier for any database, at least on its own, in Postgres 15. So from the point of view of a user considering upgrading, the impact on relminmxid is likely to be far more important. Admittedly the most likely scenario by far is that the whole feature just isn't interesting, but a small minority of advanced users (users with painful MultiXact problems) will find the relminmxid thing very compelling. -- Peter Geoghegan
On Tue, Jul 05, 2022 at 02:35:39PM -0400, Bruce Momjian wrote: > On Fri, Jul 1, 2022 at 06:21:28PM -0700, Noah Misch wrote: > > Here's what I've been trying to ask: what do you think of linking to > > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > > here? The release note text is still vague, and the docs have extensive > > coverage of the topic. The notes can just link to that extensive coverage. > > Sure. how is this patch? > --- a/doc/src/sgml/release-15.sgml > +++ b/doc/src/sgml/release-15.sgml > @@ -63,11 +63,12 @@ Author: Noah Misch <noah@leadboat.com> > permissions on the <literal>public</literal> schema has not > been changed. Databases restored from previous Postgres releases > will be restored with their current permissions. Users wishing > - to have the former permissions will need to grant > + to have the former more-open permissions will need to grant > <literal>CREATE</literal> permission for <literal>PUBLIC</literal> > on the <literal>public</literal> schema; this change can be made > on <literal>template1</literal> to cause all new databases > - to have these permissions. > + to have these permissions. This change was made to increase > + security; see <xref linkend="ddl-schemas-patterns"/>. > </para> > </listitem> I think this still puts undue weight on single-user systems moving back to the old default. The linked documentation does say how to get back to v14 permissions (and disclaims security if you do so), so let's not mention it here. The attached is how I would write it. I also reworked the "Databases restored from previous ..." sentence, since its statement is also true of databases restored v15-to-v15 (no "previous" release involved). I also moved the bit about USAGE to end, since it's just emphasizing what the reader should already assume. Any concerns?
Attachment
On Tue, Jul 5, 2022 at 11:51:31AM -0700, Peter Geoghegan wrote: > On Tue, Jul 5, 2022 at 11:09 AM Bruce Momjian <bruce@momjian.us> wrote: > > > > Actually, I was wrong. I thought that we only mentioned that we > > computed a more agressive xid, but now see I was mentioning the _frozen_ > > xid. Reading the commit, we do compute the multi-xid and store that too > > so I have updated the PG 15 release notes with the attached patch. > > It might be worth using the "symbol names" directly, since they appear > in the documentation already (under "Routine Vacuuming"). These are > <structfield>relfrozenxid</structfield> and > <structfield>relminmxid</structfield>. These are implementation > details, but they're documented in detail (though admittedly the > documentation has *lots* of problems). Well, users can look into the details if they wish. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Tue, Jul 5, 2022 at 12:53:49PM -0700, Noah Misch wrote: > On Tue, Jul 05, 2022 at 02:35:39PM -0400, Bruce Momjian wrote: > > On Fri, Jul 1, 2022 at 06:21:28PM -0700, Noah Misch wrote: > > > Here's what I've been trying to ask: what do you think of linking to > > > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > > > here? The release note text is still vague, and the docs have extensive > > > coverage of the topic. The notes can just link to that extensive coverage. > > > > Sure. how is this patch? > > > --- a/doc/src/sgml/release-15.sgml > > +++ b/doc/src/sgml/release-15.sgml > > @@ -63,11 +63,12 @@ Author: Noah Misch <noah@leadboat.com> > > permissions on the <literal>public</literal> schema has not > > been changed. Databases restored from previous Postgres releases > > will be restored with their current permissions. Users wishing > > - to have the former permissions will need to grant > > + to have the former more-open permissions will need to grant > > <literal>CREATE</literal> permission for <literal>PUBLIC</literal> > > on the <literal>public</literal> schema; this change can be made > > on <literal>template1</literal> to cause all new databases > > - to have these permissions. > > + to have these permissions. This change was made to increase > > + security; see <xref linkend="ddl-schemas-patterns"/>. > > </para> > > </listitem> > > I think this still puts undue weight on single-user systems moving back to the > old default. The linked documentation does say how to get back to v14 > permissions (and disclaims security if you do so), so let's not mention it > here. The attached is how I would write it. I also reworked the "Databases > restored from previous ..." sentence, since its statement is also true of > databases restored v15-to-v15 (no "previous" release involved). I also moved > the bit about USAGE to end, since it's just emphasizing what the reader should > already assume. Any concerns? I see where you are going --- to talk about how to convert upgraded clusters to secure clusters, rather than how to revert to the previous behavior. I assumed that the most common question would be how to get the previous behavior, rather than how to get the new behavior in upgraded clusters. However, I am fine with what you think is best. My only stylistic suggestion would be to remove "a" from "a <literal>REVOKE</literal>". -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Tue, Jul 05, 2022 at 04:35:32PM -0400, Bruce Momjian wrote: > On Tue, Jul 5, 2022 at 12:53:49PM -0700, Noah Misch wrote: > > On Tue, Jul 05, 2022 at 02:35:39PM -0400, Bruce Momjian wrote: > > > On Fri, Jul 1, 2022 at 06:21:28PM -0700, Noah Misch wrote: > > > > Here's what I've been trying to ask: what do you think of linking to > > > > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > > > > here? The release note text is still vague, and the docs have extensive > > > > coverage of the topic. The notes can just link to that extensive coverage. > > > > > > Sure. how is this patch? > > > > > --- a/doc/src/sgml/release-15.sgml > > > +++ b/doc/src/sgml/release-15.sgml > > > @@ -63,11 +63,12 @@ Author: Noah Misch <noah@leadboat.com> > > > permissions on the <literal>public</literal> schema has not > > > been changed. Databases restored from previous Postgres releases > > > will be restored with their current permissions. Users wishing > > > - to have the former permissions will need to grant > > > + to have the former more-open permissions will need to grant > > > <literal>CREATE</literal> permission for <literal>PUBLIC</literal> > > > on the <literal>public</literal> schema; this change can be made > > > on <literal>template1</literal> to cause all new databases > > > - to have these permissions. > > > + to have these permissions. This change was made to increase > > > + security; see <xref linkend="ddl-schemas-patterns"/>. > > > </para> > > > </listitem> > > > > I think this still puts undue weight on single-user systems moving back to the > > old default. The linked documentation does say how to get back to v14 > > permissions (and disclaims security if you do so), so let's not mention it > > here. The attached is how I would write it. I also reworked the "Databases > > restored from previous ..." sentence, since its statement is also true of > > databases restored v15-to-v15 (no "previous" release involved). I also moved > > the bit about USAGE to end, since it's just emphasizing what the reader should > > already assume. Any concerns? > > I see where you are going --- to talk about how to convert upgraded > clusters to secure clusters, rather than how to revert to the previous > behavior. I assumed that the most common question would be how to get > the previous behavior, rather than how to get the new behavior in > upgraded clusters. However, I am fine with what you think is best. Since having too-permissive ACLs is usually symptom-free, I share your forecast about the more-common question. Expect questions on mailing lists, stackoverflow, etc. The right way to answer those questions is roughly this: > On PostgreSQL 15, my application gets "permission denied for schema > public". What should I do? You have a choice to make. The best selection depends on the security needs of your database. See https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS for a guide to making that choice. Recommending GRANT to that two-sentence question would be negligent. One should know a database's lack of security needs before recommending GRANT. This is a key opportunity to have more users make the right decision while their attention is on the topic. > My only stylistic suggestion would be to remove "a" from "a > <literal>REVOKE</literal>". I'll plan to push with that change.
On Tue, Jul 5, 2022 at 02:57:52PM -0700, Noah Misch wrote: > Since having too-permissive ACLs is usually symptom-free, I share your > forecast about the more-common question. Expect questions on mailing lists, > stackoverflow, etc. The right way to answer those questions is roughly this: > > > On PostgreSQL 15, my application gets "permission denied for schema > > public". What should I do? > > You have a choice to make. The best selection depends on the security > needs of your database. See > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > for a guide to making that choice. > > Recommending GRANT to that two-sentence question would be negligent. One > should know a database's lack of security needs before recommending GRANT. > This is a key opportunity to have more users make the right decision while > their attention is on the topic. Yes, I think it is a question of practicality vs. desirability. We are basically telling people they have to do research to get the old behavior in their new databases and clusters. > > My only stylistic suggestion would be to remove "a" from "a > > <literal>REVOKE</literal>". > > I'll plan to push with that change. WFM. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Tue, Jul 05, 2022 at 07:47:52PM -0400, Bruce Momjian wrote: > On Tue, Jul 5, 2022 at 02:57:52PM -0700, Noah Misch wrote: > > Since having too-permissive ACLs is usually symptom-free, I share your > > forecast about the more-common question. Expect questions on mailing lists, > > stackoverflow, etc. The right way to answer those questions is roughly this: > > > > > On PostgreSQL 15, my application gets "permission denied for schema > > > public". What should I do? > > > > You have a choice to make. The best selection depends on the security > > needs of your database. See > > https://www.postgresql.org/docs/devel/ddl-schemas.html#DDL-SCHEMAS-PATTERNS > > for a guide to making that choice. > > > > Recommending GRANT to that two-sentence question would be negligent. One > > should know a database's lack of security needs before recommending GRANT. > > This is a key opportunity to have more users make the right decision while > > their attention is on the topic. > > Yes, I think it is a question of practicality vs. desirability. We are > basically telling people they have to do research to get the old > behavior in their new databases and clusters. True. I want to maximize the experience for different classes of database: 1. Databases needing user isolation and unknowingly not getting it. 2. Databases not needing user isolation, e.g. automated test environments. Expecting all of these DBAs to read a 500-word doc section is failure-prone. For the benefit of (2), I'm now thinking about adding a release note sentence, "For a new database having zero need to defend against insider threats, granting back the privilege yields the PostgreSQL 14 behavior."
On Tue, Jul 5, 2022 at 07:45:57PM -0700, Noah Misch wrote:
> On Tue, Jul 05, 2022 at 07:47:52PM -0400, Bruce Momjian wrote:
> > Yes, I think it is a question of practicality vs. desirability. We are
> > basically telling people they have to do research to get the old
> > behavior in their new databases and clusters.
>
> True. I want to maximize the experience for different classes of database:
>
> 1. Databases needing user isolation and unknowingly not getting it.
> 2. Databases not needing user isolation, e.g. automated test environments.
>
> Expecting all of these DBAs to read a 500-word doc section is failure-prone.
> For the benefit of (2), I'm now thinking about adding a release note sentence,
> "For a new database having zero need to defend against insider threats,
> granting back the privilege yields the PostgreSQL 14 behavior."
I think you would need to say "previous behavior" since people might be
upgrading from releases before PG 14. I also would change "In existing
databases" to "For existing databases". I think your big risk here is
trying to explain how to have new clusters get the old or new behavior
in the same text block, e.g.:
The new default is one of the secure schema usage patterns that
<xref linkend="ddl-schemas-patterns"/> has recommended since the
security release for CVE-2018-1058. Upgrading a cluster or restoring a
database dump will preserve existing permissions. This is a change in
the default for newly-created databases in existing clusters and for new
clusters. For existing databases, especially those having multiple
users, consider issuing a <literal>REVOKE</literal> to adopt this new
default. (<literal>USAGE</literal> permission on this schema has not
changed.) For a new database having zero need to defend against insider
threats, granting back the privilege yields the previous behavior.
Is this something we want to get into in the release notes, or perhaps
do we need to link to a wiki page for these details?
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
On Wed, Jul 06, 2022 at 09:10:53AM -0400, Bruce Momjian wrote: > On Tue, Jul 5, 2022 at 07:45:57PM -0700, Noah Misch wrote: > > On Tue, Jul 05, 2022 at 07:47:52PM -0400, Bruce Momjian wrote: > > > Yes, I think it is a question of practicality vs. desirability. We are > > > basically telling people they have to do research to get the old > > > behavior in their new databases and clusters. > > > > True. I want to maximize the experience for different classes of database: > > > > 1. Databases needing user isolation and unknowingly not getting it. > > 2. Databases not needing user isolation, e.g. automated test environments. > > > > Expecting all of these DBAs to read a 500-word doc section is failure-prone. > > For the benefit of (2), I'm now thinking about adding a release note sentence, > > "For a new database having zero need to defend against insider threats, > > granting back the privilege yields the PostgreSQL 14 behavior." > > I think you would need to say "previous behavior" since people might be > upgrading from releases before PG 14. I also would change "In existing I felt "previous behavior" was mildly ambiguous. I've changed it to "the behavior of prior releases". > databases" to "For existing databases". I think your big risk here is Done. New version attached. > trying to explain how to have new clusters get the old or new behavior > in the same text block, e.g.: > > The new default is one of the secure schema usage patterns that > <xref linkend="ddl-schemas-patterns"/> has recommended since the > security release for CVE-2018-1058. Upgrading a cluster or restoring a > database dump will preserve existing permissions. This is a change in > the default for newly-created databases in existing clusters and for new > clusters. For existing databases, especially those having multiple > users, consider issuing a <literal>REVOKE</literal> to adopt this new > default. (<literal>USAGE</literal> permission on this schema has not > changed.) For a new database having zero need to defend against insider > threats, granting back the privilege yields the previous behavior. > > Is this something we want to get into in the release notes, or perhaps > do we need to link to a wiki page for these details? No supported release has a wiki page link in its release notes. We used wiki pages in the more-distant past, but I don't recall why. I am not aware of wiki pages having relevant benefits.
Attachment
On Sat, Jul 9, 2022 at 08:19:41PM -0700, Noah Misch wrote:
> > I think you would need to say "previous behavior" since people might be
> > upgrading from releases before PG 14. I also would change "In existing
>
> I felt "previous behavior" was mildly ambiguous. I've changed it to "the
> behavior of prior releases".
Sure.
>
> > databases" to "For existing databases". I think your big risk here is
>
> Done. New version attached.
I had trouble reading the sentences in the order you used so I
restructured it:
The new default is one of the secure schema usage patterns that <xref
linkend="ddl-schemas-patterns"/> has recommended since the security
release for CVE-2018-1058. The change applies to newly-created
databases in existing clusters and for new clusters. Upgrading a
cluster or restoring a database dump will preserve existing permissions.
For existing databases, especially those having multiple users, consider
issuing <literal>REVOKE</literal> to adopt this new default. For new
databases having zero need to defend against insider threats, granting
<literal>USAGE</literal> permission on their <literal>public</literal>
schemas will yield the behavior of prior releases.
> > Is this something we want to get into in the release notes, or perhaps
> > do we need to link to a wiki page for these details?
>
> No supported release has a wiki page link in its release notes. We used wiki
> pages in the more-distant past, but I don't recall why. I am not aware of
> wiki pages having relevant benefits.
I think the wiki was good if you needed a lot of release-specific text,
or if you wanted to adjust the wording after the release.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
On Tue, May 10, 2022 at 11:41:08PM -0400, Bruce Momjian wrote: > On Tue, May 10, 2022 at 08:31:17PM -0500, Justin Pryzby wrote: > > | Store server-level statistics in shared memory (Kyotaro Horiguchi, Andres Freund, Melanie Plageman) > > > > Should this be called "cumulative" statistics? As in b3abca68106d518ce5d3c0d9a1e0ec02a647ceda. > > Uh, they are counters, which I guess is cummulative, but that doesn't > seem very descriptive. The documentation call it the statistics > collector, but I am not sure we even have that anymore with an in-memory > implementation. I am kind of not sure what to call it. What I was trying to say is that it's now called the cumulative stats system. > > | New function > > > > "The new function .." (a few times) > > Uh, I only see it once. There's still a couple of these without "The". > > Should any of these be listed as incompatible changes (some of these I asked > > before, but the others are from another list). > > ccd10a9bfa5 Fix enforcement of PL/pgSQL variable CONSTANT markings (Tom Lane) > > I didn't see not enforcing constant as an incompatibility, but rather a > bug. Yes it's a bug, but it's going to be seen as a compatibility issue for someone whose application breaks. The same goes for other things I mentioned. > > 376ce3e404b Prefer $HOME when looking up the current user's home directory. > > Uh, I didn't think so. > > > 7844c9918a4 psql: Show all query results by default > > Same. > > > 17a856d08be Change aggregated log format of pgbench. > > We have a pgbench section and I can't see it. I am trying to keep > incompatiblities as things related to in-production problems or > surprises. > > > ? 73508475d69 Remove pg_atoi() > > I don't see who would care except for internals folks. > > > ? aa64f23b029 Remove MaxBackends variable in favor of GetMaxBackends() function. > > Same. > > > ? d816f366bc4 psql: Make SSL info display more compact > > I did look at that but considered that this wouldn't be something that > would break anything. > > > ? 27b02e070fd pg_upgrade: Don't print progress status when output is not a tty. > > Same. > > > ? ab4fd4f868e Remove 'datlastsysoid'. > > Seemed too internal. FYI, removal of this column broke a tool one of my coworkers uses (navicat). I'm told that the fix will be in navicat v16.1 (but their existing users will need to pay to upgrade from v15). -- Justin
On Mon, Jul 11, 2022 at 12:39:23PM -0500, Justin Pryzby wrote: > On Tue, May 10, 2022 at 11:41:08PM -0400, Bruce Momjian wrote: > > On Tue, May 10, 2022 at 08:31:17PM -0500, Justin Pryzby wrote: > > > > | Store server-level statistics in shared memory (Kyotaro Horiguchi, Andres Freund, Melanie Plageman) > > > > > > Should this be called "cumulative" statistics? As in b3abca68106d518ce5d3c0d9a1e0ec02a647ceda. > > > > Uh, they are counters, which I guess is cummulative, but that doesn't > > seem very descriptive. The documentation call it the statistics > > collector, but I am not sure we even have that anymore with an in-memory > > implementation. I am kind of not sure what to call it. > > What I was trying to say is that it's now called the cumulative stats system. It is actually called the "cumulative statistics system", so updated; patch attached and applied. > > > | New function > > > > > > "The new function .." (a few times) > > > > Uh, I only see it once. > > There's still a couple of these without "The". Ah, found them, fixed. > > > Should any of these be listed as incompatible changes (some of these I asked > > > before, but the others are from another list). > > > > ccd10a9bfa5 Fix enforcement of PL/pgSQL variable CONSTANT markings (Tom Lane) > > > > I didn't see not enforcing constant as an incompatibility, but rather a > > bug. > > Yes it's a bug, but it's going to be seen as a compatibility issue for someone > whose application breaks. The same goes for other things I mentioned. We don't guarantee that the only breakage is listed in the incompatibilities section, only the most common ones. > > > 376ce3e404b Prefer $HOME when looking up the current user's home directory. > > > > Uh, I didn't think so. > > > > > 7844c9918a4 psql: Show all query results by default > > > > Same. > > > > > 17a856d08be Change aggregated log format of pgbench. > > > > We have a pgbench section and I can't see it. I am trying to keep > > incompatiblities as things related to in-production problems or > > surprises. > > > > > ? 73508475d69 Remove pg_atoi() > > > > I don't see who would care except for internals folks. > > > > > ? aa64f23b029 Remove MaxBackends variable in favor of GetMaxBackends() function. > > > > Same. > > > > > ? d816f366bc4 psql: Make SSL info display more compact > > > > I did look at that but considered that this wouldn't be something that > > would break anything. > > > > > ? 27b02e070fd pg_upgrade: Don't print progress status when output is not a tty. > > > > Same. > > > > > ? ab4fd4f868e Remove 'datlastsysoid'. > > > > Seemed too internal. > > FYI, removal of this column broke a tool one of my coworkers uses (navicat). > I'm told that the fix will be in navicat v16.1 (but their existing users will > need to pay to upgrade from v15). This actually supports my point --- only navicat needs to know about this renaming, it its users. Telling navicat users about this change does not help them. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Attachment
On Mon, Jul 11, 2022 at 12:39:57PM -0400, Bruce Momjian wrote: > I had trouble reading the sentences in the order you used so I > restructured it: > > The new default is one of the secure schema usage patterns that <xref > linkend="ddl-schemas-patterns"/> has recommended since the security > release for CVE-2018-1058. The change applies to newly-created > databases in existing clusters and for new clusters. Upgrading a > cluster or restoring a database dump will preserve existing permissions. I agree with the sentence order change. > For existing databases, especially those having multiple users, consider > issuing <literal>REVOKE</literal> to adopt this new default. For new > databases having zero need to defend against insider threats, granting > <literal>USAGE</literal> permission on their <literal>public</literal> > schemas will yield the behavior of prior releases. s/USAGE/CREATE/ in the last sentence. Looks good with that change.
On Mon, Jul 11, 2022 at 11:31:32PM -0700, Noah Misch wrote: > On Mon, Jul 11, 2022 at 12:39:57PM -0400, Bruce Momjian wrote: > > I had trouble reading the sentences in the order you used so I > > restructured it: > > > > The new default is one of the secure schema usage patterns that <xref > > linkend="ddl-schemas-patterns"/> has recommended since the security > > release for CVE-2018-1058. The change applies to newly-created > > databases in existing clusters and for new clusters. Upgrading a > > cluster or restoring a database dump will preserve existing permissions. > > I agree with the sentence order change. Great. > > For existing databases, especially those having multiple users, consider > > issuing <literal>REVOKE</literal> to adopt this new default. For new > > databases having zero need to defend against insider threats, granting > > <literal>USAGE</literal> permission on their <literal>public</literal> > > schemas will yield the behavior of prior releases. > > s/USAGE/CREATE/ in the last sentence. Looks good with that change. Ah, yes, of course. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Regarding this item:
Previously the code always sequentially scanned the list of values."
The todo list has an entry titled "Planning large IN lists", which links to
Did we already have a hash lookup for IN clauses with constants and the above commit adds NOT IN? If so, maybe we have enough to remove this todo item.
On Thu, Jul 14, 2022 at 01:24:41PM +0700, John Naylor wrote: > Regarding this item: > > "Allow hash lookup for NOT IN clauses with many constants (David Rowley, James > Coleman) > Previously the code always sequentially scanned the list of values." > > The todo list has an entry titled "Planning large IN lists", which links to > > https://www.postgresql.org/message-id/1178821226.6034.63.camel@goldbach > > Did we already have a hash lookup for IN clauses with constants and the above > commit adds NOT IN? If so, maybe we have enough to remove this todo item. Agreed, I have removed it now. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
> Increase hash_mem_multiplier default to 2.0 (Peter Geoghegan) > This allows query hash operations to use double the amount of work_mem memory as other operations. I wonder if it's worth pointing out that a query may end up using not just 2x more memory (since work_mem*hash_mem_multiplier is per node), but 2**N more, for N nodes. > Remove pg_dump's --no-synchronized-snapshots option since all supported server versions support synchronized snapshots(Tom Lane) It'd be better to put that after the note about dropping support for upgrading clusters older than v9.2 in psql/pg_dump/pg_upgrade. > Enable system and TOAST btree indexes to efficiently store duplicates (Peter Geoghegan) Say "btree indexes on system [and TOAST] tables" > Prevent changes to columns only indexed by BRIN indexes from disabling HOT updates (Josef Simanek) This was reverted > Generate periodic log message during slow server starts (Nitin Jadhav, Robert Haas) messages plural > Messages report the cause of the delay. The time interval for notification is controlled by the new server variable log_startup_progress_interval. *The messages > Add server variable shared_memory_size to report the size of allocated shared memory (Nathan Bossart) > Add server variable shared_memory_size_in_huge_pages to report the number of huge memory pages required (Nathan Bossart) Maybe these should say server *parameter* since they're not really "variable". > 0. Add support for LZ4 and Zstandard compression of server-side base backups (Jeevan Ladhe, Robert Haas) > 1. Allow pg_basebackup to use LZ4 and Zstandard compression on server-side base backup files (Dipesh Pandit, Jeevan Ladhe) > 2. Allow pg_basebackup's --compress option to control the compression method and options (Michael Paquier, Robert Haas) > New options include server-gzip (gzip on the server), client-gzip (same as gzip). > 3. Allow pg_basebackup to compress on the server side and decompress on the client side before storage (Dipesh Pandit) > This is accomplished by specifying compression on the server side and plain output format. I still think these expose the incremental development rather than the user-facing change. 1. It seems wrong to say "server-side" since client-side compression with LZ4/zstd is also supported. 2. It's confusing to say that the new options are server-gzip and client-gzip, since it just mentioned new algorithms; 3. I'm not sure this needs to be mentioned at all; maybe it should be a "detail" following the item about server-side compression. > Tables added to the listed schemas in the future will also be replicated. "Tables later added" is clearer. Otherwise "in the future" sounds like maybe in v16 or v17 we'll start replicating those tables. > Allow subscribers to stop logical replication application on error (Osumi Takamichi, Mark Dilger) "application" sounds off. > Add new default WAL-logged method for database creation (Dilip Kumar) "New default" sounds off. Say "Add new WAL-logged method for database creation, used by default". > Have pg_upgrade preserve relfilenodes, tablespace, and database OIDs between old and new clusters (Shruthi KC, AntoninHouska) "tablespace OIDs" or "tablespace and database OIDs and relfilenodes" > Limit support of pg_upgrade to old servers running PostgreSQL 9.2 and later (Tom Lane) The word "old" doesn't appear in the 2 release notes items about pg_dump and psql, and "old" makes it sound sounds like "antique" rather than "source". > Some internal-use-only types have also been assigned this column. this *value > Allow custom scan provders to indicate if they support projections (Sven Klemm) > The default is now that custom scan providers can't support projections, so they need to be updated for this release. Per the commit message, they don't "need" to be updated. I think this should say "The default now assumes that a custom scan provider does not support projections; to retain optimal performance, they should be updated to indicate whether that's supported.
On Mon, Jul 18, 2022 at 08:23:23PM -0500, Justin Pryzby wrote:
> > Increase hash_mem_multiplier default to 2.0 (Peter Geoghegan)
> > This allows query hash operations to use double the amount of work_mem memory as other operations.
>
> I wonder if it's worth pointing out that a query may end up using not just 2x
> more memory (since work_mem*hash_mem_multiplier is per node), but 2**N more,
> for N nodes.
Uh, I said "per operation" so people might realize there can be multiple
work_mem memory operations per query. I don't think I can do more in
this text. I can't think of a way to improve it without making it more
confusing.
> > Remove pg_dump's --no-synchronized-snapshots option since all supported server versions support synchronized
snapshots(Tom Lane)
>
> It'd be better to put that after the note about dropping support for upgrading
> clusters older than v9.2 in psql/pg_dump/pg_upgrade.
Well, I put the --no-synchronized-snapshots item in incompatibilities
since it is a user-visible change that might require script adjustments.
However, I put the limit of pg_dump to 9.2 and greater into the pg_dump
section. Are you suggesting I move the--no-synchronized-snapshots item
down there? That doesn't match with the way I have listed other
incompatibilities so I am resistant to do that.
> > Enable system and TOAST btree indexes to efficiently store duplicates (Peter Geoghegan)
>
> Say "btree indexes on system [and TOAST] tables"
Okay, updated to:
Allow btree indexes on system and TOAST tables to efficiently
store duplicates (Peter Geoghegan)
> > Prevent changes to columns only indexed by BRIN indexes from disabling HOT updates (Josef Simanek)
>
> This was reverted
Ah, yes, removed.
> > Generate periodic log message during slow server starts (Nitin Jadhav, Robert Haas)
> messages plural
>
> > Messages report the cause of the delay. The time interval for notification is controlled by the new server variable
log_startup_progress_interval.
> *The messages
Ah, yes, fixed.
> > Add server variable shared_memory_size to report the size of allocated shared memory (Nathan Bossart)
> > Add server variable shared_memory_size_in_huge_pages to report the number of huge memory pages required (Nathan
Bossart)
>
> Maybe these should say server *parameter* since they're not really "variable".
Uh, I think of parameters as something passed. We do call them server
variables, or read-only server variables. I can add "read-only" but it
seems odd.
> > 0. Add support for LZ4 and Zstandard compression of server-side base backups (Jeevan Ladhe, Robert Haas)
> > 1. Allow pg_basebackup to use LZ4 and Zstandard compression on server-side base backup files (Dipesh Pandit, Jeevan
Ladhe)
> > 2. Allow pg_basebackup's --compress option to control the compression method and options (Michael Paquier, Robert
Haas)
> > New options include server-gzip (gzip on the server), client-gzip (same as gzip).
> > 3. Allow pg_basebackup to compress on the server side and decompress on the client side before storage (Dipesh
Pandit)
> > This is accomplished by specifying compression on the server side and plain output format.
>
> I still think these expose the incremental development rather than the
> user-facing change.
Well, they are in different parts of the system, though they are clearly
all related. I am afraid merging them would be even more confusing.
> 1. It seems wrong to say "server-side" since client-side compression with
> LZ4/zstd is also supported.
Agreed. I changed it to:
Allow pg_basebackup to do LZ4 and Zstandard server-side compression
on base backup files (Dipesh Pandit, Jeevan Ladhe)
> 2. It's confusing to say that the new options are server-gzip and client-gzip,
> since it just mentioned new algorithms;
I see your point since there will be new options for LZ4 and Zstandard
too, so I just removed that paragraph.
> 3. I'm not sure this needs to be mentioned at all; maybe it should be a
> "detail" following the item about server-side compression.
See my concerns above --- it seems too complex to merge into something
else. However, I am open to an entire rewrite of these items.
> > Tables added to the listed schemas in the future will also be replicated.
>
> "Tables later added" is clearer. Otherwise "in the future" sounds like maybe
> in v16 or v17 we'll start replicating those tables.
Agreed, new wording:
Tables added later to the listed schemas will also be replicated.
> > Allow subscribers to stop logical replication application on error (Osumi Takamichi, Mark Dilger)
> "application" sounds off.
Agreed. New text is:
Allow subscribers to stop the application of logical replication
changes on error
> > Add new default WAL-logged method for database creation (Dilip Kumar)
> "New default" sounds off. Say "Add new WAL-logged method for database creation, used by default".
Agreed, new text:
Add new <acronym>WAL</acronym>-logged method for <link
linkend="sql-createdatabase">database creation</link> (Dilip Kumar)
This is the new default for database creation and avoids the need
for checkpoints during database creation; the old method is still
available.
> > Have pg_upgrade preserve relfilenodes, tablespace, and database OIDs between old and new clusters (Shruthi KC,
AntoninHouska)
>
> "tablespace OIDs" or "tablespace and database OIDs and relfilenodes"
Good point, I went with:
Have <application>pg_upgrade</application> preserve tablespace
and database OIDs, and relfilenodes between old and new clusters
(Shruthi KC, Antonin Houska)
> > Limit support of pg_upgrade to old servers running PostgreSQL 9.2 and later (Tom Lane)
>
> The word "old" doesn't appear in the 2 release notes items about pg_dump and
> psql, and "old" makes it sound sounds like "antique" rather than "source".
Uh, so pg_upgrade uses the terms "old" and "new" in its option names,
e.g., oldbindir, newbindir. I don't think "source" would be an
improvement here.
> > Some internal-use-only types have also been assigned this column.
> this *value
Good point, I went with:
Some other internal-use-only values have also been assigned to
this column.
> > Allow custom scan provders to indicate if they support projections (Sven Klemm)
> > The default is now that custom scan providers can't support projections, so they need to be updated for this
release.
>
> Per the commit message, they don't "need" to be updated.
> I think this should say "The default now assumes that a custom scan provider
> does not support projections; to retain optimal performance, they should be
> updated to indicate whether that's supported.
Okay, I went with this text:
The default is now that custom scan providers are assumed to not
support projections; those that do need to be updated for this
release.
Cumulative applied patch attached.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
Attachment
On Tue, Jul 19, 2022 at 01:24:30PM -0400, Bruce Momjian wrote: > > > Remove pg_dump's --no-synchronized-snapshots option since all supported server versions support synchronized snapshots(Tom Lane) > > > > It'd be better to put that after the note about dropping support for upgrading > > clusters older than v9.2 in psql/pg_dump/pg_upgrade. > > Well, I put the --no-synchronized-snapshots item in incompatibilities > since it is a user-visible change that might require script adjustments. > However, I put the limit of pg_dump to 9.2 and greater into the pg_dump > section. Are you suggesting I move the--no-synchronized-snapshots item > down there? That doesn't match with the way I have listed other > incompatibilities so I am resistant to do that. I'd rather see the "limit support to v9.2" be moved or added to the "incompatibilities" section, maybe with "remove --no-synchronized-snapshots" as a secondary sentence. > > > 0. Add support for LZ4 and Zstandard compression of server-side base backups (Jeevan Ladhe, Robert Haas) > > > 1. Allow pg_basebackup to use LZ4 and Zstandard compression on server-side base backup files (Dipesh Pandit, JeevanLadhe) > > > 2. Allow pg_basebackup's --compress option to control the compression method and options (Michael Paquier, Robert Haas) > > > New options include server-gzip (gzip on the server), client-gzip (same as gzip). > > > 3. Allow pg_basebackup to compress on the server side and decompress on the client side before storage (Dipesh Pandit) > > > This is accomplished by specifying compression on the server side and plain output format. > > > > I still think these expose the incremental development rather than the > > user-facing change. > > > 1. It seems wrong to say "server-side" since client-side compression with > > LZ4/zstd is also supported. > > Agreed. I changed it to: > > Allow pg_basebackup to do LZ4 and Zstandard server-side compression > on base backup files (Dipesh Pandit, Jeevan Ladhe) This still misses the point that those compression algs are also supported on the client side, so it seems misleading to mention "server-side" support. > > > Allow custom scan provders to indicate if they support projections (Sven Klemm) > > > The default is now that custom scan providers can't support projections, so they need to be updated for this release. > > > > Per the commit message, they don't "need" to be updated. > > I think this should say "The default now assumes that a custom scan provider > > does not support projections; to retain optimal performance, they should be > > updated to indicate whether that's supported. > > Okay, I went with this text: > > The default is now that custom scan providers are assumed to not > support projections; those that do need to be updated for this > release. I'd say "those that do *will need to be updated" otherwise the sentence can sound like it means "those that need to be updated [will] ..." Thanks, -- Justin
On Tue, Jul 19, 2022 at 01:13:07PM -0500, Justin Pryzby wrote:
> On Tue, Jul 19, 2022 at 01:24:30PM -0400, Bruce Momjian wrote:
> > Well, I put the --no-synchronized-snapshots item in incompatibilities
> > since it is a user-visible change that might require script adjustments.
> > However, I put the limit of pg_dump to 9.2 and greater into the pg_dump
> > section. Are you suggesting I move the--no-synchronized-snapshots item
> > down there? That doesn't match with the way I have listed other
> > incompatibilities so I am resistant to do that.
>
> I'd rather see the "limit support to v9.2" be moved or added to the
> "incompatibilities" section, maybe with "remove --no-synchronized-snapshots"
> as a secondary sentence.
Is removing support for an older version an incompatibility --- I didn't
think so.
> > > > 0. Add support for LZ4 and Zstandard compression of server-side base backups (Jeevan Ladhe, Robert Haas)
> > > > 1. Allow pg_basebackup to use LZ4 and Zstandard compression on server-side base backup files (Dipesh Pandit,
JeevanLadhe)
> > > > 2. Allow pg_basebackup's --compress option to control the compression method and options (Michael Paquier,
RobertHaas)
> > > > New options include server-gzip (gzip on the server), client-gzip (same as gzip).
> > > > 3. Allow pg_basebackup to compress on the server side and decompress on the client side before storage (Dipesh
Pandit)
> > > > This is accomplished by specifying compression on the server side and plain output format.
> > >
> > > I still think these expose the incremental development rather than the
> > > user-facing change.
> >
> > > 1. It seems wrong to say "server-side" since client-side compression with
> > > LZ4/zstd is also supported.
> >
> > Agreed. I changed it to:
> >
> > Allow pg_basebackup to do LZ4 and Zstandard server-side compression
> > on base backup files (Dipesh Pandit, Jeevan Ladhe)
>
> This still misses the point that those compression algs are also supported on
> the client side, so it seems misleading to mention "server-side" support.
I reworked that paragraph in the attached patch. What we did was to add
server-side gzip/LZ/ZSTD, and client-side LZ/ZSTD. (We already had
client-side gzip.) Hopefully the new text is clearer. You can see the
new output here:
https://momjian.us/pgsql_docs/release-15.html
> > > > Allow custom scan provders to indicate if they support projections (Sven Klemm)
> > > > The default is now that custom scan providers can't support projections, so they need to be updated for this
release.
> > >
> > > Per the commit message, they don't "need" to be updated.
> > > I think this should say "The default now assumes that a custom scan provider
> > > does not support projections; to retain optimal performance, they should be
> > > updated to indicate whether that's supported.
> >
> > Okay, I went with this text:
> >
> > The default is now that custom scan providers are assumed to not
> > support projections; those that do need to be updated for this
> > release.
>
> I'd say "those that do *will need to be updated" otherwise the sentence can
> sound like it means "those that need to be updated [will] ..."
Oh, good point, done.
Cumulative patch attached.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
Attachment
On Tue, Jul 12, 2022 at 02:47:07PM -0400, Bruce Momjian wrote:
> On Mon, Jul 11, 2022 at 11:31:32PM -0700, Noah Misch wrote:
> > On Mon, Jul 11, 2022 at 12:39:57PM -0400, Bruce Momjian wrote:
> > > I had trouble reading the sentences in the order you used so I
> > > restructured it:
> > >
> > > The new default is one of the secure schema usage patterns that <xref
> > > linkend="ddl-schemas-patterns"/> has recommended since the security
> > > release for CVE-2018-1058. The change applies to newly-created
> > > databases in existing clusters and for new clusters. Upgrading a
> > > cluster or restoring a database dump will preserve existing permissions.
> >
> > I agree with the sentence order change.
>
> Great.
>
> > > For existing databases, especially those having multiple users, consider
> > > issuing <literal>REVOKE</literal> to adopt this new default. For new
> > > databases having zero need to defend against insider threats, granting
> > > <literal>USAGE</literal> permission on their <literal>public</literal>
> > > schemas will yield the behavior of prior releases.
> >
> > s/USAGE/CREATE/ in the last sentence. Looks good with that change.
>
> Ah, yes, of course.
Patch applied, I also adjusted the second paragraph to be more
symmetric. You can see the results here:
https://momjian.us/pgsql_docs/release-15.html
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
Hi, I noticed a stray "DETAILS?" marker while going through the release notes for 15. Is that subsection still under construction or review? > <listitem> > <para> > Record and check the collation of each <link > linkend="sql-createdatabase">database</link> (Peter Eisentraut) > [...] > to match the operating system collation version. DETAILS? > </para> > </listitem> Kind regards, Matthias van de Meent
On Sat, Aug 27, 2022 at 04:03:02PM +0200, Matthias van de Meent wrote:
> Hi,
>
> I noticed a stray "DETAILS?" marker while going through the release
> notes for 15. Is that subsection still under construction or review?
>
> > <listitem>
> > <para>
> > Record and check the collation of each <link
> > linkend="sql-createdatabase">database</link> (Peter Eisentraut)
> > [...]
> > to match the operating system collation version. DETAILS?
> > </para>
> > </listitem>
Good question --- the full text is:
<listitem>
<para>
Record and check the collation of each <link
linkend="sql-createdatabase">database</link> (Peter Eisentraut)
</para>
<para>
This is designed to detect collation
mismatches to avoid data corruption. Function
<function>pg_database_collation_actual_version()</function>
reports the underlying operating system collation version, and
<command>ALTER DATABASE ... REFRESH</command> sets the database
to match the operating system collation version. DETAILS?
</para>
</listitem>
I just can't figure out what the user needs to understand about this,
and I understand very little of it.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
Indecision is a decision. Inaction is an action. Mark Batterson
On 30.08.22 22:42, Bruce Momjian wrote: > Good question --- the full text is: > > <listitem> > <para> > Record and check the collation of each <link > linkend="sql-createdatabase">database</link> (Peter Eisentraut) > </para> > > <para> > This is designed to detect collation > mismatches to avoid data corruption. Function > <function>pg_database_collation_actual_version()</function> > reports the underlying operating system collation version, and > <command>ALTER DATABASE ... REFRESH</command> sets the database > to match the operating system collation version. DETAILS? > </para> > </listitem> > > I just can't figure out what the user needs to understand about this, > and I understand very little of it. We already had this feature for (schema-level) collations, now we have it on the level of the database collation.
On Wed, Aug 31, 2022 at 11:38:33AM +0200, Peter Eisentraut wrote: > On 30.08.22 22:42, Bruce Momjian wrote: > > Good question --- the full text is: > > > > <listitem> > > <para> > > Record and check the collation of each <link > > linkend="sql-createdatabase">database</link> (Peter Eisentraut) > > </para> > > > > <para> > > This is designed to detect collation > > mismatches to avoid data corruption. Function > > <function>pg_database_collation_actual_version()</function> > > reports the underlying operating system collation version, and > > <command>ALTER DATABASE ... REFRESH</command> sets the database > > to match the operating system collation version. DETAILS? > > </para> > > </listitem> > > > > I just can't figure out what the user needs to understand about this, > > and I understand very little of it. > > We already had this feature for (schema-level) collations, now we have it on > the level of the database collation. Okay, I figured out the interplay between OS collation version support, collation libraries, and collation levels. Here is an updated patch for the release notes. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Attachment
On Wed, Aug 31, 2022 at 04:03:06PM -0400, Bruce Momjian wrote: > On Wed, Aug 31, 2022 at 11:38:33AM +0200, Peter Eisentraut wrote: > > On 30.08.22 22:42, Bruce Momjian wrote: > > > I just can't figure out what the user needs to understand about this, > > > and I understand very little of it. > > > > We already had this feature for (schema-level) collations, now we have it on > > the level of the database collation. > > Okay, I figured out the interplay between OS collation version support, > collation libraries, and collation levels. Here is an updated patch for > the release notes. Patch applied. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On 5/10/22 11:44 AM, Bruce Momjian wrote: > I have completed the first draft of the PG 15 release notes > I assume there will be major adjustments in the next few weeks based on > feedback. I wanted to propose the "major enhancements" section to see if we can get an iteration in prior to Beta 4. Please see attached patched. Do we want to include anything else, or substitute any of the items? Thanks, Jonathan
Attachment
I noticed that the v15 release notes still refer to pg_checkpointer, which
was renamed to pg_checkpoint in b9eb0ff.
diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml
index d432c2db44..362728753a 100644
--- a/doc/src/sgml/release-15.sgml
+++ b/doc/src/sgml/release-15.sgml
@@ -1255,7 +1255,7 @@ Author: Jeff Davis <jdavis@postgresql.org>
<listitem>
<para>
Add predefined role <link
- linkend="predefined-roles-table"><literal>pg_checkpointer</literal></link>
+ linkend="predefined-roles-table"><literal>pg_checkpoint</literal></link>
that allows members to run <command>CHECKPOINT</command>
(Jeff Davis)
</para>
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
On 9/4/22 2:42 PM, Nathan Bossart wrote: > I noticed that the v15 release notes still refer to pg_checkpointer, which > was renamed to pg_checkpoint in b9eb0ff. > > diff --git a/doc/src/sgml/release-15.sgml b/doc/src/sgml/release-15.sgml > index d432c2db44..362728753a 100644 > --- a/doc/src/sgml/release-15.sgml > +++ b/doc/src/sgml/release-15.sgml > @@ -1255,7 +1255,7 @@ Author: Jeff Davis <jdavis@postgresql.org> > <listitem> > <para> > Add predefined role <link > - linkend="predefined-roles-table"><literal>pg_checkpointer</literal></link> > + linkend="predefined-roles-table"><literal>pg_checkpoint</literal></link> > that allows members to run <command>CHECKPOINT</command> > (Jeff Davis) > </para> Nudging on folks to review the major features language for the docs (pg15-maj-features.patch). Separately, per[1], including dense_rank() in the list of window functions with optimizations (dense-rank.diff). Thanks, Jonathan [1] https://www.postgresql.org/message-id/CAApHDvpr6N7egNfSttGdQMfL%2BKYBjUb_Zf%2BrHULb7_2k4V%3DGGg%40mail.gmail.com
Attachment
On 2022-Sep-12, Jonathan S. Katz wrote: > + <listitem> > + <para> > + Column-level and row-level filtering on > + <link linkend="logical-replication">logical replication</link> > + publications. > + </para> > + </listitem> -column-level filtering +the ability to specify column lists > + Row-level filtering and the ability to specify column lists on > + <link linkend="logical-replication">logical replication</link> > + publications. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ "No tengo por qué estar de acuerdo con lo que pienso" (Carlos Caszeli)
On 9/13/22 7:13 AM, Alvaro Herrera wrote: > On 2022-Sep-12, Jonathan S. Katz wrote: > >> + <listitem> >> + <para> >> + Column-level and row-level filtering on >> + <link linkend="logical-replication">logical replication</link> >> + publications. >> + </para> >> + </listitem> > > -column-level filtering > +the ability to specify column lists Adjusted to be similar to your suggestion. Updated patch attached. Thanks, Jonathan
Attachment
"Jonathan S. Katz" <jkatz@postgresql.org> writes:
> Adjusted to be similar to your suggestion. Updated patch attached.
I pushed this with a bit more copy-editing.
I'm planning to do a final(?) pass over the v15 notes today,
but I thought it'd be appropriate to push this separately.
regards, tom lane
On 9/23/22 11:25 AM, Tom Lane wrote: > "Jonathan S. Katz" <jkatz@postgresql.org> writes: >> Adjusted to be similar to your suggestion. Updated patch attached. > > I pushed this with a bit more copy-editing. > > I'm planning to do a final(?) pass over the v15 notes today, > but I thought it'd be appropriate to push this separately. Thanks! RE "final pass", there's still an errant "BACKPATCHED"[1] that still needs addressing. I didn't have a chance to verify if it was indeed backpatched. Jonathan [1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=doc/src/sgml/release-15.sgml;hb=refs/heads/REL_15_STABLE#l460
Attachment
"Jonathan S. Katz" <jkatz@postgresql.org> writes:
> On 9/23/22 11:25 AM, Tom Lane wrote:
>> I'm planning to do a final(?) pass over the v15 notes today,
>> but I thought it'd be appropriate to push this separately.
> RE "final pass", there's still an errant "BACKPATCHED"[1] that still
> needs addressing. I didn't have a chance to verify if it was indeed
> backpatched.
Yeah, that one indeed needs removed (and I've done so). I see a
few other places where Bruce left notes about things that need more
clarification. I'm just finishing a pass of "update for subsequent
commits", and then I'll start on copy-editing.
regards, tom lane
On 9/23/22 1:33 PM, Tom Lane wrote: > "Jonathan S. Katz" <jkatz@postgresql.org> writes: >> On 9/23/22 11:25 AM, Tom Lane wrote: >>> I'm planning to do a final(?) pass over the v15 notes today, >>> but I thought it'd be appropriate to push this separately. > >> RE "final pass", there's still an errant "BACKPATCHED"[1] that still >> needs addressing. I didn't have a chance to verify if it was indeed >> backpatched. > > Yeah, that one indeed needs removed (and I've done so). I see a > few other places where Bruce left notes about things that need more > clarification. I'm just finishing a pass of "update for subsequent > commits", and then I'll start on copy-editing. ACK. I will available to review during the weekend (Sunday). Jonathan
Attachment
On Fri, Sep 23, 2022 at 01:33:07PM -0400, Tom Lane wrote: > "Jonathan S. Katz" <jkatz@postgresql.org> writes: > > On 9/23/22 11:25 AM, Tom Lane wrote: > >> I'm planning to do a final(?) pass over the v15 notes today, > >> but I thought it'd be appropriate to push this separately. > > > RE "final pass", there's still an errant "BACKPATCHED"[1] that still > > needs addressing. I didn't have a chance to verify if it was indeed > > backpatched. > > Yeah, that one indeed needs removed (and I've done so). I see a > few other places where Bruce left notes about things that need more > clarification. I'm just finishing a pass of "update for subsequent > commits", and then I'll start on copy-editing. Some possible changes for your consideration. + Store <application>pg_upgrade</application>'s log and + temporary files in a subdirectory of the new cluster called + <filename>pg_upgrade_output.d</filename> (Justin Pryzby) + Previously such files were left in the current directory, + requiring manual cleanup. It's still necessary to remove them + manually afterwards, but now one can just remove that whole + subdirectory. If pg_upgrade succeeds, then it removes the dir itself (so it's not "necessary"). And if it fails after starting to restore the schema, then it's necessary to remove not the "subdirectory" but the whole new-cluster dir. + Make <application>pg_upgrade</application> preserve tablespace + and database OIDs, as well as table relfilenode numbers s/table/relation/ ? You changed this to use spaces: | The new setting is <literal>log_destination = jsonlog</literal>. but then left these without spaces: | and <literal>wal_level=minimal</literal>. | This is enabled via <literal>--compress=lz4</literal> and requires + value, use the transaction start time not wall clock time to s/not/rather than/ ? + Adjust <application>psql</application> so that Readline's should use <productname>Readline ? + Previously a pound marker was inserted, but that's pretty + unhelpful in SQL. This sounds more like a candid commit message than a release note. + Improve performance of dumping databases with many objects s/of/when/ ? + New options are <literal>server</literal> to write the *The* new options + In some cases a partition child table could appear more than once. Technically "partition child table" is redundant -- Justin
Justin Pryzby <pryzby@telsasoft.com> writes:
> On Fri, Sep 23, 2022 at 01:33:07PM -0400, Tom Lane wrote:
> + Previously such files were left in the current directory,
> + requiring manual cleanup. It's still necessary to remove them
> + manually afterwards, but now one can just remove that whole
> + subdirectory.
> If pg_upgrade succeeds, then it removes the dir itself (so it's not
> "necessary").
Ah, I'd only ever paid attention to failure cases, so I didn't
realize that :-(. Text adjusted:
Previously such files were left in the current directory,
requiring manual cleanup. Now they are automatically removed on
successful completion of <application>pg_upgrade</application>.
I took most of your other suggestions, too. Thanks!
regards, tom lane
On Tue, 13 Sept 2022 at 09:31, Jonathan S. Katz <jkatz@postgresql.org> wrote: > Separately, per[1], including dense_rank() in the list of window > functions with optimizations (dense-rank.diff). This one might have been forgotten... ? I can push it shortly if nobody objects. David > [1] > https://www.postgresql.org/message-id/CAApHDvpr6N7egNfSttGdQMfL%2BKYBjUb_Zf%2BrHULb7_2k4V%3DGGg%40mail.gmail.com
David Rowley <dgrowleyml@gmail.com> writes:
> On Tue, 13 Sept 2022 at 09:31, Jonathan S. Katz <jkatz@postgresql.org> wrote:
>> Separately, per[1], including dense_rank() in the list of window
>> functions with optimizations (dense-rank.diff).
> This one might have been forgotten... ? I can push it shortly if nobody objects.
Yeah, I missed that one. We're theoretically in the wrap freeze for
15rc1, but I don't have a problem with release-note changes.
regards, tom lane
On Tue, 27 Sept 2022 at 10:45, Tom Lane <tgl@sss.pgh.pa.us> wrote: > > David Rowley <dgrowleyml@gmail.com> writes: > > On Tue, 13 Sept 2022 at 09:31, Jonathan S. Katz <jkatz@postgresql.org> wrote: > >> Separately, per[1], including dense_rank() in the list of window > >> functions with optimizations (dense-rank.diff). > > > This one might have been forgotten... ? I can push it shortly if nobody objects. > > Yeah, I missed that one. We're theoretically in the wrap freeze for > 15rc1, but I don't have a problem with release-note changes. Thanks. I've just pushed it. David