Thread: PostgreSQL 17 Beta 1 release announcement draft

PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
Hi,

Attached is a copy of the PostgreSQL 17 Beta 1 release announcement 
draft. This contains a user-facing summary of some of the features that 
will be available in the Beta, as well as a call to test. I've made an 
effort to group them logically around the different workflows they affect.

A few notes:

* The section with the features is not 80-char delimited. I will do that 
before the final copy

* There is an explicit callout that we've added in the SQL/JSON features 
that were previously reverted in PG15. I want to ensure we're 
transparent about that, but also use it as a hook to get people testing.

When reviewing:

* Please check for correctness of feature descriptions, keeping in mind 
this is targeted for a general audience

* Please indicate if you believe there's a notable omission, or if we 
should omit any of these callouts

* Please indicate if a description is confusing - I'm happy to rewrite 
to ensure it's clearer.

Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the 
beta release takes some extra effort, I want to ensure all changes are 
in with time to spare before release day.

Thanks,

Jonathan

Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
David Rowley
Date:
Thanks for writing that up. It's always exciting to see this each year.

On Thu, 16 May 2024 at 13:45, Jonathan S. Katz <jkatz@postgresql.org> wrote:
> * Please indicate if you believe there's a notable omission, or if we
> should omit any of these callouts

I'd say the streaming read stuff added in b5a9b18cd and subsequent
commits like b7b0f3f27 and 041b96802 are worth a mention. I'd be happy
to see this over the IS NOT NULL qual stuff that I worked on in there
or even the AVX512 bit counting. Speeding up a backwater aggregate
function is nice, but IMO, not compatible with reducing the number
reads.

There's some benchmarking in a youtube video:
https://youtu.be/QAYzWAlxCYc?si=L0UT6Lrf067ZBv46&t=237

> * Please indicate if a description is confusing - I'm happy to rewrite
> to ensure it's clearer.
>
> Please provide feedback no later than Wed 2024-05-22 18:00 UTC.

The only other thing I saw from a quick read was a stray "the" in "the
copy proceed even if the there is an error inserting a row."

David



Re: PostgreSQL 17 Beta 1 release announcement draft

From
Kashif Zeeshan
Date:
Hi Jonathan

Did the review and did not find any issues.

Regards
Kashif Zeeshan
Bitnine Global

On Thu, May 16, 2024 at 6:45 AM Jonathan S. Katz <jkatz@postgresql.org> wrote:
Hi,

Attached is a copy of the PostgreSQL 17 Beta 1 release announcement
draft. This contains a user-facing summary of some of the features that
will be available in the Beta, as well as a call to test. I've made an
effort to group them logically around the different workflows they affect.

A few notes:

* The section with the features is not 80-char delimited. I will do that
before the final copy

* There is an explicit callout that we've added in the SQL/JSON features
that were previously reverted in PG15. I want to ensure we're
transparent about that, but also use it as a hook to get people testing.

When reviewing:

* Please check for correctness of feature descriptions, keeping in mind
this is targeted for a general audience

* Please indicate if you believe there's a notable omission, or if we
should omit any of these callouts

* Please indicate if a description is confusing - I'm happy to rewrite
to ensure it's clearer.

Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the
beta release takes some extra effort, I want to ensure all changes are
in with time to spare before release day.

Thanks,

Jonathan

Re: PostgreSQL 17 Beta 1 release announcement draft

From
Thom Brown
Date:
On Thu, May 16, 2024, 02:45 Jonathan S. Katz <jkatz@postgresql.org> wrote:
Hi,

Attached is a copy of the PostgreSQL 17 Beta 1 release announcement
draft. This contains a user-facing summary of some of the features that
will be available in the Beta, as well as a call to test. I've made an
effort to group them logically around the different workflows they affect.

A few notes:

* The section with the features is not 80-char delimited. I will do that
before the final copy

* There is an explicit callout that we've added in the SQL/JSON features
that were previously reverted in PG15. I want to ensure we're
transparent about that, but also use it as a hook to get people testing.

When reviewing:

* Please check for correctness of feature descriptions, keeping in mind
this is targeted for a general audience

* Please indicate if you believe there's a notable omission, or if we
should omit any of these callouts

* Please indicate if a description is confusing - I'm happy to rewrite
to ensure it's clearer.

Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the
beta release takes some extra effort, I want to ensure all changes are
in with time to spare before release day.

"Now as of PostgreSQL 17, you can now use parallel index builds for [BRIN](https://www.postgresql.org/docs/17/brin.html) indexes."

The 2nd "now" is redundant.


"Finally, PostgreSQL 17 adds more explicitly SIMD instructions, including AVX-512 support for the [`bit_count](https://www.postgresql.org/docs/17/functions-bitstring.html) function."

Would "SIMD-explicit instructions" be better? Also, I know you may not be using markdown for the final version, but the bit_count backtick isn't matched by a closing backtick.


"[`COPY`](https://www.postgresql.org/docs/17/sql-copy.html), used to efficiently bulk load data into PostgreSQL"

The "used to" makes me stumble into reading it as meaning "it previously could efficiently bulk load data".

Perhaps just add a "which is" before "used"?


"PostgreSQL 17 includes a built-in collation provider that provides similar semantics to the `C` collation provided by libc."

"provider", "provides", and "provided" feels too repetitive.

How about, "PostgreSQL 17 includes a built-in collation provider with semantics similar to the `C` collation offered by libc."?


Regards

Thom

Re: PostgreSQL 17 Beta 1 release announcement draft

From
Bertrand Drouvot
Date:
Hi,

On Wed, May 15, 2024 at 09:45:35PM -0400, Jonathan S. Katz wrote:
> Hi,
> 
> Attached is a copy of the PostgreSQL 17 Beta 1 release announcement draft.

Thanks for working on it!

I've one comment:

> PostgreSQL 17 also introduces a new view,
[`pg_wait_events`](https://www.postgresql.org/docs/17/view-pg-wait-events.html),which provides descriptions about wait
eventsand can be combined with `pg_stat_activity` to give more insight into an operation.
 

Instead of "to give more insight into an operation", what about "to give more
insight about what a session is waiting for (should it be active)"?

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com



Re: PostgreSQL 17 Beta 1 release announcement draft

From
zaidagilist
Date:
Hello,

I am trying to open the 17 docs but it looks removed. Getting
following message "Page not found"

https://www.postgresql.org/docs/17/


Regards,
Zaid Shabbir

On Thu, May 16, 2024 at 10:16 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:
>
> Hi,
>
> On Wed, May 15, 2024 at 09:45:35PM -0400, Jonathan S. Katz wrote:
> > Hi,
> >
> > Attached is a copy of the PostgreSQL 17 Beta 1 release announcement draft.
>
> Thanks for working on it!
>
> I've one comment:
>
> > PostgreSQL 17 also introduces a new view,
[`pg_wait_events`](https://www.postgresql.org/docs/17/view-pg-wait-events.html),which provides descriptions about wait
eventsand can be combined with `pg_stat_activity` to give more insight into an operation. 
>
> Instead of "to give more insight into an operation", what about "to give more
> insight about what a session is waiting for (should it be active)"?
>
> Regards,
>
> --
> Bertrand Drouvot
> PostgreSQL Contributors Team
> RDS Open Source Databases
> Amazon Web Services: https://aws.amazon.com
>
>

Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
Thom Brown
Date:
On Thu, May 16, 2024, 06:37 zaidagilist <zaidagilist@gmail.com> wrote:
Hello,

I am trying to open the 17 docs but it looks removed. Getting
following message "Page not found"

https://www.postgresql.org/docs/17/


Regards,
Zaid Shabbir

That link isn't set up yet, but will be (or should be) when the announcement goes out.

Regards 

Thom

Re: PostgreSQL 17 Beta 1 release announcement draft

From
David Rowley
Date:
On Thu, 16 May 2024 at 17:37, zaidagilist <zaidagilist@gmail.com> wrote:
> I am trying to open the 17 docs but it looks removed. Getting
> following message "Page not found"
>
> https://www.postgresql.org/docs/17/

It's called "devel" for "development" until we branch sometime before July:

https://www.postgresql.org/docs/devel/

David



Re: PostgreSQL 17 Beta 1 release announcement draft

From
Jelte Fennema-Nio
Date:
On Thu, 16 May 2024 at 03:45, Jonathan S. Katz <jkatz@postgresql.org> wrote:
> Attached is a copy of the PostgreSQL 17 Beta 1 release announcement
> draft.

I think we can quickly mention c4ab7da6061 in the COPY paragraph, in
some benchmarks it improved perf by close to 2x. Something like this:
"has improved performance in PostgreSQL 17 when the source encoding
matches the destination encoding *and when sending large rows from
server to client*"

Also, I think it's a bit weird to put the current COPY paragraph under
Developer Experience. I think if you want to keep it there instead of
move it to the per section, we should put the line about IGNORE_ERROR
first instead of the perf improvements. Now the IGNORE_ERROR addition
seems more of an afterthought.

s/IGNORE_ERROR/ON_ERROR

I think it would be good to clarify if the following applies when
upgrading from or to PostgreSQL 17:
"Starting with PostgreSQL 17, you no longer need to drop logical
replication slots when using pg_upgrade"

Finally, I personally would have included a lot more links for the new
items in this document. Some that would benefit from being a link
imho:
- pg_createsubscriber
- JSON_TABLE
- SQL/JSON constructor
- SQL/JSON query functions
- ON_ERROR
- sslnegotiation
- PQchangePassword
- pg_maintain



Re: PostgreSQL 17 Beta 1 release announcement draft

From
Joe Conway
Date:
On 5/15/24 21:45, Jonathan S. Katz wrote:
> Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the
> beta release takes some extra effort, I want to ensure all changes are
> in with time to spare before release day.

"You can find information about all of the features and changes found in
PostgreSQL 17"

Sounds repetitive; maybe:

"Information about all of the features and changes in PostgreSQL 17 can 
be found in the [release notes]"


"more explicitly SIMD instructions" I think ought to be "more explicit 
SIMD instructions"


"PostgreSQL 17 includes a built-in collation provider that provides 
similar semantics to the `C` collation provided by libc."

I think that needs to mention UTF-8 encoding somehow, and "provided by 
libc" is not really true; maybe:

"PostgreSQL 17 includes a built-in collation provider that provides 
similar sorting semantics to the `C` collation except with UTF-8 
encoding rather than SQL_ASCII."


-- 
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com




Re: PostgreSQL 17 Beta 1 release announcement draft

From
Joe Conway
Date:
On 5/16/24 08:05, Joe Conway wrote:
> On 5/15/24 21:45, Jonathan S. Katz wrote:
>> Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the
>> beta release takes some extra effort, I want to ensure all changes are
>> in with time to spare before release day.

"`EXPLAIN` can now show how much time is spent for I/O block reads and 
writes"

Is that really EXPLAIN, or rather EXPLAIN ANALYZE?

-- 
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com




Re: PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
On 5/15/24 10:42 PM, David Rowley wrote:
> Thanks for writing that up. It's always exciting to see this each year.
> 
> On Thu, 16 May 2024 at 13:45, Jonathan S. Katz <jkatz@postgresql.org> wrote:
>> * Please indicate if you believe there's a notable omission, or if we
>> should omit any of these callouts
> 
> I'd say the streaming read stuff added in b5a9b18cd and subsequent
> commits like b7b0f3f27 and 041b96802 are worth a mention. I'd be happy
> to see this over the IS NOT NULL qual stuff that I worked on in there
> or even the AVX512 bit counting. Speeding up a backwater aggregate
> function is nice, but IMO, not compatible with reducing the number
> reads.
> There's some benchmarking in a youtube video:
> https://youtu.be/QAYzWAlxCYc?si=L0UT6Lrf067ZBv46&t=237

Nice! Definitely agree on including this - it wasn't initially clear to 
me on the read of the release notes. I'll update it. Please see in the 
next revision (will posted upthread), proposed text here for 
convenience, as I'm not sure I'm appropriately capturing it:

==
This release introduces adds an interface to stream I/O, and can show 
performance improvements when performing sequential scans and running 
[`ANALYZE`](https://www.postgresql.org/docs/17/sql-analyze.html).
==

The AVX-512 bit counting showed solid impact[1] on the binary distance 
functions in pgvector (I have to re-run again w/v17, as I seem to recall 
seeing some numbers that boosted it 5-7x [but recall isn't 100% ;)]).

>> * Please indicate if a description is confusing - I'm happy to rewrite
>> to ensure it's clearer.
>>
>> Please provide feedback no later than Wed 2024-05-22 18:00 UTC.
> 
> The only other thing I saw from a quick read was a stray "the" in "the
> copy proceed even if the there is an error inserting a row."

Thanks!

Jonathan

[1] https://github.com/pgvector/pgvector/pull/519


Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
On 5/16/24 1:10 AM, Thom Brown wrote:
> On Thu, May 16, 2024, 02:45 Jonathan S. Katz <jkatz@postgresql.org 
> <mailto:jkatz@postgresql.org>> wrote:
> 
>     Hi,
> 
>     Attached is a copy of the PostgreSQL 17 Beta 1 release announcement
>     draft. This contains a user-facing summary of some of the features that
>     will be available in the Beta, as well as a call to test. I've made an
>     effort to group them logically around the different workflows they
>     affect.
> 
>     A few notes:
> 
>     * The section with the features is not 80-char delimited. I will do
>     that
>     before the final copy
> 
>     * There is an explicit callout that we've added in the SQL/JSON
>     features
>     that were previously reverted in PG15. I want to ensure we're
>     transparent about that, but also use it as a hook to get people testing.
> 
>     When reviewing:
> 
>     * Please check for correctness of feature descriptions, keeping in mind
>     this is targeted for a general audience
> 
>     * Please indicate if you believe there's a notable omission, or if we
>     should omit any of these callouts
> 
>     * Please indicate if a description is confusing - I'm happy to rewrite
>     to ensure it's clearer.
> 
>     Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the
>     beta release takes some extra effort, I want to ensure all changes are
>     in with time to spare before release day.
> 
> 
> "Now as of PostgreSQL 17, you can now use parallel index builds for 
> [BRIN](https://www.postgresql.org/docs/17/brin.html 
> <https://www.postgresql.org/docs/17/brin.html>) indexes."
> 
> The 2nd "now" is redundant.
> 
> 
> "Finally, PostgreSQL 17 adds more explicitly SIMD instructions, 
> including AVX-512 support for the 
> [`bit_count](https://www.postgresql.org/docs/17/functions-bitstring.html 
> <https://www.postgresql.org/docs/17/functions-bitstring.html>) function."
> 
> Would "SIMD-explicit instructions" be better? Also, I know you may not 
> be using markdown for the final version, but the bit_count backtick 
> isn't matched by a closing backtick.
> 
> 
> "[`COPY`](https://www.postgresql.org/docs/17/sql-copy.html 
> <https://www.postgresql.org/docs/17/sql-copy.html>), used to efficiently 
> bulk load data into PostgreSQL"
> 
> The "used to" makes me stumble into reading it as meaning "it previously 
> could efficiently bulk load data".
> 
> Perhaps just add a "which is" before "used"?
> 
> 
> "PostgreSQL 17 includes a built-in collation provider that provides 
> similar semantics to the `C` collation provided by libc."
> 
> "provider", "provides", and "provided" feels too repetitive.
> 
> How about, "PostgreSQL 17 includes a built-in collation provider with 
> semantics similar to the `C` collation offered by libc."?

Thanks - I accepted (with modifications) most of the suggestions here. 
I'll include in the next version of the draft.

Jonathan


Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
On 5/16/24 1:15 AM, Bertrand Drouvot wrote:
> Hi,
> 
> On Wed, May 15, 2024 at 09:45:35PM -0400, Jonathan S. Katz wrote:
>> Hi,
>>
>> Attached is a copy of the PostgreSQL 17 Beta 1 release announcement draft.
> 
> Thanks for working on it!
> 
> I've one comment:
> 
>> PostgreSQL 17 also introduces a new view,
[`pg_wait_events`](https://www.postgresql.org/docs/17/view-pg-wait-events.html),which provides descriptions about wait
eventsand can be combined with `pg_stat_activity` to give more insight into an operation.
 
> 
> Instead of "to give more insight into an operation", what about "to give more
> insight about what a session is waiting for (should it be active)"?

I put:

"to give more in insight into why a session is blocked."

Does that work?

Jonathan


Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
On 5/16/24 6:41 AM, Jelte Fennema-Nio wrote:
> On Thu, 16 May 2024 at 03:45, Jonathan S. Katz <jkatz@postgresql.org> wrote:
>> Attached is a copy of the PostgreSQL 17 Beta 1 release announcement
>> draft.
> 
> I think we can quickly mention c4ab7da6061 in the COPY paragraph, in
> some benchmarks it improved perf by close to 2x. Something like this:
> "has improved performance in PostgreSQL 17 when the source encoding
> matches the destination encoding *and when sending large rows from
> server to client*"

(I'm going to make a note to test this with loading large vectors :) 
I've modified the text to reflect this. Please see the new language 
upthread.

> Also, I think it's a bit weird to put the current COPY paragraph under
> Developer Experience. I think if you want to keep it there instead of
> move it to the per section, we should put the line about IGNORE_ERROR
> first instead of the perf improvements. Now the IGNORE_ERROR addition
> seems more of an afterthought.

I don't agree with this. I think we want to push COPY as a developer 
feature - I see a lot of people not utilizing COPY appropriate when it 
would really benefit the performance of their app, and I think 
emphasizing it as the way to do bulk loads (while touting that it's even 
faster!) will help make it more apparent.

> s/IGNORE_ERROR/ON_ERROR

Thanks.

> I think it would be good to clarify if the following applies when
> upgrading from or to PostgreSQL 17:
> "Starting with PostgreSQL 17, you no longer need to drop logical
> replication slots when using pg_upgrade"

Adjusted.

> Finally, I personally would have included a lot more links for the new
> items in this document. Some that would benefit from being a link
> imho:
> - pg_createsubscriber
> - JSON_TABLE
> - SQL/JSON constructor
> - SQL/JSON query functions
> - ON_ERROR
> - sslnegotiation
> - PQchangePassword
> - pg_maintain

I have to check if these have deep links or not, but I was planning to 
make another pass once the copy (no pun intended) is closer to 
finalized, so I don't have to constantly edit markdown.

Jonathan

Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
On 5/16/24 8:05 AM, Joe Conway wrote:
> On 5/15/24 21:45, Jonathan S. Katz wrote:
>> Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the
>> beta release takes some extra effort, I want to ensure all changes are
>> in with time to spare before release day.
> 
> "You can find information about all of the features and changes found in
> PostgreSQL 17"
> 
> Sounds repetitive; maybe:
> 
> "Information about all of the features and changes in PostgreSQL 17 can 
> be found in the [release notes]"

The first is active voice, the suggestion passive. However, I tightened 
the language:

You can find information about all of the PostgreSQL 17 features and 
changes in the [release 
notes](https://www.postgresql.org/docs/17/release-17.html):

> "PostgreSQL 17 includes a built-in collation provider that provides 
> similar semantics to the `C` collation provided by libc."
> 
> I think that needs to mention UTF-8 encoding somehow, and "provided by 
> libc" is not really true; maybe:
> 
> "PostgreSQL 17 includes a built-in collation provider that provides 
> similar sorting semantics to the `C` collation except with UTF-8 
> encoding rather than SQL_ASCII."

WFM. Taken verbatim.

Thanks,

Jonathan

Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
On 5/15/24 9:45 PM, Jonathan S. Katz wrote:
> Hi,
> 
> Attached is a copy of the PostgreSQL 17 Beta 1 release announcement 
> draft. This contains a user-facing summary of some of the features that 
> will be available in the Beta, as well as a call to test. I've made an 
> effort to group them logically around the different workflows they affect.
> 
> A few notes:
> 
> * The section with the features is not 80-char delimited. I will do that 
> before the final copy
> 
> * There is an explicit callout that we've added in the SQL/JSON features 
> that were previously reverted in PG15. I want to ensure we're 
> transparent about that, but also use it as a hook to get people testing.
> 
> When reviewing:
> 
> * Please check for correctness of feature descriptions, keeping in mind 
> this is targeted for a general audience
> 
> * Please indicate if you believe there's a notable omission, or if we 
> should omit any of these callouts
> 
> * Please indicate if a description is confusing - I'm happy to rewrite 
> to ensure it's clearer.
> 
> Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the 
> beta release takes some extra effort, I want to ensure all changes are 
> in with time to spare before release day.

Thanks for all the feedback to date. Please see the next revision. 
Again, please provide feedback no later than 2024-05-22 18:00 UTC.

Thanks,

Jonathan


Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
Erik Rijkers
Date:
Op 5/19/24 om 23:34 schreef Jonathan S. Katz:
> On 5/15/24 9:45 PM, Jonathan S. Katz wrote:
>> Hi,
>>
>> Attached is a copy of the PostgreSQL 17 Beta 1 release announcement 

'This release introduces adds an interface'  should be:
'This release adds an interface'
    (or 'introduces'; just not both...)

Thanks,

Erik Rijkers




Re: PostgreSQL 17 Beta 1 release announcement draft

From
David Rowley
Date:
On Mon, 20 May 2024 at 09:35, Jonathan S. Katz <jkatz@postgresql.org> wrote:
> Thanks for all the feedback to date. Please see the next revision.
> Again, please provide feedback no later than 2024-05-22 18:00 UTC.

Thanks for the updates.

> [`COPY`](https://www.postgresql.org/docs/17/sql-copy.html) is used to efficiently bulk load data into PostgreSQL, and
withPostgreSQL 17 shows a 2x performance improvement when loading large rows.
 

The 2x thing mentioned by Jelte is for COPY TO rather than COPY FROM.
So I think "exporting" or "sending large rows to the client"  rather
than "loading".

There's also a stray "with" in that sentence.

David



Re: PostgreSQL 17 Beta 1 release announcement draft

From
John Naylor
Date:
Hi Jon,

Regarding vacuum "has shown up to a 6x improvement in overall time to
complete its work" -- I believe I've seen reported numbers close to
that only 1) when measuring the index phase in isolation or maybe 2)
the entire vacuum of unlogged tables with one, perfectly-correlated
index (testing has less variance with WAL out of the picture). I
believe tables with many indexes would show a lot of improvement, but
I'm not aware of testing that case specifically. Can you clarify where
6x came from?



Re: PostgreSQL 17 Beta 1 release announcement draft

From
Bertrand Drouvot
Date:
Hi,

On Sun, May 19, 2024 at 05:10:10PM -0400, Jonathan S. Katz wrote:
> On 5/16/24 1:15 AM, Bertrand Drouvot wrote:
> > Hi,
> > 
> > On Wed, May 15, 2024 at 09:45:35PM -0400, Jonathan S. Katz wrote:
> > > Hi,
> > > 
> > > Attached is a copy of the PostgreSQL 17 Beta 1 release announcement draft.
> > 
> > Thanks for working on it!
> > 
> > I've one comment:
> > 
> > > PostgreSQL 17 also introduces a new view,
[`pg_wait_events`](https://www.postgresql.org/docs/17/view-pg-wait-events.html),which provides descriptions about wait
eventsand can be combined with `pg_stat_activity` to give more insight into an operation.
 
> > 
> > Instead of "to give more insight into an operation", what about "to give more
> > insight about what a session is waiting for (should it be active)"?
> 
> I put:
> 
> "to give more in insight into why a session is blocked."

Thanks!

> 
> Does that work?
> 

I think using "waiting" is better (as the view is "pg_wait_events" and the
join with pg_stat_activity would be on the "wait_event_type" and "wait_event"
columns).

The reason I mentioned "should it be active" is because wait_event and wait_event_type
could be non empty in pg_stat_activity while the session is not in an active state
anymore (then not waiting).

A right query would be like the one in [1]:

"
SELECT a.pid, a.wait_event, w.description
  FROM pg_stat_activity a JOIN
       pg_wait_events w ON (a.wait_event_type = w.type AND
                            a.wait_event = w.name)
  WHERE a.wait_event is NOT NULL and a.state = 'active';
"

means filtering on the "active" state too, and that's what the description
proposal I made was trying to highlight.

[1]: https://www.postgresql.org/docs/devel/monitoring-stats.html

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com



Re: PostgreSQL 17 Beta 1 release announcement draft

From
Alvaro Herrera
Date:
On 2024-May-19, Jonathan S. Katz wrote:

> ### Query and Operational Performance Improvements

In this section I'd add mention the new GUCs to control SLRU memory
size, which is going to be a huge performance boon for cases where the
current fixed-size buffers cause bottlenecks.  Perhaps something like

"Increase scalability of transaction, subtransaction and multixact
shared memory buffer handling, and make their buffer sizes configurable".

I don't know if we have any published numbers of the performance
improvement achieved, but with this patch (or ancestor ones) some
systems go from completely unoperational to working perfectly fine.
Maybe the best link is here
https://www.postgresql.org/docs/devel/runtime-config-resource.html#GUC-MULTIXACT-MEMBER-BUFFERS
though exactly which GUC affects any particular user is workload-
dependant, so I'm not sure how best to do it.

> ### Developer Experience

I think this section should also include the libpq query cancellation
improvements Jelte wrote.  Maybe something like "On the client side,
PostgreSQL 17 provides better support for asynchronous and more secure
query cancellation routines in libpq."  --> link to
https://www.postgresql.org/docs/17/libpq-cancel.html

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"At least to kernel hackers, who really are human, despite occasional
rumors to the contrary" (LWN.net)



Re: PostgreSQL 17 Beta 1 release announcement draft

From
Alvaro Herrera
Date:
On 2024-May-16, David Rowley wrote:

> On Thu, 16 May 2024 at 17:37, zaidagilist <zaidagilist@gmail.com> wrote:
> > I am trying to open the 17 docs but it looks removed. Getting
> > following message "Page not found"
> >
> > https://www.postgresql.org/docs/17/
> 
> It's called "devel" for "development" until we branch sometime before July:
> 
> https://www.postgresql.org/docs/devel/

Hmm, but that would mean that the Beta1 announce would ship full of
links that will remain broken until July.  I'm not sure what the
workflow for this is, but I hope the /17/ URLs would become valid with
beta1, later this week.

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"No renuncies a nada. No te aferres a nada."



Re: PostgreSQL 17 Beta 1 release announcement draft

From
David Rowley
Date:
On Mon, 20 May 2024 at 22:11, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2024-May-16, David Rowley wrote:
>
> > On Thu, 16 May 2024 at 17:37, zaidagilist <zaidagilist@gmail.com> wrote:
> > > I am trying to open the 17 docs but it looks removed. Getting
> > > following message "Page not found"
> > >
> > > https://www.postgresql.org/docs/17/
> >
> > It's called "devel" for "development" until we branch sometime before July:
> >
> > https://www.postgresql.org/docs/devel/
>
> Hmm, but that would mean that the Beta1 announce would ship full of
> links that will remain broken until July.  I'm not sure what the
> workflow for this is, but I hope the /17/ URLs would become valid with
> beta1, later this week.

I didn't quite click that it was Jonathan's links that were being
complained about.

I don't know how the website picks up where to link the doc page for a
given version.  I see from e0b82fc8e8 that the PACKAGE_VERSION was
changed from 16devel to 16beta1. Does the website have something that
extracts "devel" from the former and "16" from the latter?  I see the
release announcement for 16beta1 had /16/ links per [1].  So, I guess
it works. I just don't know how.

David

[1] https://www.postgresql.org/about/news/postgresql-16-beta-1-released-2643/



Re: PostgreSQL 17 Beta 1 release announcement draft

From
Thom Brown
Date:
On Mon, 20 May 2024 at 00:24, David Rowley <dgrowleyml@gmail.com> wrote:
On Mon, 20 May 2024 at 09:35, Jonathan S. Katz <jkatz@postgresql.org> wrote:
> Thanks for all the feedback to date. Please see the next revision.
> Again, please provide feedback no later than 2024-05-22 18:00 UTC.

Thanks for the updates.

> [`COPY`](https://www.postgresql.org/docs/17/sql-copy.html) is used to efficiently bulk load data into PostgreSQL, and with PostgreSQL 17 shows a 2x performance improvement when loading large rows.

The 2x thing mentioned by Jelte is for COPY TO rather than COPY FROM.
So I think "exporting" or "sending large rows to the client"  rather
than "loading".

There's also a stray "with" in that sentence.

Are you referring to the "with" in "and with PostgreSQL 17"? If so, it looks valid to me.
--
Thom

Re: PostgreSQL 17 Beta 1 release announcement draft

From
David Rowley
Date:
On Mon, 20 May 2024 at 23:16, Thom Brown <thom@linux.com> wrote:
>
> On Mon, 20 May 2024 at 00:24, David Rowley <dgrowleyml@gmail.com> wrote:
>>
>> On Mon, 20 May 2024 at 09:35, Jonathan S. Katz <jkatz@postgresql.org> wrote:
>> > Thanks for all the feedback to date. Please see the next revision.
>> > Again, please provide feedback no later than 2024-05-22 18:00 UTC.
>>
>> Thanks for the updates.
>>
>> > [`COPY`](https://www.postgresql.org/docs/17/sql-copy.html) is used to efficiently bulk load data into PostgreSQL,
andwith PostgreSQL 17 shows a 2x performance improvement when loading large rows.
 
>>
>> The 2x thing mentioned by Jelte is for COPY TO rather than COPY FROM.
>> So I think "exporting" or "sending large rows to the client"  rather
>> than "loading".
>>
>> There's also a stray "with" in that sentence.
>
>
> Are you referring to the "with" in "and with PostgreSQL 17"? If so, it looks valid to me.

Yes that one.  It sounds wrong to me, but that's from a British
English point of view. I'm continuing to learn the subtle differences
with American English. Maybe this is one.

It would make sense to me if it was "and with PostgreSQL 17, a 2x
...". From my point of view either "with" shouldn't be there or
"shows" could be replaced with a comma. However, if you're ok with it,
I'll say no more. I know this is well into your territory.

David



Re: PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
On 5/19/24 7:24 PM, David Rowley wrote:
> On Mon, 20 May 2024 at 09:35, Jonathan S. Katz <jkatz@postgresql.org> wrote:
>> Thanks for all the feedback to date. Please see the next revision.
>> Again, please provide feedback no later than 2024-05-22 18:00 UTC.
> 
> Thanks for the updates.
> 
>> [`COPY`](https://www.postgresql.org/docs/17/sql-copy.html) is used to efficiently bulk load data into PostgreSQL,
andwith PostgreSQL 17 shows a 2x performance improvement when loading large rows.
 
> 
> The 2x thing mentioned by Jelte is for COPY TO rather than COPY FROM.
> So I think "exporting" or "sending large rows to the client"  rather
> than "loading".

Thanks for the clarification - I've edited it as such. That also brings 
up a good point to highlight that COPY is not just for loading (since my 
bias is to do loads these days :) Now it reads:

[`COPY`](https://www.postgresql.org/docs/17/sql-copy.html) is used to 
efficiently bulk load and export data from PostgreSQL, and now with 
PostgreSQL 17 you may see up to a 2x performance improvement when 
exporting large rows.

> There's also a stray "with" in that sentence.

Thanks, fixed.

Jonathan


Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
On 5/20/24 2:58 AM, John Naylor wrote:
> Hi Jon,
> 
> Regarding vacuum "has shown up to a 6x improvement in overall time to
> complete its work" -- I believe I've seen reported numbers close to
> that only 1) when measuring the index phase in isolation or maybe 2)
> the entire vacuum of unlogged tables with one, perfectly-correlated
> index (testing has less variance with WAL out of the picture). I
> believe tables with many indexes would show a lot of improvement, but
> I'm not aware of testing that case specifically. Can you clarify where
> 6x came from?

Sawada-san showed me the original context, but I can't rapidly find it 
in the thread. Sawada-san, can you please share the numbers behind this?

We can adjust the claim - but I'd like to ensure we highlight how the 
changes to vacuum will visibly impact users.

Jonathan

Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
jian he
Date:
On Mon, May 20, 2024 at 5:35 AM Jonathan S. Katz <jkatz@postgresql.org> wrote:
>
> On 5/15/24 9:45 PM, Jonathan S. Katz wrote:
> > Hi,
> >
> > Attached is a copy of the PostgreSQL 17 Beta 1 release announcement
> > draft. This contains a user-facing summary of some of the features that
> > will be available in the Beta, as well as a call to test. I've made an
> > effort to group them logically around the different workflows they affect.
> >
> > A few notes:
> >
> > * The section with the features is not 80-char delimited. I will do that
> > before the final copy
> >
> > * There is an explicit callout that we've added in the SQL/JSON features
> > that were previously reverted in PG15. I want to ensure we're
> > transparent about that, but also use it as a hook to get people testing.
> >
> > When reviewing:
> >
> > * Please check for correctness of feature descriptions, keeping in mind
> > this is targeted for a general audience
> >
> > * Please indicate if you believe there's a notable omission, or if we
> > should omit any of these callouts
> >
> > * Please indicate if a description is confusing - I'm happy to rewrite
> > to ensure it's clearer.
> >
> > Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the
> > beta release takes some extra effort, I want to ensure all changes are
> > in with time to spare before release day.
>
> Thanks for all the feedback to date. Please see the next revision.
> Again, please provide feedback no later than 2024-05-22 18:00 UTC.
>
> Thanks,
>
> Jonathan
>

release note (https://momjian.us/pgsql_docs/release-17.html)
is
"Add jsonpath methods to convert JSON values to other JSON data types
(Jeevan Chalke)"


>> Additionally, PostgreSQL 17 adds more functionality to its `jsonpath` implementation, including the ability to
convertJSON values to different data types. 
so, I am not sure this is 100% correct.

Maybe we can rephrase it like:

>> Additionally, PostgreSQL 17 adds more functionality to its `jsonpath` implementation, including the ability to
convertJSON values to other JSON data types. 



Re: PostgreSQL 17 Beta 1 release announcement draft

From
Masahiko Sawada
Date:
On Mon, May 20, 2024 at 8:47 PM Jonathan S. Katz <jkatz@postgresql.org> wrote:
>
> On 5/20/24 2:58 AM, John Naylor wrote:
> > Hi Jon,
> >
> > Regarding vacuum "has shown up to a 6x improvement in overall time to
> > complete its work" -- I believe I've seen reported numbers close to
> > that only 1) when measuring the index phase in isolation or maybe 2)
> > the entire vacuum of unlogged tables with one, perfectly-correlated
> > index (testing has less variance with WAL out of the picture). I
> > believe tables with many indexes would show a lot of improvement, but
> > I'm not aware of testing that case specifically. Can you clarify where
> > 6x came from?
>
> Sawada-san showed me the original context, but I can't rapidly find it
> in the thread. Sawada-san, can you please share the numbers behind this?
>

I referenced the numbers that I measured during the development[1]
(test scripts are here[2]). IIRC I used unlogged tables and indexes,
and these numbers were the entire vacuum execution time including heap
scanning, index vacuuming and heap vacuuming.

FYI today I've run the same script with PG17 and measured the
execution times. Here are results:

monotonically ordered int column index:
system usage: CPU: user: 1.72 s, system: 0.47 s, elapsed: 2.20 s

uuid column index:
system usage: CPU: user: 3.62 s, system: 0.89 s, elapsed: 4.52 s

int & uuid indexes in parallel:
system usage: CPU: user: 2.24 s, system: 0.44 s, elapsed: 5.01 s

These numbers are better than ones I measured with v62 patch set as we
now introduced some optimization into tidstore (8a1b31e6 and f35bd9b).

Regards,

[1] https://www.postgresql.org/message-id/CAD21AoBci3Hujzijubomo1tdwH3XtQ9F89cTNQ4bsQijOmqnEw%40mail.gmail.com
[2] https://www.postgresql.org/message-id/CANWCAZYqWibTRCWs5mV57mLj1A0nbKX-eV5G%2Bd-KmBOGHTVY-w%40mail.gmail.com

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com



Re: PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
On 5/20/24 8:31 AM, jian he wrote:

> release note (https://momjian.us/pgsql_docs/release-17.html)
> is
> "Add jsonpath methods to convert JSON values to other JSON data types
> (Jeevan Chalke)"
> 
> 
>>> Additionally, PostgreSQL 17 adds more functionality to its `jsonpath` implementation, including the ability to
convertJSON values to different data types.
 
> so, I am not sure this is 100% correct.
> 
> Maybe we can rephrase it like:
> 
>>> Additionally, PostgreSQL 17 adds more functionality to its `jsonpath` implementation, including the ability to
convertJSON values to other JSON data types.
 

The release note goes on to state:

==
The jsonpath methods are .bigint(), .boolean(), .date(), 
.decimal([precision [, scale]]), .integer(), .number(), .string(), 
.time(), .time_tz(), .timestamp(), and .timestamp_tz().
==

And reviewing the docs[1], these are converted to a PostgreSQL native 
types, and not JSON types (additionally a bunch of those are not JSON 
types).

Jeevan: can you please confirm that this work converts into the 
PostgreSQL native types?

Thanks,

Jonathan

[1] https://www.postgresql.org/docs/devel/functions-json.html

Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
On 5/20/24 6:32 AM, David Rowley wrote:
> On Mon, 20 May 2024 at 22:11, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>>
>> On 2024-May-16, David Rowley wrote:
>>
>>> On Thu, 16 May 2024 at 17:37, zaidagilist <zaidagilist@gmail.com> wrote:
>>>> I am trying to open the 17 docs but it looks removed. Getting
>>>> following message "Page not found"
>>>>
>>>> https://www.postgresql.org/docs/17/
>>>
>>> It's called "devel" for "development" until we branch sometime before July:
>>>
>>> https://www.postgresql.org/docs/devel/
>>
>> Hmm, but that would mean that the Beta1 announce would ship full of
>> links that will remain broken until July.  I'm not sure what the
>> workflow for this is, but I hope the /17/ URLs would become valid with
>> beta1, later this week.
> 
> I didn't quite click that it was Jonathan's links that were being
> complained about.
> 
> I don't know how the website picks up where to link the doc page for a
> given version.  I see from e0b82fc8e8 that the PACKAGE_VERSION was
> changed from 16devel to 16beta1. Does the website have something that
> extracts "devel" from the former and "16" from the latter?  I see the
> release announcement for 16beta1 had /16/ links per [1].  So, I guess
> it works. I just don't know how.

The tl;dr is that the /17/ links will be available on release day. I've 
validated the current links using the /devel/ heading.

Jonathan


Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
John Naylor
Date:
On Mon, May 20, 2024 at 8:41 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Mon, May 20, 2024 at 8:47 PM Jonathan S. Katz <jkatz@postgresql.org> wrote:
> >
> > On 5/20/24 2:58 AM, John Naylor wrote:
> > > Hi Jon,
> > >
> > > Regarding vacuum "has shown up to a 6x improvement in overall time to
> > > complete its work" -- I believe I've seen reported numbers close to
> > > that only 1) when measuring the index phase in isolation or maybe 2)
> > > the entire vacuum of unlogged tables with one, perfectly-correlated
> > > index (testing has less variance with WAL out of the picture). I
> > > believe tables with many indexes would show a lot of improvement, but
> > > I'm not aware of testing that case specifically. Can you clarify where
> > > 6x came from?
> >
> > Sawada-san showed me the original context, but I can't rapidly find it
> > in the thread. Sawada-san, can you please share the numbers behind this?
> >
>
> I referenced the numbers that I measured during the development[1]
> (test scripts are here[2]). IIRC I used unlogged tables and indexes,
> and these numbers were the entire vacuum execution time including heap
> scanning, index vacuuming and heap vacuuming.

Thanks for confirming.

The wording "has a new internal data structure that reduces memory
usage and has shown up to a 6x improvement in overall time to complete
its work" is specific for runtime, and the memory use is less
specific. Unlogged tables are not the norm, so I'd be cautious of
reporting numbers specifically designed (for testing) to isolate the
thing that changed.

I'm wondering if it might be both more impressive-sounding and more
realistic for the average user experience to reverse that: specific on
memory, and less specific on speed. The best-case memory reduction
occurs for table update patterns that are highly localized, such as
the most recently inserted records, and I'd say those are a lot more
common than the use of unlogged tables.

Maybe something like "has a new internal data structure that reduces
overall time to complete its work and can use up to 20x less memory."

Now, it is true that when dead tuples are sparse and evenly spaced
(e.g. 1 every 100 pages), vacuum can now actually use more memory than
v16. However, the nature of that scenario also means that the number
of TIDs just can't get very big to begin with. In contrast, while the
runtime improvement for normal (logged) tables is likely not
earth-shattering, I believe it will always be at least somewhat
faster, and never slower.



Re: PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
On 5/20/24 5:34 AM, Bertrand Drouvot wrote:
> Hi,
> 
> On Sun, May 19, 2024 at 05:10:10PM -0400, Jonathan S. Katz wrote:
>> On 5/16/24 1:15 AM, Bertrand Drouvot wrote:
>>> Hi,
>>>
>>> On Wed, May 15, 2024 at 09:45:35PM -0400, Jonathan S. Katz wrote:
>>>> Hi,
>>>>
>>>> Attached is a copy of the PostgreSQL 17 Beta 1 release announcement draft.
>>>
>>> Thanks for working on it!
>>>
>>> I've one comment:
>>>
>>>> PostgreSQL 17 also introduces a new view,
[`pg_wait_events`](https://www.postgresql.org/docs/17/view-pg-wait-events.html),which provides descriptions about wait
eventsand can be combined with `pg_stat_activity` to give more insight into an operation.
 
>>>
>>> Instead of "to give more insight into an operation", what about "to give more
>>> insight about what a session is waiting for (should it be active)"?
>>
>> I put:
>>
>> "to give more in insight into why a session is blocked."
> 
> Thanks!
> 
>>
>> Does that work?
>>
> 
> I think using "waiting" is better (as the view is "pg_wait_events" and the
> join with pg_stat_activity would be on the "wait_event_type" and "wait_event"
> columns).
> 
> The reason I mentioned "should it be active" is because wait_event and wait_event_type
> could be non empty in pg_stat_activity while the session is not in an active state
> anymore (then not waiting).
> 
> A right query would be like the one in [1]:
> 
> "
> SELECT a.pid, a.wait_event, w.description
>    FROM pg_stat_activity a JOIN
>         pg_wait_events w ON (a.wait_event_type = w.type AND
>                              a.wait_event = w.name)
>    WHERE a.wait_event is NOT NULL and a.state = 'active';
> "
> 
> means filtering on the "active" state too, and that's what the description
> proposal I made was trying to highlight.

Thanks. As such I made it:

"which provides descriptions about wait events and can be combined with 
`pg_stat_activity` to give more insight into why an active session is 
waiting."

Jonathan

Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
On 5/19/24 6:15 PM, Erik Rijkers wrote:
> Op 5/19/24 om 23:34 schreef Jonathan S. Katz:
>> On 5/15/24 9:45 PM, Jonathan S. Katz wrote:
>>> Hi,
>>>
>>> Attached is a copy of the PostgreSQL 17 Beta 1 release announcement 
> 
> 'This release introduces adds an interface'  should be:
> 'This release adds an interface'
>     (or 'introduces'; just not both...)

Thanks; adjusted in the next copy.

Jonathan


Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
On 5/20/24 6:08 AM, Alvaro Herrera wrote:
> On 2024-May-19, Jonathan S. Katz wrote:
> 
>> ### Query and Operational Performance Improvements
> 
> In this section I'd add mention the new GUCs to control SLRU memory
> size, which is going to be a huge performance boon for cases where the
> current fixed-size buffers cause bottlenecks.  Perhaps something like
> 
> "Increase scalability of transaction, subtransaction and multixact
> shared memory buffer handling, and make their buffer sizes configurable".
> 
> I don't know if we have any published numbers of the performance
> improvement achieved, but with this patch (or ancestor ones) some
> systems go from completely unoperational to working perfectly fine.
> Maybe the best link is here
> https://www.postgresql.org/docs/devel/runtime-config-resource.html#GUC-MULTIXACT-MEMBER-BUFFERS
> though exactly which GUC affects any particular user is workload-
> dependant, so I'm not sure how best to do it.

I originally had penciled in this change, but didn't have a good way of 
describing it. The above solves that problem. I went with:

"PostgreSQL 17 also includes configuration parameters that can control 
scalability of [transaction, subtransaction and multixact 
buffers](https://www.postgresql.org/docs/devel/runtime-config-resource.html#GUC-MULTIXACT-MEMBER-BUFFERS)."

>> ### Developer Experience
> 
> I think this section should also include the libpq query cancellation
> improvements Jelte wrote.  Maybe something like "On the client side,
> PostgreSQL 17 provides better support for asynchronous and more secure
> query cancellation routines in libpq."  --> link to
> https://www.postgresql.org/docs/17/libpq-cancel.html

I went with:

PostgreSQL 17 also provides better support for [asynchronous and more 
secure query cancellation 
routines](https://www.postgresql.org/docs/17/libpq-cancel.html), which 
drivers can adopt using the libpq API.

Thanks,

Jonathan


Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
On 5/21/24 6:40 AM, John Naylor wrote:
> On Mon, May 20, 2024 at 8:41 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>>
>> On Mon, May 20, 2024 at 8:47 PM Jonathan S. Katz <jkatz@postgresql.org> wrote:
>>>
>>> On 5/20/24 2:58 AM, John Naylor wrote:
>>>> Hi Jon,
>>>>
>>>> Regarding vacuum "has shown up to a 6x improvement in overall time to
>>>> complete its work" -- I believe I've seen reported numbers close to
>>>> that only 1) when measuring the index phase in isolation or maybe 2)
>>>> the entire vacuum of unlogged tables with one, perfectly-correlated
>>>> index (testing has less variance with WAL out of the picture). I
>>>> believe tables with many indexes would show a lot of improvement, but
>>>> I'm not aware of testing that case specifically. Can you clarify where
>>>> 6x came from?
>>>
>>> Sawada-san showed me the original context, but I can't rapidly find it
>>> in the thread. Sawada-san, can you please share the numbers behind this?
>>>
>>
>> I referenced the numbers that I measured during the development[1]
>> (test scripts are here[2]). IIRC I used unlogged tables and indexes,
>> and these numbers were the entire vacuum execution time including heap
>> scanning, index vacuuming and heap vacuuming.
> 
> Thanks for confirming.
> 
> The wording "has a new internal data structure that reduces memory
> usage and has shown up to a 6x improvement in overall time to complete
> its work" is specific for runtime, and the memory use is less
> specific. Unlogged tables are not the norm, so I'd be cautious of
> reporting numbers specifically designed (for testing) to isolate the
> thing that changed.
> 
> I'm wondering if it might be both more impressive-sounding and more
> realistic for the average user experience to reverse that: specific on
> memory, and less specific on speed. The best-case memory reduction
> occurs for table update patterns that are highly localized, such as
> the most recently inserted records, and I'd say those are a lot more
> common than the use of unlogged tables.
> 
> Maybe something like "has a new internal data structure that reduces
> overall time to complete its work and can use up to 20x less memory."
> 
> Now, it is true that when dead tuples are sparse and evenly spaced
> (e.g. 1 every 100 pages), vacuum can now actually use more memory than
> v16. However, the nature of that scenario also means that the number
> of TIDs just can't get very big to begin with. In contrast, while the
> runtime improvement for normal (logged) tables is likely not
> earth-shattering, I believe it will always be at least somewhat
> faster, and never slower.

Thanks for the feedback. I flipped it around, per your suggestion:

"has a new internal data structure that has shown up to a 20x memory 
reduction for vacuum, along with improvements in overall time to 
complete its work."

Thanks,

Jonathan

Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
On 5/19/24 5:34 PM, Jonathan S. Katz wrote:
> On 5/15/24 9:45 PM, Jonathan S. Katz wrote:
>> Hi,
>>
>> Attached is a copy of the PostgreSQL 17 Beta 1 release announcement 
>> draft. This contains a user-facing summary of some of the features 
>> that will be available in the Beta, as well as a call to test. I've 
>> made an effort to group them logically around the different workflows 
>> they affect.
>>
>> A few notes:
>>
>> * The section with the features is not 80-char delimited. I will do 
>> that before the final copy
>>
>> * There is an explicit callout that we've added in the SQL/JSON 
>> features that were previously reverted in PG15. I want to ensure we're 
>> transparent about that, but also use it as a hook to get people testing.
>>
>> When reviewing:
>>
>> * Please check for correctness of feature descriptions, keeping in 
>> mind this is targeted for a general audience
>>
>> * Please indicate if you believe there's a notable omission, or if we 
>> should omit any of these callouts
>>
>> * Please indicate if a description is confusing - I'm happy to rewrite 
>> to ensure it's clearer.
>>
>> Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the 
>> beta release takes some extra effort, I want to ensure all changes are 
>> in with time to spare before release day.
> 
> Thanks for all the feedback to date. Please see the next revision. 
> Again, please provide feedback no later than 2024-05-22 18:00 UTC.

Thanks again everyone for all your feedback. Attached is the final(-ish, 
as I'll do one more readthrough before release) draft of the release 
announcement.

If you catch something and are able to post it prior to 2024-05-23 12:00 
UTC, I may be able to incorporate into the announcement.

Thanks!

Jonathan


Attachment

Re: PostgreSQL 17 Beta 1 release announcement draft

From
Bertrand Drouvot
Date:
Hi,

On Wed, May 22, 2024 at 07:01:54PM -0400, Jonathan S. Katz wrote:
> Thanks. As such I made it:
> 
> "which provides descriptions about wait events and can be combined with
> `pg_stat_activity` to give more insight into why an active session is
> waiting."
> 

Thanks! Works for me.

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com



Re: PostgreSQL 17 Beta 1 release announcement draft

From
Alvaro Herrera
Date:
Looks good.  Some minor changes:

On 2024-May-22, Jonathan S. Katz wrote:

> ### Query and Operational Performance Improvements
> 
> PostgreSQL 17 builds on recent releases and continues to improve performance across the entire system.
[Vacuum](https://www.postgresql.org/docs/17/routine-vacuuming.html),the PostgreSQL process responsible for reclaiming
storage,has a new internal data structure that has shown up to a 20x memory reduction for vacuum,
 

This reads funny:
"Vacuum ... has shown a memory reduction for vacuum"
Maybe just removing the "for vacuum" words at the end of the phrase is a
sufficient fix.

> PostgreSQL 17 can now use both planner statistics and the sort order of [common table
expressions](https://www.postgresql.org/docs/17/queries-with.html)(aka [`WITH`
queries](https://www.postgresql.org/docs/17/queries-with.html))to further
 

Is usage of "aka" typical?  I would have expected "a.k.a." but maybe I'm
just outdated.


> Finally, PostgreSQL 17 adds more explicit SIMD instructions, including AVX-512 support for the
[`bit_count](https://www.postgresql.org/docs/17/functions-bitstring.html)function.
 

Note the lack of closing backtick in [`bit_count`].

> ### Developer Experience
> 
> PostgreSQL 17 continues to build on the SQL/JSON standard, adding support for the `JSON_TABLE` features that can
convertJSON to a standard PostgreSQL table, and SQL/JSON constructor (`JSON`, `JSON_SCALAR`, `JSON_SERIALIZE`) and
queryfunctions (`JSON_EXISTS`, `JSON_QUERY`, `JSON_VALUE`). Notably, these features were originally planned for the
PostgreSQL15 release but were reverted during the beta period due to design considerations, which is one reason we ask
foryou to help us test features during beta! Additionally, PostgreSQL 17 adds more functionality to its `jsonpath`
implementation,including the ability to convert JSON values to different data types.
 

I'm not sure it's accurate to say that converting JSON values to
different datatypes is part of the jsonpath implementation; as I
understand, jsonpath is the representation used to search for elements
within JSON values.  If you replace "including" with "and", the result
seems reasonable.

> PostgreSQL 17 adds a new connection parameter, `sslnegotation`, which allows PostgreSQL to perform direct TLS
handshakeswhen using [ALPN](https://en.wikipedia.org/wiki/Application-Layer_Protocol_Negotiation), eliminating a
networkroundtrip. PostgreSQL is registered as `postgresql` in the ALPN directory.
 

Typo here "sslnegotation" missing an i, sslnegotiation.


> PostgreSQL 17 normalizes the parameters for `CALL` in
[`pg_stat_statements`](https://www.postgresql.org/docs/17/pgstatstatements.html),reducing the number of entries for
frequentlycalled stored procedures. Additionally, [`VACUUM` progress
reporting](https://www.postgresql.org/docs/devel/progress-reporting.html#VACUUM-PROGRESS-REPORTING)now shows the
progressof vacuuming indexes. PostgreSQL 17 also introduces a new view,
[`pg_wait_events`](https://www.postgresql.org/docs/17/view-pg-wait-events.html),which provides descriptions about wait
eventsand can be combined with `pg_stat_activity` to give more insight into why an active session is waiting.
Additionally,some information in the
[`pg_stat_bgwriter`](https://www.postgresql.org/docs/17/monitoring-stats.html#MONITORING-PG-STAT-BGWRITER-VIEW)view is
nowsplit out into the new
[`pg_stat_checkpointer`](https://www.postgresql.org/docs/17/monitoring-stats.html#MONITORING-PG-STAT-CHECKPOINTER-VIEW)
view.

Note use of one link to "/devel/" here.

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/
"Always assume the user will do much worse than the stupidest thing
you can imagine."                                (Julien PUYDT)



Re: PostgreSQL 17 Beta 1 release announcement draft

From
"Jonathan S. Katz"
Date:
On 5/23/24 8:00 AM, Alvaro Herrera wrote:
> Looks good.  Some minor changes:

Thanks for this - right at the deadline! :D

>> ### Query and Operational Performance Improvements
>>
>> PostgreSQL 17 builds on recent releases and continues to improve performance across the entire system.
[Vacuum](https://www.postgresql.org/docs/17/routine-vacuuming.html),the PostgreSQL process responsible for reclaiming
storage,has a new internal data structure that has shown up to a 20x memory reduction for vacuum,
 
> 
> This reads funny:
> "Vacuum ... has shown a memory reduction for vacuum"
> Maybe just removing the "for vacuum" words at the end of the phrase is a
> sufficient fix.

Fixed.

>> PostgreSQL 17 can now use both planner statistics and the sort order of [common table
expressions](https://www.postgresql.org/docs/17/queries-with.html)(aka [`WITH`
queries](https://www.postgresql.org/docs/17/queries-with.html))to further
 
> 
> Is usage of "aka" typical?  I would have expected "a.k.a." but maybe I'm
> just outdated.

Removed, just b/c I can't quickly verify this.

>> Finally, PostgreSQL 17 adds more explicit SIMD instructions, including AVX-512 support for the
[`bit_count](https://www.postgresql.org/docs/17/functions-bitstring.html)function.
 
> 
> Note the lack of closing backtick in [`bit_count`].

Fixed.

>> ### Developer Experience
>>
>> PostgreSQL 17 continues to build on the SQL/JSON standard, adding support for the `JSON_TABLE` features that can
convertJSON to a standard PostgreSQL table, and SQL/JSON constructor (`JSON`, `JSON_SCALAR`, `JSON_SERIALIZE`) and
queryfunctions (`JSON_EXISTS`, `JSON_QUERY`, `JSON_VALUE`). Notably, these features were originally planned for the
PostgreSQL15 release but were reverted during the beta period due to design considerations, which is one reason we ask
foryou to help us test features during beta! Additionally, PostgreSQL 17 adds more functionality to its `jsonpath`
implementation,including the ability to convert JSON values to different data types.
 
> 
> I'm not sure it's accurate to say that converting JSON values to
> different datatypes is part of the jsonpath implementation; as I
> understand, jsonpath is the representation used to search for elements
> within JSON values.  If you replace "including" with "and", the result
> seems reasonable.

Fixed.

>> PostgreSQL 17 adds a new connection parameter, `sslnegotation`, which allows PostgreSQL to perform direct TLS
handshakeswhen using [ALPN](https://en.wikipedia.org/wiki/Application-Layer_Protocol_Negotiation), eliminating a
networkroundtrip. PostgreSQL is registered as `postgresql` in the ALPN directory.
 
> 
> Typo here "sslnegotation" missing an i, sslnegotiation.

Fixed.

>> PostgreSQL 17 normalizes the parameters for `CALL` in
[`pg_stat_statements`](https://www.postgresql.org/docs/17/pgstatstatements.html),reducing the number of entries for
frequentlycalled stored procedures. Additionally, [`VACUUM` progress
reporting](https://www.postgresql.org/docs/devel/progress-reporting.html#VACUUM-PROGRESS-REPORTING)now shows the
progressof vacuuming indexes. PostgreSQL 17 also introduces a new view,
[`pg_wait_events`](https://www.postgresql.org/docs/17/view-pg-wait-events.html),which provides descriptions about wait
eventsand can be combined with `pg_stat_activity` to give more insight into why an active session is waiting.
Additionally,some information in the
[`pg_stat_bgwriter`](https://www.postgresql.org/docs/17/monitoring-stats.html#MONITORING-PG-STAT-BGWRITER-VIEW)view is
nowsplit out into the new
[`pg_stat_checkpointer`](https://www.postgresql.org/docs/17/monitoring-stats.html#MONITORING-PG-STAT-CHECKPOINTER-VIEW)
view.
> 
> Note use of one link to "/devel/" here.

Fixed.

Thanks!

Jonathan


Attachment