Thread: Re: IMPORTANT: Out-of-cycle release scheduled for November 21, 2024

Re: IMPORTANT: Out-of-cycle release scheduled for November 21, 2024

From
Bruce Momjian
Date:
On Fri, Nov 15, 2024 at 06:28:42PM -0500, Jonathan Katz wrote:
> We're scheduling an out-of-cycle release on November 21, 2024 to address two
> regressions that were released as part of the November 14, 2024 update
> release[1]. As part of this release, we will issue fixes for all supported
> versions (17.2, 16.6, 15.10, 14.15, 13.20), and for 12.22, even though
> PostgreSQL 12 is now EOL.
> 
> A high-level description of the regressions are as follows.
> 
> 1. The fix for CVE-2024-10978 prevented `ALTER USER ... SET ROLE ...` from
> having any effect[2]. This will be fixed in the upcoming release.
> 
> 2. Certain PostgreSQL extensions took a dependency on an Application Build
> Interface (ABI) that was modified in this release and caused them to
> break[3]. Currently, this can be mitigated by rebuilding the extensions
> against the updated definition.
> 
> Please follow all standard guidelines for commits ahead of the release.
> Thanks for your help in assisting with this release,

I want to point out a complexity of this out-of-cycle release.  Our
17.1, etc. releases had four CVEs:

    https://www.postgresql.org/message-id/173159332163.1547975.13346191756810493274@wrigleys.postgresql.org

so when we decided to remove the downloads and encourage people to wait
for the 17.2 etc. releases, we had the known CVEs in Postgres releases
with no recommended way to fix them.

I am not sure what we could have done differently, but I am surprised we
didn't get more complaints about the security situation we put them in.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"



Re: IMPORTANT: Out-of-cycle release scheduled for November 21, 2024

From
"David G. Johnston"
Date:
On Wed, Nov 20, 2024 at 7:18 PM Bruce Momjian <bruce@momjian.us> wrote:
so when we decided to remove the downloads

Can you elaborate on who "we" is here?

I don't recall this event happening.

I suppose "encouraging people to wait" is arguably a bad position to take compared to directing them to a page on our wiki where the risk factors are laid out so they can make an informed decision based upon their situation.  But that seems like a person-to-person matter and not something the project can take responsibility for or control.  So, "immediately create a wiki page when PR-level problems arise" could be added to the "could have done better" list, so people have a URL to send instead of off-the-cuff advice.

Obviously "alter role set role" is a quite common usage in our community yet we lack any regression or tap tests exercising it.  That we could have done better and caught the bug in the CVE fix.

If the CVEs do have mitigations available those should probably be noted even if we expect people to apply the minor updates that remove the vulnerability.  If we didn't reason through and write out such mitigations for any of these 4 that would be something to consider going forward.

David J.

"David G. Johnston" <david.g.johnston@gmail.com> writes:
> On Wed, Nov 20, 2024 at 7:18 PM Bruce Momjian <bruce@momjian.us> wrote:
>> so when we decided to remove the downloads

> Can you elaborate on who "we" is here?

More to the point, what downloads were removed?  I still see the
source tarballs in the usual place [1].  If some packager(s) removed
or never posted derived packages, that's on them not the project.

            regards, tom lane

[1] https://www.postgresql.org/ftp/source/



Re: IMPORTANT: Out-of-cycle release scheduled for November 21, 2024

From
"Jonathan S. Katz"
Date:
On 11/20/24 9:18 PM, Bruce Momjian wrote:
> On Fri, Nov 15, 2024 at 06:28:42PM -0500, Jonathan Katz wrote:
>> We're scheduling an out-of-cycle release on November 21, 2024 to address two
>> regressions that were released as part of the November 14, 2024 update
>> release[1]. As part of this release, we will issue fixes for all supported
>> versions (17.2, 16.6, 15.10, 14.15, 13.20), and for 12.22, even though
>> PostgreSQL 12 is now EOL.
>>
>> A high-level description of the regressions are as follows.
>>
>> 1. The fix for CVE-2024-10978 prevented `ALTER USER ... SET ROLE ...` from
>> having any effect[2]. This will be fixed in the upcoming release.
>>
>> 2. Certain PostgreSQL extensions took a dependency on an Application Build
>> Interface (ABI) that was modified in this release and caused them to
>> break[3]. Currently, this can be mitigated by rebuilding the extensions
>> against the updated definition.
>>
>> Please follow all standard guidelines for commits ahead of the release.
>> Thanks for your help in assisting with this release,
> 
> I want to point out a complexity of this out-of-cycle release.  Our
> 17.1, etc. releases had four CVEs:
> 
>     https://www.postgresql.org/message-id/173159332163.1547975.13346191756810493274@wrigleys.postgresql.org
> 
> so when we decided to remove the downloads and encourage people to wait
> for the 17.2 etc. releases, we had the known CVEs in Postgres releases
> with no recommended way to fix them.
> 
> I am not sure what we could have done differently, but I am surprised we
> didn't get more complaints about the security situation we put them in.

The announcement[1] specified the issues and advised waiting if users 
were impacted by them directly (and tried to be as specific as possible) 
and gave guidance to prevent help users avoid upgrading and then ending 
up in a situation where they're broken, regardless if they're impacted 
by the CVE or not (e.g. they don't have PL/Perl installed).

That said, while it's certainly advisable to upgrade based on having 
CVEs in a release, many upgrade patterns are determined by the CVE 
score[2]. For example, a HIGH score (7.0 - 8.9 - our highest for this 
release was 8.8; 3 of them were less than 5.0) often dictates upgrading 
within 14-30 days of announcing the CVE, and lower scores having more 
time. This could be why people didn't complain, particularly because we 
got the announcement out 36 hours after the release, and stated the 
updates would be available within the next week.

Thanks,

Jonathan

[1] 
https://www.postgresql.org/about/news/out-of-cycle-release-scheduled-for-november-21-2024-2958/
[2] https://www.first.org/cvss/v3.1/specification-document



Re: IMPORTANT: Out-of-cycle release scheduled for November 21, 2024

From
"Jonathan S. Katz"
Date:
On 11/20/24 9:48 PM, Tom Lane wrote:
> "David G. Johnston" <david.g.johnston@gmail.com> writes:
>> On Wed, Nov 20, 2024 at 7:18 PM Bruce Momjian <bruce@momjian.us> wrote:
>>> so when we decided to remove the downloads
> 
>> Can you elaborate on who "we" is here?
> 
> More to the point, what downloads were removed?  I still see the
> source tarballs in the usual place [1].  If some packager(s) removed
> or never posted derived packages, that's on them not the project.

Downloads weren't removed, and I don't see why we'd want to do so in 
this case.

Jonathan



Re: IMPORTANT: Out-of-cycle release scheduled for November 21, 2024

From
"Jonathan S. Katz"
Date:
On 11/20/24 9:50 PM, Jonathan S. Katz wrote:
> On 11/20/24 9:48 PM, Tom Lane wrote:
>> "David G. Johnston" <david.g.johnston@gmail.com> writes:
>>> On Wed, Nov 20, 2024 at 7:18 PM Bruce Momjian <bruce@momjian.us> wrote:
>>>> so when we decided to remove the downloads
>>
>>> Can you elaborate on who "we" is here?
>>
>> More to the point, what downloads were removed?  I still see the
>> source tarballs in the usual place [1].  If some packager(s) removed
>> or never posted derived packages, that's on them not the project.
> 
> Downloads weren't removed, and I don't see why we'd want to do so in 
> this case.

Maybe here's the confusion - EDB doesn't have the downloads for the 
latest released posted on the Windows installer:

https://www.enterprisedb.com/downloads/postgres-postgresql-downloads

Jonathan



Re: IMPORTANT: Out-of-cycle release scheduled for November 21, 2024

From
Bruce Momjian
Date:
On Wed, Nov 20, 2024 at 07:40:36PM -0700, David G. Johnston wrote:
> On Wed, Nov 20, 2024 at 7:18 PM Bruce Momjian <bruce@momjian.us> wrote:
> 
>     so when we decided to remove the downloads
> 
> 
> Can you elaborate on who "we" is here?
> 
> I don't recall this event happening.

Uh, I only see 17.0 available for Windows, MacOS, and all EDB downloads,
not 17.1:

    https://www.enterprisedb.com/downloads/postgres-postgresql-downloads

I am not sure if other distributions removed 17.1.

> I suppose "encouraging people to wait" is arguably a bad position to take
> compared to directing them to a page on our wiki where the risk factors are
> laid out so they can make an informed decision based upon their situation.  But
> that seems like a person-to-person matter and not something the project can
> take responsibility for or control.  So, "immediately create a wiki page when
> PR-level problems arise" could be added to the "could have done better" list,
> so people have a URL to send instead of off-the-cuff advice.

Interesting.

> Obviously "alter role set role" is a quite common usage in our community yet we
> lack any regression or tap tests exercising it.  That we could have done better
> and caught the bug in the CVE fix.

Yes, I saw a lot of reports about this failure.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"



Re: IMPORTANT: Out-of-cycle release scheduled for November 21, 2024

From
Bruce Momjian
Date:
On Wed, Nov 20, 2024 at 09:51:09PM -0500, Jonathan Katz wrote:
> On 11/20/24 9:50 PM, Jonathan S. Katz wrote:
> > On 11/20/24 9:48 PM, Tom Lane wrote:
> > > "David G. Johnston" <david.g.johnston@gmail.com> writes:
> > > > On Wed, Nov 20, 2024 at 7:18 PM Bruce Momjian <bruce@momjian.us> wrote:
> > > > > so when we decided to remove the downloads
> > > 
> > > > Can you elaborate on who "we" is here?
> > > 
> > > More to the point, what downloads were removed?  I still see the
> > > source tarballs in the usual place [1].  If some packager(s) removed
> > > or never posted derived packages, that's on them not the project.
> > 
> > Downloads weren't removed, and I don't see why we'd want to do so in
> > this case.
> 
> Maybe here's the confusion - EDB doesn't have the downloads for the latest
> released posted on the Windows installer:
> 
> https://www.enterprisedb.com/downloads/postgres-postgresql-downloads

Yes, or for MacOS.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"



Re: IMPORTANT: Out-of-cycle release scheduled for November 21, 2024

From
Bruce Momjian
Date:
On Wed, Nov 20, 2024 at 09:49:27PM -0500, Jonathan Katz wrote:
> That said, while it's certainly advisable to upgrade based on having CVEs in
> a release, many upgrade patterns are determined by the CVE score[2]. For
> example, a HIGH score (7.0 - 8.9 - our highest for this release was 8.8; 3
> of them were less than 5.0) often dictates upgrading within 14-30 days of
> announcing the CVE, and lower scores having more time. This could be why
> people didn't complain, particularly because we got the announcement out 36
> hours after the release, and stated the updates would be available within
> the next week.

Makes sense.  This is the discussion I wanted to have.  Thanks.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"



Re: IMPORTANT: Out-of-cycle release scheduled for November 21, 2024

From
"Jonathan S. Katz"
Date:
On 11/20/24 10:08 PM, Bruce Momjian wrote:
> On Wed, Nov 20, 2024 at 09:51:09PM -0500, Jonathan Katz wrote:
>> On 11/20/24 9:50 PM, Jonathan S. Katz wrote:
>>> On 11/20/24 9:48 PM, Tom Lane wrote:
>>>> "David G. Johnston" <david.g.johnston@gmail.com> writes:
>>>>> On Wed, Nov 20, 2024 at 7:18 PM Bruce Momjian <bruce@momjian.us> wrote:
>>>>>> so when we decided to remove the downloads
>>>>
>>>>> Can you elaborate on who "we" is here?
>>>>
>>>> More to the point, what downloads were removed?  I still see the
>>>> source tarballs in the usual place [1].  If some packager(s) removed
>>>> or never posted derived packages, that's on them not the project.
>>>
>>> Downloads weren't removed, and I don't see why we'd want to do so in
>>> this case.
>>
>> Maybe here's the confusion - EDB doesn't have the downloads for the latest
>> released posted on the Windows installer:
>>
>> https://www.enterprisedb.com/downloads/postgres-postgresql-downloads
> 
> Yes, or for MacOS.

Well, why did EDB remove them? We didn't issue any guidance to remove 
downloads. We only provided guidance to users on decision making about 
whether to wait or not around the upgrade. All of the other packages 
hosted on community infrastructure (and AFAICT other OS distros) are all 
available.

Jonathan



Re: IMPORTANT: Out-of-cycle release scheduled for November 21, 2024

From
Bruce Momjian
Date:
On Wed, Nov 20, 2024 at 10:12:55PM -0500, Jonathan Katz wrote:
> On 11/20/24 10:08 PM, Bruce Momjian wrote:
> > Yes, or for MacOS.
> 
> Well, why did EDB remove them? We didn't issue any guidance to remove
> downloads. We only provided guidance to users on decision making about
> whether to wait or not around the upgrade. All of the other packages hosted
> on community infrastructure (and AFAICT other OS distros) are all available.

I don't know.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"