Thread: pgvector 0.7.3 - New upstream version

Hello All,

pgvector has released 0.7.3 and I have update the packaging on Salsa [1] to
update the Debian package. I'd like to request a review and upload, as
cycles permit.

There was a build failure for sid/i386 in Salsa's CI pipeline. I suspect
this was caused by the new addition of gcc-14 to sid since the
problematic code was unchanged between 0.7.2 and 0.7.3. Reviewing the
console output from Salsa's pipeline for 0.7.2 [2], shows that gcc-13
was used for building 0.7.2. I was able to resolve the build failure by
conditionally adding -msse2 to PG_CFLAGS when DEB_HOST_ARCH is i386.

I have triggered a build on pgdg to verify that 0.7.3 successfully
builds on all support distributions and architectures.

Let me know if anything needs changes.

Thanks,

--Bradford

[1]: https://salsa.debian.org/postgresql/pgvector
[2]: https://salsa.debian.org/postgresql/pgvector/-/jobs/5850369

Re: Bradford Boyle
> pgvector has released 0.7.3 and I have update the packaging on Salsa [1] to
> update the Debian package. I'd like to request a review and upload, as
> cycles permit.

Hi Bradford,

thanks, uploaded!

> There was a build failure for sid/i386 in Salsa's CI pipeline. I suspect
> this was caused by the new addition of gcc-14 to sid since the
> problematic code was unchanged between 0.7.2 and 0.7.3. Reviewing the
> console output from Salsa's pipeline for 0.7.2 [2], shows that gcc-13
> was used for building 0.7.2. I was able to resolve the build failure by
> conditionally adding -msse2 to PG_CFLAGS when DEB_HOST_ARCH is i386.

Having seen how much time you had to spend on resolving this, I wonder
it it is finally time to sunset the support for 32-bit architectures
in PostgreSQL on Debian. I can't even remember when I've seen a 32-bit
cluster in the wild, and there's been zero complaints when I disabled
i386 support on apt.postgresql.org for bullseye. There is a steady
stream of extension bugs specific to 32-bit, upstreams have little way
and incentive to fix that, and we waste a lot of time for probably no
users.

Comments? Disable it all (but keep libpq5 for applications)? Continue
to build the server since it works, but disable building all
extensions?

Christoph

On 7/28/24 23:44, Christoph Berg wrote:
> 
> Having seen how much time you had to spend on resolving this, I wonder
> it it is finally time to sunset the support for 32-bit architectures
> in PostgreSQL on Debian. I can't even remember when I've seen a 32-bit
> cluster in the wild, and there's been zero complaints when I disabled
> i386 support on apt.postgresql.org for bullseye. 

Well, not quite zero [1]. It took me a while to notice because we test 
32-bit on the oldest Debian/Ubuntu and Debian 11 only because the oldest 
this summer.

> There is a steady
> stream of extension bugs specific to 32-bit, upstreams have little way
> and incentive to fix that, and we waste a lot of time for probably no
> users.
> 
> Comments? Disable it all (but keep libpq5 for applications)? Continue
> to build the server since it works, but disable building all
> extensions?

It's been years since I've had any evidence that anybody is running 
32-bit in a production environment. Probably time to drop support for it.

Regards,
David

--
[1] 

https://www.postgresql.org/message-id/flat/1b5a6c90-37ab-4a0c-818f-b2ae3a62850d%40pgmasters.net#5a75b557124a1e3523be1fd9809d04e2

On 7/29/24 16:39, Magnus Hagander wrote:
> On Sun, Jul 28, 2024 at 6:44 PM Christoph Berg <myon@debian.org 
> <mailto:myon@debian.org>> wrote:
> 
>     Re: Bradford Boyle
>      > pgvector has released 0.7.3 and I have update the packaging on
>     Salsa [1] to
>      > update the Debian package. I'd like to request a review and
>     upload, as
>      > cycles permit.
> 
>     Hi Bradford,
> 
>     thanks, uploaded!
> 
>      > There was a build failure for sid/i386 in Salsa's CI pipeline. I
>     suspect
>      > this was caused by the new addition of gcc-14 to sid since the
>      > problematic code was unchanged between 0.7.2 and 0.7.3. Reviewing the
>      > console output from Salsa's pipeline for 0.7.2 [2], shows that gcc-13
>      > was used for building 0.7.2. I was able to resolve the build
>     failure by
>      > conditionally adding -msse2 to PG_CFLAGS when DEB_HOST_ARCH is i386.
> 
>     Having seen how much time you had to spend on resolving this, I wonder
>     it it is finally time to sunset the support for 32-bit architectures
>     in PostgreSQL on Debian. I can't even remember when I've seen a 32-bit
>     cluster in the wild, and there's been zero complaints when I disabled
>     i386 support on apt.postgresql.org <http://apt.postgresql.org> for
>     bullseye. There is a steady
>     stream of extension bugs specific to 32-bit, upstreams have little way
>     and incentive to fix that, and we waste a lot of time for probably no
>     users.
> 
>     Comments? Disable it all (but keep libpq5 for applications)? Continue
>     to build the server since it works, but disable building all
>     extensions?
> 
> Isn't Raspberry Pi still used pretty frequently in 32-bit? Not that they 
> are great big PostgreSQL users, but it's not nothing.  They do their own 
> downstream I believe, but if upstream dropped postgres I'm sure so would 
> they.

Pi OS has had a 64-bit version for two years now, so I think that would 
be the way to go for anyone needing compatibility.

> That said they're also a lot less likely to use the advanced extensions 
> I would guess, so maybe a middle ground could be to provide the base 
> postgresql packages only?

This was one of the options Christoph proposed so I'm certainly OK with it.

Regards,
-David

[Dropping Andrew from Cc as this isn't pgvector-specific]

Re: David Steele
> > Isn't Raspberry Pi still used pretty frequently in 32-bit? Not that they
> > are great big PostgreSQL users, but it's not nothing.  They do their own
> > downstream I believe, but if upstream dropped postgres I'm sure so would
> > they.
> 
> Pi OS has had a 64-bit version for two years now, so I think that would be
> the way to go for anyone needing compatibility.

The first Pi were 32-bit only. I have one, but don't use PG on it. Now
the question would be, is that sort of machines reason enough to keep
supporting it? I'd tend to "no".

> > That said they're also a lot less likely to use the advanced extensions
> > I would guess, so maybe a middle ground could be to provide the base
> > postgresql packages only?
> 
> This was one of the options Christoph proposed so I'm certainly OK with it.

Right, that makes sense for the server. There have been no real 32-bit
problems there yet that I can remember.

The problem with extensions is that if we said we'd continue to
support 32-bit archs as long as they work out of the box, someone
still has to do the assessment what broke and then arrange to pull the
plug, again wasting 10 or 20 minutes of time. I'd rather prefer that
we disabled all extensions at the same time. That would also be less
confusing to users (but we assume there's none anyway).

Christoph

>> Pi OS has had a 64-bit version for two years now, so I think that would be
>> the way to go for anyone needing compatibility.

And Debian's Pi images for even longer. As long as your Postgres server doesn't need proprietary GPU-accelerated video
codecs,I strongly recommend using these. 

-nik

There are still 32-bit embedded systems that ship with PostgreSQL, but the ones (one?) I am aware of build from source
already.

As a quick reference point, I spent a few minutes looking at how the
PostgreSQL Yum Repository handles architectures and distributions. It
looks like when a repo is EOLed, they stop providing updates to any
package except PostgreSQL RPMs [1]. Additionally, it looks like RHEL 6
was the last distribution to support x86.

Looking at the current release notes for Debian trixie [2], i686 will
still be an officially supported architecture. I think the proposal to
continue building the server, but disable building all extensions for
32-bit architectures is very reasonable.

-- Bradford

[1]: https://yum.postgresql.org/news/rhel7-end-of-life/
[2]: https://www.debian.org/releases/trixie/release-notes/whats-new.en.html#supported-architectures

Re: Bradford Boyle
> As a quick reference point, I spent a few minutes looking at how the
> PostgreSQL Yum Repository handles architectures and distributions. It
> looks like when a repo is EOLed, they stop providing updates to any
> package except PostgreSQL RPMs [1]. Additionally, it looks like RHEL 6
> was the last distribution to support x86.

Ack.

> Looking at the current release notes for Debian trixie [2], i686 will
> still be an officially supported architecture. I think the proposal to
> continue building the server, but disable building all extensions for
> 32-bit architectures is very reasonable.

I'd propose this plan:

* Remove buster-pgdg from apt.pg.o (buster-lts was EOLed last month)

* Remove i386 from sid-pgdg (packages are still preserved on
  apt-archive.postgresql.org)

* Upload postgresql-17 to Debian unstable, with all architectures
  supported [*] (once rc1 is there)

* When re-uploading all extensions to Debian to transition them from
  16 to 17, disable extensions on 32-bit archs. Adding this should do
  the trick:

  Build-Depends: architecture-is-64-bit <!pkg.postgresql.32-bit>,

  That way, people could still build extension packages manually if
  they really wanted (dpkg-buildpackage -Ppkg.postgresql.32-bit).

* Remove postgresql-16 from unstable

Christoph

[*] On a side note, 32-bit powerpc did actually start to break:
https://buildd.debian.org/status/logs.php?pkg=postgresql-17&ver=17%7Ebeta2-1&arch=powerpc
I'll tell the package to ignore the regression test results there with
the next postgresql-common upload:
https://salsa.debian.org/postgresql/postgresql-common/-/commit/593fa32fa0c6d2a963a7b3b514d1d6a2423ef534

Re: Christoph Berg
> * When re-uploading all extensions to Debian to transition them from
>   16 to 17, disable extensions on 32-bit archs. Adding this should do
>   the trick:
>
>   Build-Depends: architecture-is-64-bit <!pkg.postgresql.32-bit>,
>
>   That way, people could still build extension packages manually if
>   they really wanted (dpkg-buildpackage -Ppkg.postgresql.32-bit).

Do you have script to help with re-uploading all extensions and/or
updating Build-Depends for a package? Or is it mostly a manual process?
If there is no automation, is there anything I can do to assist with
this?

-- Bradford

Re: Bradford Boyle
> Do you have script to help with re-uploading all extensions and/or
> updating Build-Depends for a package? Or is it mostly a manual process?

I have an ugly script that applies varies sed tweaks and invokes my
build/upload scripts:

https://github.com/df7cb/dotfiles/blob/master/bin/pg16

The script has gone through several PG releases and many tweaks in
there have been applied everywhere and could possibly be removed now.
I'll adjust it as I go.

> If there is no automation, is there anything I can do to assist with
> this?

Thanks for the offer, but it's probably easiest if I just run through
all the packages. Most shouldn't require any special attention anyway.

Christoph

Re: To debian-devel
> it has probably been a decade since I've last seen a PostgreSQL
> installation in the wild on a 32-bit machine. PostgreSQL itself works
> fine there, but more and more extensions are not compatible anymore,
> either deliberately, or (more common) because of subtle bugs in printf
> patterns, double precision values not fitting in a PG Datum, or other
> porting problems. Each time this happens, someone has to go digging
> what's wrong this time, while in reality, there are likely no users at
> all for this PG extension on 32-bit architectures.

A new example for this has just arrived, in pgpool2 4.5.3:

https://pgdgbuild.dus.dg-i.net/job/pgpool2-binaries/architecture=i386,distribution=sid/130/console

10:39:04 snprintf.c:409:1: error: conflicting types for ‘strchrnul’; have ‘const char *(const char *, int)’
10:39:04   409 | strchrnul(const char *s, int c)
10:39:04       | ^~~~~~~~~
10:39:04 In file included from snprintf.c:43:
10:39:04 /usr/include/string.h:286:14: note: previous declaration of ‘strchrnul’ with type ‘char *(const char *, int)’
10:39:04   286 | extern char *strchrnul (const char *__s, int __c)
10:39:04       |              ^~~~~~~~~

64-bit builds are fine.

So we'll definitely proceed with removing 32-bit extensions.

Christoph

> On pgsql-pkg-debian I proposed this plan:
> 
> * Remove buster-pgdg from apt.pg.o (buster-lts was EOLed last month)
> 
> * Remove i386 from sid-pgdg (packages are still preserved on
>   apt-archive.postgresql.org)
> 
> * Upload postgresql-17 to Debian unstable, with all architectures
>   supported [*] (once rc1 is there)

The first three steps have happened now.

buster-pgdg has been copied to apt-archive.postgresql.org; the copy on
apt.postgresql.org will not receive any updates anymore and will be
removed in three weeks with the 17.0 release.

If you still need packages for buster, point your sources.list to
https://apt-archive.postgresql.org/pub/repos/apt/ buster-pgdg main

At the same time, mantic-pgdg has been copied over, and work on
supporting oracular-pgdg has begun.

TBD:

> * When re-uploading all extensions to Debian to transition them from
>   16 to 17, disable extensions on 32-bit archs. Adding this should do
>   the trick:
> 
>   Build-Depends: architecture-is-64-bit <!pkg.postgresql.32-bit>,
> 
>   That way, people could still build extension packages manually if
>   they really wanted (dpkg-buildpackage -Ppkg.postgresql.32-bit).
> 
> * Remove postgresql-16 from unstable

Christoph

Re: To PostgreSQL in Debian
> > On pgsql-pkg-debian I proposed this plan:
> > 
> > * Remove buster-pgdg from apt.pg.o (buster-lts was EOLed last month)
> > 
> > * Remove i386 from sid-pgdg (packages are still preserved on
> >   apt-archive.postgresql.org)
> > 
> > * Upload postgresql-17 to Debian unstable, with all architectures
> >   supported [*] (once rc1 is there)
> 
> The first three steps have happened now.
> 
> buster-pgdg has been copied to apt-archive.postgresql.org; the copy on
> apt.postgresql.org will not receive any updates anymore and will be
> removed in three weeks with the 17.0 release.
> 
> If you still need packages for buster, point your sources.list to
> https://apt-archive.postgresql.org/pub/repos/apt/ buster-pgdg main
> 
> At the same time, mantic-pgdg has been copied over, and work on
> supporting oracular-pgdg has begun.
> 
> TBD:
> 
> > * When re-uploading all extensions to Debian to transition them from
> >   16 to 17, disable extensions on 32-bit archs. Adding this should do
> >   the trick:
> > 
> >   Build-Depends: architecture-is-64-bit <!pkg.postgresql.32-bit>,
> > 
> >   That way, people could still build extension packages manually if
> >   they really wanted (dpkg-buildpackage -Ppkg.postgresql.32-bit).

Buster has been removed from apt.postgresql.org now, along with mantic
and everything i386. oracular is new. And of course postgresql-17.

> > * Remove postgresql-16 from unstable

That will happen shortly.

Christoph