Re: RFC: Additional Directory for Extensions - Mailing list pgsql-hackers

Hi everyone,

I have finally been able to test the patch in a Kubernetes environment with CloudNativePG, thanks to Niccolò Fei and Marco Nenciarini, who created a pilot patch for CloudNativePG (https://github.com/cloudnative-pg/cloudnative-pg/pull/6546).

In the meantime, Kubernetes is likely adding the ImageVolume feature starting from the upcoming version 1.33. I will write a blog post soon about how CloudNativePG will benefit from this feature. See https://github.com/kubernetes/enhancements/issues/4639.

Although the steps are not easily reproducible by everyone, I can confirm that I successfully mounted a volume in the Postgres pod using a container image that includes only pgvector (with a size of 1.6MB - see https://github.com/EnterpriseDB/pgvector/blob/dev/5645/Dockerfile.cnpg).

By setting:

dynamic_library_path = '$libdir:/extensions/pgvector/lib'
extension_control_path = '$system:/extensions/pgvector/share'

I was able to run the following queries:

postgres=# SELECT * FROM pg_available_extensions WHERE name = 'vector';
-[ RECORD 1 ]-----+-----------------------------------------------------
name              | vector
default_version   | 0.8.0
installed_version |
comment           | vector data type and ivfflat and hnsw access methods

postgres=# SELECT * FROM pg_available_extensions WHERE name = 'vector';
-[ RECORD 1 ]-----+-----------------------------------------------------
name              | vector
default_version   | 0.8.0
installed_version |
comment           | vector data type and ivfflat and hnsw access methods

I also successfully ran the following:

postgres=# SELECT * FROM pg_extension_update_paths('vector');

By emptying the content of `extension_control_path`, the vector extension disappeared from the list.

postgres=# SHOW extension_control_path ;
 extension_control_path
------------------------
 $system
(1 row)

postgres=# SELECT * FROM pg_available_extensions WHERE name = 'vector';
 name | default_version | installed_version | comment
------+-----------------+-------------------+---------
(0 rows)

postgres=# SELECT * FROM pg_available_extension_versions WHERE name = 'vector';
 name | version | installed | superuser | trusted | relocatable | schema | requires | comment
------+---------+-----------+-----------+---------+-------------+--------+----------+---------
(0 rows)

In my opinion, the patch already helps a lot and does what I can reasonably expect from a first iteration of improvements in enabling immutable container images that ship a self-contained extension to be dynamically loaded and unloaded from a Postgres cluster in Kubernetes. From here, it is all about learning how to improve things with an exploratory mindset with future versions of Postgres and Kubernetes. It's a long journey but this is a fundamental step in the right direction.

Let me know if there's more testing for me to do. The documentation looks clear to me.

Thank you to everyone who contributed to this patch, from the initial discussions to the development phase. I sincerely hope this is included in Postgres 18.

Ciao,

Gabriele