Thread: Redis & SQLite FDW packages

Re: pgsqlitegis@tutamail.com
> Hello, community!
> 
> I am a contibutor of https://github.com/pgspider/sqlite_fdw and https://github.com/pg-redis-fdw/redis_fdw but I am
frustratedabout deb package constructing instruments. I am ready to discuss about all details of compile process of
thisFDWs. Can anyone help me to make and publish deb packages for this FDWs or there is some guide for my case?
 

Hi Michel,

not sure which part was the hard one for you - I wrote this condensed how-to a while ago:

https://salsa.debian.org/postgresql/postgresql-common/-/blob/master/doc/postgresql-debian-packaging.md

Christoph

Re: pgsqlitegis@tutamail.com
> I have absolutely no deb package expirience. I am confused about better way of packege creating. For example SQLite
FDWsupports PostgreSQL 11..17, x86 and arm architectures at least and 2 modes - with GIS support and without GIS
support.
> For example there are possible
> pg 17 + GIS + x86
> pg 17 + GIS + arm
> pg 17 + GIS + ...
> pg 17 + noGIS + x86
> pg 17 + noGIS + arm
> pg 17 + noGIS + ...
> 
> pg 16 + GIS + x86

Is there any value in creating a separate "no GIS" variant? We usually
just enable all features.

Looping over the PG versions will be handled by pg_buildext in the
packaging toolchain. Looping over architectures is handled by invoking
that build separate on each architecture.

> MySQL FDW system of packages seems me ununderstandable and very hard.

Check any other extension then?

> By https://salsa.debian.org/postgresql/postgresql-common/-/blob/master/doc/postgresql-debian-packaging.md I think
better1st step is source code package, isn't it?
 

You create a source package (well, you create debian/ and then the
source package is built from that). From that, the binary packages are
built.

> I can prepare debian/control and something other debian/ files, but don't know anythink about PGDG apt package
buildinginfrastructure and metadata.
 

debian/ is all that is required.

Christoph

I wanted to check and see if you were still interested in packaging

sqlite_fdw and redis_fdw for Debian and Ubuntu.

I spent some time

looking at sqlite_fdw and I can help with preparing the package.

 I have

prepared an initial package that builds locally but before pushing

anything, I wanted to check and see if you have started the packaging

effort.

 If not, I can setup the package repo under the PostgreSQL team

on salsa.debian.org.

Hi Michel,

I've reviewed your branch and here is my feedback:

* d/changelog is for the Debian version of the package

Since the first packaged version of the extension is 2.5.0, the
changelog should start with that version. Older versions are not
included since they haven't been uploaded to the Debian archive.

* Potentially incorrect version format

This depends on whether the package is intended to be a native or
non-native package; see [0] for a summary of the differences between
the two. The majority of PostgreSQL Debian packages that I have
reviewed/worked on have been non-native. For a non-native package,
the Debian package version is a combination of the upstream version
(e.g., 2.5.0) and a Debian component (e.g., 2.5.0-1).

* Invalid Source and Package name

Per Debian policy [1]:

> Package names (both source and binary, see Package) must consist
> only of lower case letters (a-z), digits (0-9), plus (+) and minus
> (-) signs, and periods (.). They must be at least two characters
> long and must start with an alphanumeric character.

So the correct name for the source package would be sqlite-fdw and
for the binary package postgresql-PGVERSION-sqlite-fdw

* Missing some common Build-Depends that dh_make_pgxs includes when
creating a new source package

* Incorrect matching pattern in d/watch

The matching pattern needs to match hrefs in the web page at found
at the URL. In this case, it needs to match the tag format of
v(x.y.z).tar.gz

* Additional packages are required for running the installcheck test

I needed to include locales-all, sqlite3, and tzdata-legacy in the
tests dependencies for the test to run

* Needed to pass REGRESS_PREFIX to make when running installcheck

The latest version of PostgreSQL in Debian unstable is 17.2. When
running the installcheck target, this version was expanded into a
directory path to specify which SQL queries to run during testing
but the there is no 17.2 directory under expected. This will
probably require a change to upstream sqlite_fdw in order for the
test to successfully run on the full matrix of supported PG
versions.

I have pushed my work-in-progress (which incorporates your work) to a
personal repo on salsa.debian.org [3].

To try building the binary package localy, you can install the
devscripts package and then try running debuild -i -us -uc -b. I have my
development environment setup to build packages with sbuild [2] so I
haven't tested this. I wouldn't recommend setting up sbuild unless you
intend to really get involved with broader Debian packaging work.

Regards,
-- Bradford

[0]: https://www.debian.org/doc/debian-policy/ch-source.html
[1]: https://www.debian.org/doc/debian-policy/ch-controlfields.html#source
[2]: https://salsa.debian.org/bradfordboyle/sqlite-fdw
[3]: https://wiki.debian.org/sbuild

Hi Michel,

I have provided a some detailed comments about the files under debian/
below. However, as a high-level comment I am somewhat confused about the
layout of the repository. It looks like have both the unpacked source
package form and the original upstream source are being tracked in the
same repo. I know that it is mentioned that this is possible in the
postgresql-debian-packaging [1] documentation but I don't think I have
seen this done of additional tooling like gbp [2] which it doesn't look
like is the case here. When I attempt to build the package using either
sbuild or dpkg-buildpackge, it complains that no upstream tarball can be
found. What I've seen most commonly with packages maintained by the
Debian PostgreSQL Maintainers is that unpacked source format only is
maintained in a separate git repo (usually hosted on salsa.debian.org).
As a concrete example, the source package for pgvector is maintained at
https://salsa.debian.org/postgresql/pgvector and it only contains the
content of the debian/ directory. The upstream source tar archive is
downloaded from the GitHub repo for pgvector
https://github.com/pgvector/pgvector. This downloading can be done
wither manually or with uscan.

Assuming the desired end-goal is that the sqlite_fdw is available for
download and installation from the PGDG archive for all supported
versions of PostgreSQL, these are the steps that need to be done:

1. File an intent-to-package (ITP) bug [3] with bugs.debian.org
    * This will be the source of the bug number that will be referenced
      in the d/changelog line "Initial release. (Closes: #XXXXXX)"
2. Set up a Git repo on salsa.debian.org for maintaining the unpacked
   source package form and get that passing Salsa's standard CI/CD
   pipeline
    * Most the work creating the unpacked source package form is done
      and hosting on s.d.o is not strictly required but follows the
      convention most other PostgreSQL packages are following. Passing
      Salsa's standard CI/CD pipeline will give a good indication of
      whether the package will successfully upload to the Debian archive
3. Add configuration to pgapt [4] to configure the PGDG archive to
   build and publish the package for all supported Debian/Ubuntu
   distributions and all supported PostgreSQL versions


1. d/NEWS
    * You do not need the asterisk per [5]
2. d/changelog
    * should be UNRELEASED since the package has not been uploaded to
      the Debian archive yet. In my experience as a non-uploading team
      member, this is changed to unstable when the uploader uploads to
      the archive. It is set to unstable because that's the distribution
      that the upload is going to; it will automatically enter testing
      based on a set of criteria outlined in [6]. These terms are
      referring to the Debian distribution and not a reflection of the
      state of the upstream project. When the package is re-uploaded to
      the PGDG repos, a new changelog entry will be automatically
      appended with the appropriate distribution substituted.

    * The content of d/changelog is the changelog for the Debian source
      package not the upstream project -- I would not include content
      from any of the releases prior to the initial release version but
      this is my opinion/preference and not hard technical requirement.

3. d/control
    * The Maintainer field  should be set to the package maintainer's
      name and email address [7]
    * The Maintainer field is currently just as "Taiga Katayama" but
      this field is intended for the individual(s) who are maintaining
      the Debian package, not necessarily the upstream maintainer. This
      could either be set to yourself or to the Debian PostgreSQL
      Maintainers. Either way, you will most likely need to also include
      an Uploaders field which includes the name and email address of a
      uploader. [8]
    * There should only be one binary package for PostgreSQL 17 listed
      in d/control. The Debian archive only has one version of the
      PostgreSQL server packages available which for unstable is
      currently 17. Including the other versions and their build
      dependencies in the source package will fail to build when
      uploading to the archive. The archive hosted at PGDG will use
      d/control.in to re-build the package for all PostgreSQL versions
      available in the PGDG archive.
4. d/copyright
    * I am unsure about the use of 'PostgreSQL_like' in the License
      field. My overly cautious reading of [9] indicates that this field
      should use a standard short name and this name is not referenced
      in either [9] or [10]. Based on a quick reading of [11], I think
      the short name should be either 'PostgreSQL License' or
      'PostgreSQL'. However, I am not an expert on copyright matters and
      this is my opinion informed by the referenced links.

5. d/tests
    * I anticipate that the autopkgtests will fail as-is when building
      the packages on PGDG because of the hard-coding of the minor
      version in the paths to the SQL queries. I have a patch that uses
      wildcards to match the PostgreSQL major version to the appropriate
      tests.


[1]: https://salsa.debian.org/postgresql/postgresql-common/-/blob/master/doc/postgresql-debian-packaging.md
[2]: https://wiki.debian.org/PackagingWithGit
[3]: https://wiki.debian.org/ITP
[4]: https://git.postgresql.org/gitweb/?p=pgapt.git;a=summary
[5]:
https://www.debian.org/doc/manuals/developers-reference/best-pkging-practices.en.html#supplementing-changelogs-with-news-debian-files
[6]: https://wiki.debian.org/DebianTesting
[7]: https://www.debian.org/doc/debian-policy/ch-controlfields.html#maintainer
[8]: https://www.debian.org/doc/debian-policy/ch-binary.html#s-maintainer
[9]: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/#license-short-name
[10]: https://spdx.org/licenses/
[11]: https://spdx.org/licenses/PostgreSQL.html