Thread: Question and suggestion about application binary compatibility policy
Question and suggestion about application binary compatibility policy
From
"Tsunakawa, Takayuki"
Date:
Hello, I'd like to ask you about the policy of application binary compatibility. And have a suggestion as well. QUESTION ================================================== My customer asked me if the following usage is correct. - Build an embedded SQL C application with PostgreSQL 9.2. - Distribute the app without ecpg nor libpq libraries. Require users to install PostgreSQL client which includes ecpg andlibpq libraries. - Use the app with newer PostgreSQL major versions without rebuilding the app. That is, the users just replaces the PostgreSQLclient. I expect this is legal, because the so_major versions of ecpg and libpq are 6 and 5 respectively for all PostgreSQL 9.x versionsso far. However, I could not find any description of this binary compatibility policy in the manual, so I haven'tbeen able to answer the customer. What's the official policy of application binary compatibility? I found the same discussion in the below thread, but I'mafraid any clear answer wasn't given: https://www.postgresql.org/message-id/522F0B6B.1040006@4js.com SUGGESTION ================================================== How about adding an article about application binary compatibility in the following section, as well as chapters for libpq,ECPG, etc? 17.6. Upgrading a PostgreSQL Clust https://www.postgresql.org/docs/devel/static/upgrading.html There are three kinds of application assets that users are concerned about when upgrading. Are there anything else to mention? * libpq app * ECPG app * C-language user defined function (and other stuff dependent on it, such as extensions, UDTs, etc.) Regards Takayuki Tsunakawa
Hi, > - Build an embedded SQL C application with PostgreSQL 9.2. > - Distribute the app without ecpg nor libpq libraries. Require users > to install PostgreSQL client which includes ecpg and libpq libraries. > - Use the app with newer PostgreSQL major versions without rebuilding > the app. That is, the users just replaces the PostgreSQL client. > > I expect this is legal, because the so_major versions of ecpg and > libpq are 6 and 5 respectively for all PostgreSQL 9.x versions so > far. However, I could not find any description of this binary > compatibility policy in the manual, so I haven't been able to answer > the customer. Sorry I fail to understand what you mean with "legal"? Are you wondering about license restrictions? There are none. As for compatibility, that's what major version numbers are for. This is not a PostgreSQL topic, but a general system one as for instance the same holds for libc. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Meskes at (Debian|Postgresql) dot Org Jabber: michael at xmpp dot meskes dot org VfL Borussia! Força Barça! SF 49ers! Use Debian GNU/Linux, PostgreSQL
Re: Question and suggestion about application binary compatibility policy
From
"Tsunakawa, Takayuki"
Date:
> From: pgsql-hackers-owner@postgresql.org > Sorry I fail to understand what you mean with "legal"? Are you wondering > about license restrictions? There are none. Sorry, I just meant "correct" or "valid". > As for compatibility, that's what major version numbers are for. This is > not a PostgreSQL topic, but a general system one as for instance the same > holds for libc. Yes, but Windows users probably don't understand or know it. So, I suggested explicitly describing the application binarycompatibility policy in the PostgreSQL manual. What do you think about it? Regards Takayuki Tsunakawa
> > Sorry I fail to understand what you mean with "legal"? Are you > > wondering > > about license restrictions? There are none. > > Sorry, I just meant "correct" or "valid". Ah ok, thanks for clarifying. > > As for compatibility, that's what major version numbers are for. > > This is > > not a PostgreSQL topic, but a general system one as for instance > > the same > > holds for libc. > > Yes, but Windows users probably don't understand or know it. So, I > suggested explicitly describing the application binary compatibility > policy in the PostgreSQL manual. What do you think about it? Couldn't you point your customer to the system documentation? Personally I don't think standard system behavior should be documented for each application relying on it but ymmv. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Meskes at (Debian|Postgresql) dot Org Jabber: michael at xmpp dot meskes dot org VfL Borussia! Força Barça! SF 49ers! Use Debian GNU/Linux, PostgreSQL
Re: Question and suggestion about application binary compatibility policy
From
"Tsunakawa, Takayuki"
Date:
> From: Michael Meskes [mailto:meskes@postgresql.org] > > Yes, but Windows users probably don't understand or know it. So, I > > suggested explicitly describing the application binary compatibility > > policy in the PostgreSQL manual. What do you think about it? > > Couldn't you point your customer to the system documentation? > > Personally I don't think standard system behavior should be documented for > each application relying on it but ymmv. I couldn't find appropriate system documentation. Regarding Linux, I remember I saw some HOWTO on tldp.org website whichexplains the concept of shared library soname, but it's not very friendly for users who just want to know the applicationbinary compatibility policy of PostgreSQL. And I don't think there's suitable documentation on Windows. Evenif there is any, users will not be sure whether PostgreSQL follows those platform-specific conventions. They may havedoubts about it, because even the product version "PostgreSQL x.y.z" causes misconception that x is the major versionand y is the minor one. So, I suggested documenting the compatibility policy for clarification and user friendliness as in the Oracle Database documentationbelow. http://docs.oracle.com/database/121/UPGRD/app.htm#UPGRD12547 BTW, although this may be a separate topic, it may be better that we add the major version in the library names like libpq5.dlland libecpg6.dll, so that the application can fail to run with the incompatible versions of libpq and libecpg. FYI: https://en.wikipedia.org/wiki/Side-by-side_assembly [Excerpt] Microsoft Visual C++ 2005 and 2008 employ SxS with all C runtime libraries. However, runtime libraries in Visual C++ 2010no longer use this technology; instead, they include the version number of a DLL in its file name, which means that differentversions of one DLL will technically be completely different DLLs now. Any comments on these? If there's no strong objection, I think I'll submit a documentation patch in the future. Regards Takayuki Tsunakawa
On 31/05/2016 08:10, Tsunakawa, Takayuki wrote: >> From: Michael Meskes [mailto:meskes@postgresql.org] >>> Yes, but Windows users probably don't understand or know it. So, I >>> suggested explicitly describing the application binary compatibility >>> policy in the PostgreSQL manual. What do you think about it? >> >> Couldn't you point your customer to the system documentation? >> >> Personally I don't think standard system behavior should be documented for >> each application relying on it but ymmv. > > I couldn't find appropriate system documentation. Regarding Linux, I remember I saw some HOWTO on tldp.org website whichexplains the concept of shared library soname, but it's not very friendly for users who just want to know the applicationbinary compatibility policy of PostgreSQL. And I don't think there's suitable documentation on Windows. Evenif there is any, users will not be sure whether PostgreSQL follows those platform-specific conventions. They may havedoubts about it, because even the product version "PostgreSQL x.y.z" causes misconception that x is the major versionand y is the minor one. > > So, I suggested documenting the compatibility policy for clarification and user friendliness as in the Oracle Databasedocumentation below. > > http://docs.oracle.com/database/121/UPGRD/app.htm#UPGRD12547 > > > BTW, although this may be a separate topic, it may be better that we add the major version in the library names like libpq5.dlland libecpg6.dll, so that the application can fail to run with the incompatible versions of libpq and libecpg. FYI: > > https://en.wikipedia.org/wiki/Side-by-side_assembly > > [Excerpt] > Microsoft Visual C++ 2005 and 2008 employ SxS with all C runtime libraries. However, runtime libraries in Visual C++ 2010no longer use this technology; instead, they include the version number of a DLL in its file name, which means that differentversions of one DLL will technically be completely different DLLs now. > > > Any comments on these? If there's no strong objection, I think I'll submit a documentation patch in the future. > > Regards > Takayuki Tsunakawa > Hi, on cygwin the postgresql binary package already include the library versions: /usr/bin/cygecpg-6.dll /usr/bin/cygecpg_compat-3.dll /usr/bin/cygpgtypes-3.dll /usr/bin/cygpq-5.dll attached the patch used for the build. Regards Marco
Attachment
> I couldn't find appropriate system documentation. Regarding Linux, I > remember I saw some HOWTO on tldp.org website which explains the > concept of shared library soname, but it's not very friendly for > users who just want to know the application binary compatibility > policy of PostgreSQL. And I don't think there's suitable I would expect Unix sysadmins to understand that howto, but there are others, e.g. a random hit from google: https://www.bottomupcs.com/libra ries_and_the_linker.xhtml There even is a wikipedia page about it: https://en.wikipedia.org/wiki/ Soname > documentation on Windows. Even if there is any, users will not be > sure whether PostgreSQL follows those platform-specific > conventions. They may have doubts about it, because even the product > version "PostgreSQL x.y.z" causes misconception that x is the major > version and y is the minor one. I don't know anything about the Windows setup in PostgreSQL, but I would find it fair to assume that PostgreSQL follows conventions. > BTW, although this may be a separate topic, it may be better that we > add the major version in the library names like libpq5.dll and > libecpg6.dll, so that the application can fail to run with the > incompatible versions of libpq and libecpg. FYI: Does this mean you cannot have to versions of libpq installed on the same Windows system at the same time? Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Meskes at (Debian|Postgresql) dot Org Jabber: michael at xmpp dot meskes dot org VfL Borussia! Força Barça! SF 49ers! Use Debian GNU/Linux, PostgreSQL
Re: Question and suggestion about application binary compatibility policy
From
"Tsunakawa, Takayuki"
Date:
> From: pgsql-hackers-owner@postgresql.org > [mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Marco Atzeri > on cygwin the postgresql binary package already include the library > versions: > > /usr/bin/cygecpg-6.dll > /usr/bin/cygecpg_compat-3.dll > /usr/bin/cygpgtypes-3.dll > /usr/bin/cygpq-5.dll > > attached the patch used for the build. Thanks for the information. I didn't know there's a PostgreSQL binary package for Cygwin. Regards Takayuki Tsunakawa
Re: Question and suggestion about application binary compatibility policy
From
"Tsunakawa, Takayuki"
Date:
> From: Michael Meskes [mailto:meskes@postgresql.org] > e.g. a random hit from google:=C2=A0https://www.bottomupcs.com/libra > ries_and_the_linker.xhtml > > There even is a wikipedia page about > it:=C2=A0https://en.wikipedia.org/wiki/ > Soname Thank you for good pointers. The former is particularly nice. > > BTW, although this may be a separate topic, it may be better that we > > add the major version in the library names like libpq5.dll and > > libecpg6.dll, so that the application can fail to run with the > > incompatible versions of libpq and libecpg.=C2=A0=C2=A0FYI: > > Does this mean you cannot have to versions of libpq installed on the same > Windows system at the same time? No, you can have different versions in separate folders, as in: C:\Program Files\PostgreSQL\9.2 C:\Program Files\PostgreSQL\9.5 and change the PATH environment variable to point to a newer version when you want to use the existing application withoutrebuilding it. However, the problem I pointed out is that when the new library is incompatible with the older one, say the major versionof libpq.dll changes from 5 to 6, the application user and/or developer cannot notice the incompatibility immediatelyand easily. On Unix/Linux, the application fails to start because the older library is not found. On the otherhand, the application will start on Windows and probably cause difficult trouble due to the incompatibility. Regards Takayuki Tsunakawa
On 30 May 2016 at 11:04, Tsunakawa, Takayuki <tsunakawa.takay@jp.fujitsu.com> wrote:
Hello,
I'd like to ask you about the policy of application binary compatibility. And have a suggestion as well.
QUESTION
==================================================
My customer asked me if the following usage is correct.
- Build an embedded SQL C application with PostgreSQL 9.2.
- Distribute the app without ecpg nor libpq libraries. Require users to install PostgreSQL client which includes ecpg and libpq libraries.
Why?
While that's probably OK, it's not especially desirable. The typical Windows deployment model involves the application bundling all its direct dependencies except when those are provided by a runtime redistributable designed for that purpose.
- Use the app with newer PostgreSQL major versions without rebuilding the app. That is, the users just replaces the PostgreSQL client.
... especially since there isn't a client-only PostgreSQL distribution Windows.
How about adding an article about application binary compatibility in the following section, as well as chapters for libpq, ECPG, etc?
It would be sensible to better define the binary compatibility expectations that clients may rely upon and, when they are broken, a managed way in which they're broken (soname bump, etc).
If you have an interest in the area it might be worth drafting a proposal after taking a look at past binary compatibility discussions on -hackers.
There are three kinds of application assets that users are concerned about when upgrading. Are there anything else to mention?
* libpq app
* ECPG app
* C-language user defined function (and other stuff dependent on it, such as extensions, UDTs, etc.)
- On-disk format
- Wire protocol
- SQL-level (data types, syntax, encoding handling, settings, ...)
- ...
> On the other hand, the application will start on Windows and probably cause difficult trouble due to the incompatibility.
Yeah, we just write 'libpq.dll' on Windows.
The simplest solution would be to incorporate the soname, so it becomes libpq0509.dll for example. Like VS does with the VS runtime. The main downside is that because it's not a true soname and the OS has no such concept, the linker for everything compiled against that DLL must specify the versioned DLL name, it can't just link to 'libpq' .
Re: Question and suggestion about application binary compatibility policy
From
"Tsunakawa, Takayuki"
Date:
From: pgsql-hackers-owner@postgresql.org [mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Craig Ringer
While that's probably OK, it's not especially desirable. The typical Windows deployment model involves the application bundling all its direct dependencies except when those are provided by a runtime redistributable designed for that purpose.
I think so, too, but I'm not confident that's typical. Some people may think of PostgreSQL binaries as a shared component for their applications and place it in one place, just like the PostgreSQL library package is in /usr/lib/pgsql.
- Use the app with newer PostgreSQL major versions without rebuilding the app. That is, the users just replaces the PostgreSQL client.
... especially since there isn't a client-only PostgreSQL distribution Windows.
There's a client-only installation method as follows, although I haven't checked whether EnterpriseDB, OpenSCG, or any other PostgreSQL-based products provide client-only installation.
https://www.postgresql.org/docs/devel/static/install-windows-full.html#AEN30192
[Excerpt]
--------------------------------------------------
If you want to install only the client applications and interface libraries, then you can use these commands:
install c:\destination\directory client
--------------------------------------------------
How about adding an article about application binary compatibility in the following section, as well as chapters for libpq, ECPG, etc?
It would be sensible to better define the binary compatibility expectations that clients may rely upon and, when they are broken, a managed way in which they're broken (soname bump, etc).
If you have an interest in the area it might be worth drafting a proposal after taking a look at past binary compatibility discussions on -hackers.
Sure, I'll submit a patch to pgsql-docs. Thanks to Michael's confirmation, I could answer the customer's question, so this is not an immediate task now. But I'll do.
- On-disk format
- Wire protocol
- SQL-level (data types, syntax, encoding handling, settings, ...)
Yes, I recognize these items. I omitted them because:
- On-disk format: this is handled in the PostgreSQL manual article about upgrading
- Wire protocol: I believe the compatibility will be retained
- SQL-level: ditto
But if you feel a need for their compatibility description for completeness, I'll add it. ... Yes, the explicit explanation may be necessary so that users are assured that the PostgreSQL compatibility policy matches their expectation.
The simplest solution would be to incorporate the soname, so it becomes libpq0509.dll for example. Like VS does with the VS runtime. The main downside is that because it's not a true soname and the OS has no such concept, the linker for everything compiled against that DLL must specify the versioned DLL name, it can't just link to 'libpq' .
Although I haven’t examined yet, some directive in .def file might enable applications to specify libpq.lib at build time and to link with libpq5.dll at run time.
Regards
Takayuki Tsunakawa
On 1 June 2016 at 13:09, Tsunakawa, Takayuki <tsunakawa.takay@jp.fujitsu.com> wrote:
From: pgsql-hackers-owner@postgresql.org [mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Craig Ringer
While that's probably OK, it's not especially desirable. The typical Windows deployment model involves the application bundling all its direct dependencies except when those are provided by a runtime redistributable designed for that purpose.
I think so, too, but I'm not confident that's typical. Some people may think of PostgreSQL binaries as a shared component for their applications and place it in one place, just like the PostgreSQL library package is in /usr/lib/pgsql.
(Your reply formatting seems mangled, mixing my text with yours)
/usr/lib/pgsql works because *nix systems don't typically do binary distribution.
Windows apps that rely on binary distribution should bundle the libraries they require.
Maybe a note on windows distribution in the libpq manual would be warranted. I thought it was so accepted as the way it's done that nobody would really do anything else.
(Then again, EDB's installers don't bundle their Python, Perl, etc runtimes, but I think that's partly a legal thing).
There's a client-only installation method as follows, although I haven't checked whether EnterpriseDB, OpenSCG, or any other PostgreSQL-based products provide client-only installation.
https://www.postgresql.org/docs/devel/static/install-windows-full.html#AEN30192
Right, and EDBs installers also let you install just the client AFAIK, but there's no simple client library redist package, like you'd expect if it was intended for use that way.
> However, the problem I pointed out is that when the new library is > incompatible with the older one, say the major version of libpq.dll > changes from 5 to 6, the application user and/or developer cannot > notice the incompatibility immediately and easily. On Unix/Linux, > the application fails to start because the older library is not > found. On the other hand, the application will start on Windows and > probably cause difficult trouble due to the incompatibility. I don't think this is a very good situation, but I have no idea if this can be solved. However, I'd prefer a technical solution over a documentation one. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Meskes at (Debian|Postgresql) dot Org Jabber: michael at xmpp dot meskes dot org VfL Borussia! Força Barça! SF 49ers! Use Debian GNU/Linux, PostgreSQL
On Mon, May 30, 2016 at 03:04:24AM +0000, Tsunakawa, Takayuki wrote: > Hello, > > I'd like to ask you about the policy of application binary compatibility. And have a suggestion as well. > > QUESTION > ================================================== > > My customer asked me if the following usage is correct. > > - Build an embedded SQL C application with PostgreSQL 9.2. > - Distribute the app without ecpg nor libpq libraries. Require users to install PostgreSQL client which includes ecpgand libpq libraries. > - Use the app with newer PostgreSQL major versions without rebuilding the app. That is, the users just replaces the PostgreSQLclient. > > I expect this is legal, because the so_major versions of ecpg and libpq are 6 and 5 respectively for all PostgreSQL 9.xversions so far. However, I could not find any description of this binary compatibility policy in the manual, so I haven'tbeen able to answer the customer. > > What's the official policy of application binary compatibility? I found the same discussion in the below thread, but I'mafraid any clear answer wasn't given: > > https://www.postgresql.org/message-id/522F0B6B.1040006@4js.com > > > SUGGESTION > ================================================== > > How about adding an article about application binary compatibility in the following section, as well as chapters for libpq,ECPG, etc? > > 17.6. Upgrading a PostgreSQL Clust > https://www.postgresql.org/docs/devel/static/upgrading.html > > There are three kinds of application assets that users are concerned about when upgrading. Are there anything else tomention? > > * libpq app > * ECPG app > * C-language user defined function (and other stuff dependent on it, such as extensions, UDTs, etc.) I know this thread is old but it bounced around a lot of ideas. I think there are some open questions: * Will a new application link against an older minor-version libpq? * Will an old application link against a newer minor-version libpq? I think we are all in agreement that a binary cannot run using a libpq with a different major version number. We have this text in src/tools/RELEASE_CHANGES: Major Version=============The major version number should be updated whenever the source of thelibrary changes to make itbinary incompatible. Such changes include,but are not limited to:1. Removing a public function or structure (or typedef,enum, ...)2. Modifying a public functions arguments.3. Removing a field from a public structure.4. Adding a fieldto a public structure, unless steps have beenpreviously taken to shield users from such a change, for example bysuchstructures only ever being allocated/instantiated by a libraryfunction which would give the new field a suitable defaultvalue.Adding a new function should NOT force an increase in the major versionnumber. (Packagers will see the standardminor number update and installthe new library.) When the major version is increased all applicationswhich linkto the library MUST be recompiled - this is not desirable. Whenthe major version is updated the minor version gets reset.MinorVersion=============The minor version number should be updated whenever the functionality ofthe library has changed,typically a change in source code between releaseswould mean an increase in the minor version number so long as itdoes notrequire a major version increase.Given that we make at least minor changes to our libraries in every majorPostgreSQLversion, we always bump all minor library version numbers at thestart of each development cycle as a matterof policy. This is saying running against a mismatched minor version should be fine for a binary. ecpg is a little tricker because it has knowledge of SQL data types and might not support certain features if the ecpg library isn't _run_ against the same major server version. My guess is that older ecpg libraries will run fine against newer servers, but the opposite might not be true. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: Question and suggestion about application binary compatibility policy
From
"Tsunakawa, Takayuki"
Date:
> From: Bruce Momjian [mailto:bruce@momjian.us] > We have this text in src/tools/RELEASE_CHANGES: > ... > This is saying running against a mismatched minor version should be fine > for a binary. Thanks for a good rationale. > I know this thread is old but it bounced around a lot of ideas. I think > there are some open questions: > > * Will a new application link against an older minor-version libpq? > * Will an old application link against a newer minor-version libpq? The former does not always hold true, if the application uses a new libpq function which is not in an old miner-version. But I think the backward-compatibility is enough. Regards Takayuki Tsunakawa
On Thu, Jun 23, 2016 at 06:42:57AM +0000, Tsunakawa, Takayuki wrote: > > From: Bruce Momjian [mailto:bruce@momjian.us] > > We have this text in src/tools/RELEASE_CHANGES: > > ... > > This is saying running against a mismatched minor version should be fine > > for a binary. > > Thanks for a good rationale. > > > > I know this thread is old but it bounced around a lot of ideas. I think > > there are some open questions: > > > > * Will a new application link against an older minor-version libpq? > > * Will an old application link against a newer minor-version libpq? > > The former does not always hold true, if the application uses a new libpq function which is not in an old miner-version. But I think the backward-compatibility is enough. Yes, I think that is correct, and I think that is covered in the file posted: Adding a new function should NOT force an increase in the major versionnumber. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +