Thread: Does psycopg2 support Kerberos for Postgres?
Dear Psycopg Dev,
Not sure if this is the right channel but thought to give it a try anyway.
This is Yang Gao from Two Sigma. We are trying to use your driver along with SqlAlchemy to access a Postgres DB with Kerberos support and failed to make it work. We have done quite some googling but found almost not relevant info. The driver does not seem to recognize the “krbsrvname” or “gsslib” param suggested by Postgres doc.
We were wondering if Kerberos is supported at all by the driver? And if yes, is there any documentation or example?
Thank you very much.
Yang
On 12/02/2016 10:46 AM, Yang Gao wrote: > Dear Psycopg Dev, > > > > Not sure if this is the right channel but thought to give it a try anyway. > > > > This is Yang Gao from Two Sigma. We are trying to use your driver > along with SqlAlchemy to access a Postgres DB with Kerberos support and > failed to make it work. We have done quite some googling but found > almost not relevant info. The driver does not seem to recognize the > “krbsrvname” or “gsslib” param suggested by Postgres doc. What is the actual connection string you are sending? > > > > We were wondering if Kerberos is supported at all by the driver? And > if yes, is there any documentation or example? > > > > Thank you very much. > > > > Yang > > > -- Adrian Klaver adrian.klaver@aklaver.com
On 12/02/2016 10:46 AM, Yang Gao wrote: > Dear Psycopg Dev, > > > > Not sure if this is the right channel but thought to give it a try anyway. > > > > This is Yang Gao from Two Sigma. We are trying to use your driver > along with SqlAlchemy to access a Postgres DB with Kerberos support and > failed to make it work. We have done quite some googling but found > almost not relevant info. The driver does not seem to recognize the > “krbsrvname” or “gsslib” param suggested by Postgres doc. > > > > We were wondering if Kerberos is supported at all by the driver? And > if yes, is there any documentation or example? Second question. Was your Postgres database built with the proper support?: https://www.postgresql.org/docs/9.5/static/auth-methods.html GSSAPI is an industry-standard protocol for secure authentication defined in RFC 2743. PostgreSQL supports GSSAPI with Kerberos authentication according to RFC 1964. GSSAPI provides automatic authentication (single sign-on) for systems that support it. The authentication itself is secure, but the data sent over the database connection will be sent unencrypted unless SSL is used. GSSAPI support has to be enabled when PostgreSQL is built; see Chapter 15 for more information. > > > > Thank you very much. > > > > Yang > > > -- Adrian Klaver adrian.klaver@aklaver.com
Hi, Adrian,
Thanks for the reply.
The Postgres server is Kerberos enabled, as I can log in using either psql CLI, or pyodbc lib, or pg_admin client UI
withoutpassword.
With the bare bone code as below, when I run against the localhost, I am able to log in. However, it fails if it's
theremote host. The error says GSSAPI authentication not supported as below.
Please advise.
Thank you.
Yang
----
Traceback (most recent call last):
File "/home/yangg/IdeaProjects/PyPlayground/psycopg2_test.py", line 15, in <module>
conn = psycopg2.connect(conn_string)
File "/home/yangg/.conda/envs/py2/lib/python2.7/site-packages/psycopg2/__init__.py", line 164, in connect
conn = _connect(dsn, connection_factory=connection_factory, async=async)
psycopg2.OperationalError: GSSAPI authentication not supported
----
from pprint import pprint as p
import psycopg2
import os
conn_string = "host='remote_host_XXX' dbname='table_YYY'"
conn = psycopg2.connect(conn_string)
cursor = conn.cursor()
cursor.execute("SELECT version()")
records = cursor.fetchall()
p(records)
----
-----Original Message-----
From: Adrian Klaver [mailto:adrian.klaver@aklaver.com]
Sent: Saturday, December 03, 2016 5:53 PM
To: Yang Gao; 'psycopg@postgresql.org'
Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres?
On 12/02/2016 10:46 AM, Yang Gao wrote:
> Dear Psycopg Dev,
>
>
>
> Not sure if this is the right channel but thought to give it a try anyway.
>
>
>
> This is Yang Gao from Two Sigma. We are trying to use your driver
> along with SqlAlchemy to access a Postgres DB with Kerberos support
> and failed to make it work. We have done quite some googling but found
> almost not relevant info. The driver does not seem to recognize the
> "krbsrvname" or "gsslib" param suggested by Postgres doc.
>
>
>
> We were wondering if Kerberos is supported at all by the driver? And
> if yes, is there any documentation or example?
Second question. Was your Postgres database built with the proper support?:
https://www.postgresql.org/docs/9.5/static/auth-methods.html
GSSAPI is an industry-standard protocol for secure authentication defined in RFC 2743. PostgreSQL supports GSSAPI with
Kerberosauthentication according to RFC 1964. GSSAPI provides automatic authentication (single sign-on) for systems
thatsupport it. The authentication itself is secure, but the data sent over the database connection will be sent
unencryptedunless SSL is used.
GSSAPI support has to be enabled when PostgreSQL is built; see Chapter
15 for more information.
>
>
>
> Thank you very much.
>
>
>
> Yang
>
>
>
--
Adrian Klaver
adrian.klaver@aklaver.com
On 12/05/2016 08:38 AM, Yang Gao wrote: > Hi, Adrian, > > Thanks for the reply. > > The Postgres server is Kerberos enabled, as I can log in using either psql CLI, or pyodbc lib, or pg_admin client UIwithout password. I assume both the server at localhost and the remote_host you mention below have Kerberos enabled. > > With the bare bone code as below, when I run against the localhost, I am able to log in. However, it fails if it's theremote host. The error says GSSAPI authentication not supported as below. So does that apply to the other clients also, they can log in locally but not remotely? Or can they log in to the remote server using GSSAPI? > Please advise. > > Thank you. > > Yang > ---- > > Traceback (most recent call last): > File "/home/yangg/IdeaProjects/PyPlayground/psycopg2_test.py", line 15, in <module> > conn = psycopg2.connect(conn_string) > File "/home/yangg/.conda/envs/py2/lib/python2.7/site-packages/psycopg2/__init__.py", line 164, in connect > conn = _connect(dsn, connection_factory=connection_factory, async=async) > psycopg2.OperationalError: GSSAPI authentication not supported > > ---- > > from pprint import pprint as p > import psycopg2 > import os > > conn_string = "host='remote_host_XXX' dbname='table_YYY'" I don't use Kerberos but I gotta believe the below is important: https://www.postgresql.org/docs/9.5/static/libpq-connect.html krbsrvname Kerberos service name to use when authenticating with GSSAPI. This must match the service name specified in the server configuration for Kerberos authentication to succeed. (See also Section 19.3.3.) https://www.postgresql.org/docs/9.5/static/auth-methods.html#GSSAPI-AUTH When GSSAPI uses Kerberos, it uses a standard principal in the format servicename/hostname@realm. The PostgreSQL server will accept any principal that is included in the keytab used by the server, but care needs to be taken to specify the correct principal details when making the connection from the client using the krbsrvname connection parameter. (See also Section 31.1.2.) The installation default can be changed from the default postgres at build time using ./configure --with-krb-srvnam=whatever. In most environments, this parameter never needs to be changed. Some Kerberos implementations might require a different service name, such as Microsoft Active Directory which requires the service name to be in upper case (POSTGRES). > conn = psycopg2.connect(conn_string) > cursor = conn.cursor() > cursor.execute("SELECT version()") > records = cursor.fetchall() > p(records) > > ---- > -----Original Message----- > From: Adrian Klaver [mailto:adrian.klaver@aklaver.com] > Sent: Saturday, December 03, 2016 5:53 PM > To: Yang Gao; 'psycopg@postgresql.org' > Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres? > > On 12/02/2016 10:46 AM, Yang Gao wrote: >> Dear Psycopg Dev, >> >> >> >> Not sure if this is the right channel but thought to give it a try anyway. >> >> >> >> This is Yang Gao from Two Sigma. We are trying to use your driver >> along with SqlAlchemy to access a Postgres DB with Kerberos support >> and failed to make it work. We have done quite some googling but found >> almost not relevant info. The driver does not seem to recognize the >> "krbsrvname" or "gsslib" param suggested by Postgres doc. >> >> >> >> We were wondering if Kerberos is supported at all by the driver? And >> if yes, is there any documentation or example? > > Second question. Was your Postgres database built with the proper support?: > > https://www.postgresql.org/docs/9.5/static/auth-methods.html > > GSSAPI is an industry-standard protocol for secure authentication defined in RFC 2743. PostgreSQL supports GSSAPI withKerberos authentication according to RFC 1964. GSSAPI provides automatic authentication (single sign-on) for systemsthat support it. The authentication itself is secure, but the data sent over the database connection will be sentunencrypted unless SSL is used. > > GSSAPI support has to be enabled when PostgreSQL is built; see Chapter > 15 for more information. > >> >> >> >> Thank you very much. >> >> >> >> Yang >> >> >> > > > -- > Adrian Klaver > adrian.klaver@aklaver.com > -- Adrian Klaver adrian.klaver@aklaver.com
Logging in a local host w/o password is fine for psycopg2. Not sure if it goes though GSSAPI though, but the remote
serverdefinitely has GSSAPI enabled.
I did try the krbsrvname option, but got a complaint from _connect:
psycopg2.OperationalError: invalid connection option "krbsrvname"
Thanks.
Yang
-----Original Message-----
From: Adrian Klaver [mailto:adrian.klaver@aklaver.com]
Sent: Monday, December 05, 2016 11:57 AM
To: Yang Gao; 'psycopg@postgresql.org'
Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres?
On 12/05/2016 08:38 AM, Yang Gao wrote:
> Hi, Adrian,
>
> Thanks for the reply.
>
> The Postgres server is Kerberos enabled, as I can log in using either psql CLI, or pyodbc lib, or pg_admin client
UIwithout password.
I assume both the server at localhost and the remote_host you mention below have Kerberos enabled.
>
> With the bare bone code as below, when I run against the localhost, I am able to log in. However, it fails if it's
theremote host. The error says GSSAPI authentication not supported as below.
So does that apply to the other clients also, they can log in locally but not remotely?
Or can they log in to the remote server using GSSAPI?
> Please advise.
>
> Thank you.
>
> Yang
> ----
>
> Traceback (most recent call last):
> File "/home/yangg/IdeaProjects/PyPlayground/psycopg2_test.py", line 15, in <module>
> conn = psycopg2.connect(conn_string)
> File "/home/yangg/.conda/envs/py2/lib/python2.7/site-packages/psycopg2/__init__.py", line 164, in connect
> conn = _connect(dsn, connection_factory=connection_factory,
> async=async)
> psycopg2.OperationalError: GSSAPI authentication not supported
>
> ----
>
> from pprint import pprint as p
> import psycopg2
> import os
>
> conn_string = "host='remote_host_XXX' dbname='table_YYY'"
I don't use Kerberos but I gotta believe the below is important:
https://www.postgresql.org/docs/9.5/static/libpq-connect.html
krbsrvname
Kerberos service name to use when authenticating with GSSAPI. This must match the service name specified in the
serverconfiguration for Kerberos authentication to succeed. (See also Section 19.3.3.)
https://www.postgresql.org/docs/9.5/static/auth-methods.html#GSSAPI-AUTH
When GSSAPI uses Kerberos, it uses a standard principal in the format servicename/hostname@realm. The PostgreSQL server
willaccept any principal that is included in the keytab used by the server, but care needs to be taken to specify the
correctprincipal details when making the connection from the client using the krbsrvname connection parameter. (See
alsoSection 31.1.2.) The installation default can be changed from the default postgres at build time using ./configure
--with-krb-srvnam=whatever.In most environments, this parameter never needs to be changed. Some Kerberos
implementationsmight require a different service name, such as Microsoft Active Directory which requires the service
nameto be in upper case (POSTGRES).
> conn = psycopg2.connect(conn_string)
> cursor = conn.cursor()
> cursor.execute("SELECT version()")
> records = cursor.fetchall()
> p(records)
>
> ----
> -----Original Message-----
> From: Adrian Klaver [mailto:adrian.klaver@aklaver.com]
> Sent: Saturday, December 03, 2016 5:53 PM
> To: Yang Gao; 'psycopg@postgresql.org'
> Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres?
>
> On 12/02/2016 10:46 AM, Yang Gao wrote:
>> Dear Psycopg Dev,
>>
>>
>>
>> Not sure if this is the right channel but thought to give it a try anyway.
>>
>>
>>
>> This is Yang Gao from Two Sigma. We are trying to use your driver
>> along with SqlAlchemy to access a Postgres DB with Kerberos support
>> and failed to make it work. We have done quite some googling but
>> found almost not relevant info. The driver does not seem to recognize
>> the "krbsrvname" or "gsslib" param suggested by Postgres doc.
>>
>>
>>
>> We were wondering if Kerberos is supported at all by the driver?
>> And if yes, is there any documentation or example?
>
> Second question. Was your Postgres database built with the proper support?:
>
> https://www.postgresql.org/docs/9.5/static/auth-methods.html
>
> GSSAPI is an industry-standard protocol for secure authentication defined in RFC 2743. PostgreSQL supports GSSAPI
withKerberos authentication according to RFC 1964. GSSAPI provides automatic authentication (single sign-on) for
systemsthat support it. The authentication itself is secure, but the data sent over the database connection will be
sentunencrypted unless SSL is used.
>
> GSSAPI support has to be enabled when PostgreSQL is built; see Chapter
> 15 for more information.
>
>>
>>
>>
>> Thank you very much.
>>
>>
>>
>> Yang
>>
>>
>>
>
>
> --
> Adrian Klaver
> adrian.klaver@aklaver.com
>
--
Adrian Klaver
adrian.klaver@aklaver.com
On 12/05/2016 10:17 AM, Yang Gao wrote: > Logging in a local host w/o password is fine for psycopg2. Not sure if it goes though GSSAPI though, but the remote serverdefinitely has GSSAPI enabled. Best guess is that the local connections are being done via a socket and pg_hba.conf is letting those connections in under trust auth: https://www.postgresql.org/docs/9.5/static/client-authentication.html Remember in pg_hba.conf first match wins. You might want to look in this file on both the local and remote servers > > I did try the krbsrvname option, but got a complaint from _connect: > psycopg2.OperationalError: invalid connection option "krbsrvname" So what was your connection string in the code? > > Thanks. > > Yang > > -----Original Message----- > From: Adrian Klaver [mailto:adrian.klaver@aklaver.com] > Sent: Monday, December 05, 2016 11:57 AM > To: Yang Gao; 'psycopg@postgresql.org' > Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres? > > On 12/05/2016 08:38 AM, Yang Gao wrote: >> Hi, Adrian, >> >> Thanks for the reply. >> >> The Postgres server is Kerberos enabled, as I can log in using either psql CLI, or pyodbc lib, or pg_admin client UIwithout password. > > I assume both the server at localhost and the remote_host you mention below have Kerberos enabled. > >> >> With the bare bone code as below, when I run against the localhost, I am able to log in. However, it fails if it's theremote host. The error says GSSAPI authentication not supported as below. > > So does that apply to the other clients also, they can log in locally but not remotely? > > Or can they log in to the remote server using GSSAPI? > >> Please advise. >> >> Thank you. >> >> Yang >> ---- >> >> Traceback (most recent call last): >> File "/home/yangg/IdeaProjects/PyPlayground/psycopg2_test.py", line 15, in <module> >> conn = psycopg2.connect(conn_string) >> File "/home/yangg/.conda/envs/py2/lib/python2.7/site-packages/psycopg2/__init__.py", line 164, in connect >> conn = _connect(dsn, connection_factory=connection_factory, >> async=async) >> psycopg2.OperationalError: GSSAPI authentication not supported >> >> ---- >> >> from pprint import pprint as p >> import psycopg2 >> import os >> >> conn_string = "host='remote_host_XXX' dbname='table_YYY'" > > I don't use Kerberos but I gotta believe the below is important: > > https://www.postgresql.org/docs/9.5/static/libpq-connect.html > > krbsrvname > > Kerberos service name to use when authenticating with GSSAPI. This must match the service name specified in the serverconfiguration for Kerberos authentication to succeed. (See also Section 19.3.3.) > > https://www.postgresql.org/docs/9.5/static/auth-methods.html#GSSAPI-AUTH > > When GSSAPI uses Kerberos, it uses a standard principal in the format servicename/hostname@realm. The PostgreSQL serverwill accept any principal that is included in the keytab used by the server, but care needs to be taken to specifythe correct principal details when making the connection from the client using the krbsrvname connection parameter.(See also Section 31.1.2.) The installation default can be changed from the default postgres at build time using./configure --with-krb-srvnam=whatever. In most environments, this parameter never needs to be changed. Some Kerberosimplementations might require a different service name, such as Microsoft Active Directory which requires the servicename to be in upper case (POSTGRES). > > > >> conn = psycopg2.connect(conn_string) >> cursor = conn.cursor() >> cursor.execute("SELECT version()") >> records = cursor.fetchall() >> p(records) >> >> ---- >> -----Original Message----- >> From: Adrian Klaver [mailto:adrian.klaver@aklaver.com] >> Sent: Saturday, December 03, 2016 5:53 PM >> To: Yang Gao; 'psycopg@postgresql.org' >> Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres? >> >> On 12/02/2016 10:46 AM, Yang Gao wrote: >>> Dear Psycopg Dev, >>> >>> >>> >>> Not sure if this is the right channel but thought to give it a try anyway. >>> >>> >>> >>> This is Yang Gao from Two Sigma. We are trying to use your driver >>> along with SqlAlchemy to access a Postgres DB with Kerberos support >>> and failed to make it work. We have done quite some googling but >>> found almost not relevant info. The driver does not seem to recognize >>> the "krbsrvname" or "gsslib" param suggested by Postgres doc. >>> >>> >>> >>> We were wondering if Kerberos is supported at all by the driver? >>> And if yes, is there any documentation or example? >> >> Second question. Was your Postgres database built with the proper support?: >> >> https://www.postgresql.org/docs/9.5/static/auth-methods.html >> >> GSSAPI is an industry-standard protocol for secure authentication defined in RFC 2743. PostgreSQL supports GSSAPI withKerberos authentication according to RFC 1964. GSSAPI provides automatic authentication (single sign-on) for systemsthat support it. The authentication itself is secure, but the data sent over the database connection will be sentunencrypted unless SSL is used. >> >> GSSAPI support has to be enabled when PostgreSQL is built; see Chapter >> 15 for more information. >> >>> >>> >>> >>> Thank you very much. >>> >>> >>> >>> Yang >>> >>> >>> >> >> >> -- >> Adrian Klaver >> adrian.klaver@aklaver.com >> > > > -- > Adrian Klaver > adrian.klaver@aklaver.com > -- Adrian Klaver adrian.klaver@aklaver.com
On 12/06/2016 07:20 AM, Yang Gao wrote:
> You are most likely correct about local log in.
>
> This is the string that I'm using:
> sqlalchemy.create_engine("postgresql+psycopg2://fqdn_of_host/db_name")
> I tried adding "? krbsrvname=postgres" at the end as well, but got the error aforementioned.
It would have been nice to know you where using SQLAlchemy above
psycopg2 at the beginning of this conversation.
So again can you show the actual complete string you are using?
Not sure if this a cut and paste error but you have:
? krbsrvname=postgres
Pretty sure the space between ? and krbsrvname is not good.
Also what happens if you bypass SQLAlchemy and try to connect using
psycopg2 directly?
>
> Thanks.
>
> Yang
>
>
--
Adrian Klaver
adrian.klaver@aklaver.com
You are most likely correct about local log in.
This is the string that I'm using:
sqlalchemy.create_engine("postgresql+psycopg2://fqdn_of_host/db_name")
I tried adding "? krbsrvname=postgres" at the end as well, but got the error aforementioned.
Thanks.
Yang
-----Original Message-----
From: Adrian Klaver [mailto:adrian.klaver@aklaver.com]
Sent: Monday, December 05, 2016 3:49 PM
To: Yang Gao; 'psycopg@postgresql.org'
Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres?
On 12/05/2016 10:17 AM, Yang Gao wrote:
> Logging in a local host w/o password is fine for psycopg2. Not sure if it goes though GSSAPI though, but the remote
serverdefinitely has GSSAPI enabled.
Best guess is that the local connections are being done via a socket and pg_hba.conf is letting those connections in
undertrust auth:
https://www.postgresql.org/docs/9.5/static/client-authentication.html
Remember in pg_hba.conf first match wins. You might want to look in this file on both the local and remote servers
>
> I did try the krbsrvname option, but got a complaint from _connect:
> psycopg2.OperationalError: invalid connection option "krbsrvname"
So what was your connection string in the code?
>
> Thanks.
>
> Yang
>
> -----Original Message-----
> From: Adrian Klaver [mailto:adrian.klaver@aklaver.com]
> Sent: Monday, December 05, 2016 11:57 AM
> To: Yang Gao; 'psycopg@postgresql.org'
> Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres?
>
> On 12/05/2016 08:38 AM, Yang Gao wrote:
>> Hi, Adrian,
>>
>> Thanks for the reply.
>>
>> The Postgres server is Kerberos enabled, as I can log in using either psql CLI, or pyodbc lib, or pg_admin client
UIwithout password.
>
> I assume both the server at localhost and the remote_host you mention below have Kerberos enabled.
>
>>
>> With the bare bone code as below, when I run against the localhost, I am able to log in. However, it fails if it's
theremote host. The error says GSSAPI authentication not supported as below.
>
> So does that apply to the other clients also, they can log in locally but not remotely?
>
> Or can they log in to the remote server using GSSAPI?
>
>> Please advise.
>>
>> Thank you.
>>
>> Yang
>> ----
>>
>> Traceback (most recent call last):
>> File "/home/yangg/IdeaProjects/PyPlayground/psycopg2_test.py", line 15, in <module>
>> conn = psycopg2.connect(conn_string)
>> File "/home/yangg/.conda/envs/py2/lib/python2.7/site-packages/psycopg2/__init__.py", line 164, in connect
>> conn = _connect(dsn, connection_factory=connection_factory,
>> async=async)
>> psycopg2.OperationalError: GSSAPI authentication not supported
>>
>> ----
>>
>> from pprint import pprint as p
>> import psycopg2
>> import os
>>
>> conn_string = "host='remote_host_XXX' dbname='table_YYY'"
>
> I don't use Kerberos but I gotta believe the below is important:
>
> https://www.postgresql.org/docs/9.5/static/libpq-connect.html
>
> krbsrvname
>
> Kerberos service name to use when authenticating with GSSAPI.
> This must match the service name specified in the server configuration
> for Kerberos authentication to succeed. (See also Section 19.3.3.)
>
> https://www.postgresql.org/docs/9.5/static/auth-methods.html#GSSAPI-AU
> TH
>
> When GSSAPI uses Kerberos, it uses a standard principal in the format servicename/hostname@realm. The PostgreSQL
serverwill accept any principal that is included in the keytab used by the server, but care needs to be taken to
specifythe correct principal details when making the connection from the client using the krbsrvname connection
parameter.(See also Section 31.1.2.) The installation default can be changed from the default postgres at build time
using./configure --with-krb-srvnam=whatever. In most environments, this parameter never needs to be changed. Some
Kerberosimplementations might require a different service name, such as Microsoft Active Directory which requires the
servicename to be in upper case (POSTGRES).
>
>
>
>> conn = psycopg2.connect(conn_string)
>> cursor = conn.cursor()
>> cursor.execute("SELECT version()")
>> records = cursor.fetchall()
>> p(records)
>>
>> ----
>> -----Original Message-----
>> From: Adrian Klaver [mailto:adrian.klaver@aklaver.com]
>> Sent: Saturday, December 03, 2016 5:53 PM
>> To: Yang Gao; 'psycopg@postgresql.org'
>> Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres?
>>
>> On 12/02/2016 10:46 AM, Yang Gao wrote:
>>> Dear Psycopg Dev,
>>>
>>>
>>>
>>> Not sure if this is the right channel but thought to give it a try anyway.
>>>
>>>
>>>
>>> This is Yang Gao from Two Sigma. We are trying to use your driver
>>> along with SqlAlchemy to access a Postgres DB with Kerberos support
>>> and failed to make it work. We have done quite some googling but
>>> found almost not relevant info. The driver does not seem to
>>> recognize the "krbsrvname" or "gsslib" param suggested by Postgres doc.
>>>
>>>
>>>
>>> We were wondering if Kerberos is supported at all by the driver?
>>> And if yes, is there any documentation or example?
>>
>> Second question. Was your Postgres database built with the proper support?:
>>
>> https://www.postgresql.org/docs/9.5/static/auth-methods.html
>>
>> GSSAPI is an industry-standard protocol for secure authentication defined in RFC 2743. PostgreSQL supports GSSAPI
withKerberos authentication according to RFC 1964. GSSAPI provides automatic authentication (single sign-on) for
systemsthat support it. The authentication itself is secure, but the data sent over the database connection will be
sentunencrypted unless SSL is used.
>>
>> GSSAPI support has to be enabled when PostgreSQL is built; see
>> Chapter
>> 15 for more information.
>>
>>>
>>>
>>>
>>> Thank you very much.
>>>
>>>
>>>
>>> Yang
>>>
>>>
>>>
>>
>>
>> --
>> Adrian Klaver
>> adrian.klaver@aklaver.com
>>
>
>
> --
> Adrian Klaver
> adrian.klaver@aklaver.com
>
--
Adrian Klaver
adrian.klaver@aklaver.com
Yes, we are trying to use sqlAlchemy over psycopg2. However, my experiments have been on standalone psycopg2. Because I
wasworking back and forth, I took the wrong connection string.
Here's the exact string that I'm using in stand-alone psycopg2:
1. conn_string = "host=fqdn_hostname' dbname='dbname' krbsrvname='postgres'"
or
2. conn_string = "host=fqdn_hostname' dbname='dbname'"
The 1 gives the error of: psycopg2.OperationalError: invalid connection option "krbsrvname"
The 2 gives the error of: psycopg2.OperationalError: GSSAPI authentication not supported
Sorry for the earlier confusion.
Thanks.
Yang
-----Original Message-----
From: Adrian Klaver [mailto:adrian.klaver@aklaver.com]
Sent: Tuesday, December 06, 2016 10:32 AM
To: Yang Gao; 'psycopg@postgresql.org'
Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres?
On 12/06/2016 07:20 AM, Yang Gao wrote:
> You are most likely correct about local log in.
>
> This is the string that I'm using:
> sqlalchemy.create_engine("postgresql+psycopg2://fqdn_of_host/db_name")
> I tried adding "? krbsrvname=postgres" at the end as well, but got the error aforementioned.
It would have been nice to know you where using SQLAlchemy above
psycopg2 at the beginning of this conversation.
So again can you show the actual complete string you are using?
Not sure if this a cut and paste error but you have:
? krbsrvname=postgres
Pretty sure the space between ? and krbsrvname is not good.
Also what happens if you bypass SQLAlchemy and try to connect using
psycopg2 directly?
>
> Thanks.
>
> Yang
>
>
--
Adrian Klaver
adrian.klaver@aklaver.com
On 12/06/2016 07:41 AM, Yang Gao wrote:
> Yes, we are trying to use sqlAlchemy over psycopg2. However, my experiments have been on standalone psycopg2. Because
Iwas working back and forth, I took the wrong connection string.
I have to apologize. I was going back over the thread to see if I missed
anything and I found you did mention using SLQAlchemy in your first post.
> Here's the exact string that I'm using in stand-alone psycopg2:
>
> 1. conn_string = "host=fqdn_hostname' dbname='dbname' krbsrvname='postgres'"
> or
> 2. conn_string = "host=fqdn_hostname' dbname='dbname'"
>
> The 1 gives the error of: psycopg2.OperationalError: invalid connection option "krbsrvname"
> The 2 gives the error of: psycopg2.OperationalError: GSSAPI authentication not supported
This is to the remote host, correct?
Also what are the pg_hba.conf entries for the local and remote hosts?
Where does your libpq come from?
>
> Sorry for the earlier confusion.
>
> Thanks.
>
> Yang
>
>
>
> -----Original Message-----
> From: Adrian Klaver [mailto:adrian.klaver@aklaver.com]
> Sent: Tuesday, December 06, 2016 10:32 AM
> To: Yang Gao; 'psycopg@postgresql.org'
> Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres?
>
> On 12/06/2016 07:20 AM, Yang Gao wrote:
>> You are most likely correct about local log in.
>>
>> This is the string that I'm using:
>> sqlalchemy.create_engine("postgresql+psycopg2://fqdn_of_host/db_name")
>> I tried adding "? krbsrvname=postgres" at the end as well, but got the error aforementioned.
>
> It would have been nice to know you where using SQLAlchemy above
> psycopg2 at the beginning of this conversation.
>
> So again can you show the actual complete string you are using?
>
> Not sure if this a cut and paste error but you have:
>
> ? krbsrvname=postgres
>
> Pretty sure the space between ? and krbsrvname is not good.
>
> Also what happens if you bypass SQLAlchemy and try to connect using
> psycopg2 directly?
>
>
>>
>> Thanks.
>>
>> Yang
>>
>>
>
>
>
--
Adrian Klaver
adrian.klaver@aklaver.com
On 12/06/2016 08:28 AM, Yang Gao wrote: > That's a very good question. I pulled out the 2 libqp libraries. One in the default python installation that my psycopg2uses and the other used in my pyodbc experiment where Kerberos works. > They are of the same name, but different builds. Looks like I was using a lib without Kerberos support! > > If I replace the lib with the Kerberos supporting one, everything starts to work. > > Thank you very much! > > A further question, does psycopg2 respect the unixODBC's odbcinst.ini and odbc.ini (didn't find it in documentation)? Isthere way to explicitly specify the driver library? psycopg2 knows nothing about ODBC, so it will not look at those files. In other words psycopg2 != psqlodbc. > > Yang > > yangg@home:/proc/24196$ ldd /home/yangg/.conda/envs/py2/lib/libpq.so.5.8 > shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory > linux-vdso.so.1 => (0x00007ffd38db5000) > libssl.so.1.0.0 => /home/yangg/.conda/envs/py2/lib/./libssl.so.1.0.0 (0x00007fcecf7e7000) > libcrypto.so.1.0.0 => /home/yangg/.conda/envs/py2/lib/./libcrypto.so.1.0.0 (0x00007fcecf3b0000) > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fcecf178000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fcecedeb000) > libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fcecebe7000) > /lib64/ld-linux-x86-64.so.2 (0x0000559f3b33b000) > yangg@home:/proc/24196$ ldd /home/yangg/vats_code/.base_universe/current-architecture/ext/public/psqlodbc/09/05/0100/dist/lib/libpq.so.5.8 > shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory > linux-vdso.so.1 => (0x00007ffee39e1000) > libssl-opt.so.1.0.0 => /opt/openssl/1.0/lib/libssl-opt.so.1.0.0 (0x00007f0b7e1e9000) > libcrypto-opt.so.1.0.0 => /opt/openssl/1.0/lib/libcrypto-opt.so.1.0.0 (0x00007f0b7dda7000) > libgssapi_krb5.so.2 => /opt/mitkrb5/lib/libgssapi_krb5.so.2 (0x00007f0b7db61000) > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0b7d92a000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0b7d59d000) > libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f0b7d398000) > libkrb5.so.3 => /opt/mitkrb5/lib/libkrb5.so.3 (0x00007f0b7d0d6000) > libk5crypto.so.3 => /opt/mitkrb5/lib/libk5crypto.so.3 (0x00007f0b7cea7000) > libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f0b7cca2000) > libkrb5support.so.0 => /opt/mitkrb5/lib/libkrb5support.so.0 (0x00007f0b7ca97000) > libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f0b7c881000) > /lib64/ld-linux-x86-64.so.2 (0x000055ec0f899000) > -- Adrian Klaver adrian.klaver@aklaver.com
On 12/06/2016 10:09 AM, Yang Gao wrote: > Got it, so psycopq uses the Postgres c-lib directly. FYI as of 09.05.0100 so does psqlodbc: https://odbc.postgresql.org/docs/release.html psqlODBC 09.05.0100 Release Changes: Use libpq for all communication with the server Previously, libpq was only used for authentication. Using it for all communication lets us remove a lot of duplicated code. libpq is now required for building or using libpq. Still they serve two different roles, psycopg2 as Python binding to libpq and psqlodbc as ODBC driver. Then, what is the recommended way of managing libpq? Looks like when I installed psycopq from PyPi, it also pulled down this dependency which has no Kerberos support. Is there better way to overwrite this dependency rather than manually swap it out? Build psycopg2 from source and point it at the pg_config for the GSSAPI server? The ? because of this: http://initd.org/psycopg/docs/install.html#install-from-source " Note The libpq header files used to compile psycopg2 should match the version of the library linked at runtime. If you get errors about missing or mismatching libraries when importing psycopg2 check (e.g. using ldd) if the module psycopg2/_psycopg.so is linked to the right libpq.so. " Someone more versed in the build process then I will need to comment. How did install the Postgres servers in question? > > Thanks. > > Yang > > -----Original Message----- > From: Adrian Klaver [mailto:adrian.klaver@aklaver.com] > Sent: Tuesday, December 06, 2016 12:15 PM > To: Yang Gao; 'psycopg@postgresql.org' > Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres? > > On 12/06/2016 08:28 AM, Yang Gao wrote: >> That's a very good question. I pulled out the 2 libqp libraries. One in the default python installation that my psycopg2uses and the other used in my pyodbc experiment where Kerberos works. >> They are of the same name, but different builds. Looks like I was using a lib without Kerberos support! >> >> If I replace the lib with the Kerberos supporting one, everything starts to work. >> >> Thank you very much! >> >> A further question, does psycopg2 respect the unixODBC's odbcinst.ini and odbc.ini (didn't find it in documentation)?Is there way to explicitly specify the driver library? > > psycopg2 knows nothing about ODBC, so it will not look at those files. > > In other words psycopg2 != psqlodbc. > >> >> Yang >> -- Adrian Klaver adrian.klaver@aklaver.com
That's a very good question. I pulled out the 2 libqp libraries. One in the default python installation that my
psycopg2uses and the other used in my pyodbc experiment where Kerberos works.
They are of the same name, but different builds. Looks like I was using a lib without Kerberos support!
If I replace the lib with the Kerberos supporting one, everything starts to work.
Thank you very much!
A further question, does psycopg2 respect the unixODBC's odbcinst.ini and odbc.ini (didn't find it in documentation)?
Isthere way to explicitly specify the driver library?
Yang
yangg@home:/proc/24196$ ldd /home/yangg/.conda/envs/py2/lib/libpq.so.5.8
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
linux-vdso.so.1 => (0x00007ffd38db5000)
libssl.so.1.0.0 => /home/yangg/.conda/envs/py2/lib/./libssl.so.1.0.0 (0x00007fcecf7e7000)
libcrypto.so.1.0.0 => /home/yangg/.conda/envs/py2/lib/./libcrypto.so.1.0.0 (0x00007fcecf3b0000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fcecf178000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fcecedeb000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fcecebe7000)
/lib64/ld-linux-x86-64.so.2 (0x0000559f3b33b000)
yangg@home:/proc/24196$ ldd
/home/yangg/vats_code/.base_universe/current-architecture/ext/public/psqlodbc/09/05/0100/dist/lib/libpq.so.5.8
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
linux-vdso.so.1 => (0x00007ffee39e1000)
libssl-opt.so.1.0.0 => /opt/openssl/1.0/lib/libssl-opt.so.1.0.0 (0x00007f0b7e1e9000)
libcrypto-opt.so.1.0.0 => /opt/openssl/1.0/lib/libcrypto-opt.so.1.0.0 (0x00007f0b7dda7000)
libgssapi_krb5.so.2 => /opt/mitkrb5/lib/libgssapi_krb5.so.2 (0x00007f0b7db61000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0b7d92a000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0b7d59d000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f0b7d398000)
libkrb5.so.3 => /opt/mitkrb5/lib/libkrb5.so.3 (0x00007f0b7d0d6000)
libk5crypto.so.3 => /opt/mitkrb5/lib/libk5crypto.so.3 (0x00007f0b7cea7000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f0b7cca2000)
libkrb5support.so.0 => /opt/mitkrb5/lib/libkrb5support.so.0 (0x00007f0b7ca97000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f0b7c881000)
/lib64/ld-linux-x86-64.so.2 (0x000055ec0f899000)
-----Original Message-----
From: Adrian Klaver [mailto:adrian.klaver@aklaver.com]
Sent: Tuesday, December 06, 2016 10:59 AM
To: Yang Gao; 'psycopg@postgresql.org'
Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres?
On 12/06/2016 07:41 AM, Yang Gao wrote:
> Yes, we are trying to use sqlAlchemy over psycopg2. However, my experiments have been on standalone psycopg2. Because
Iwas working back and forth, I took the wrong connection string.
I have to apologize. I was going back over the thread to see if I missed anything and I found you did mention using
SLQAlchemyin your first post.
> Here's the exact string that I'm using in stand-alone psycopg2:
>
> 1. conn_string = "host=fqdn_hostname' dbname='dbname' krbsrvname='postgres'"
> or
> 2. conn_string = "host=fqdn_hostname' dbname='dbname'"
>
> The 1 gives the error of: psycopg2.OperationalError: invalid connection option "krbsrvname"
> The 2 gives the error of: psycopg2.OperationalError: GSSAPI
> authentication not supported
This is to the remote host, correct?
Also what are the pg_hba.conf entries for the local and remote hosts?
Where does your libpq come from?
>
> Sorry for the earlier confusion.
>
> Thanks.
>
> Yang
>
>
>
> -----Original Message-----
> From: Adrian Klaver [mailto:adrian.klaver@aklaver.com]
> Sent: Tuesday, December 06, 2016 10:32 AM
> To: Yang Gao; 'psycopg@postgresql.org'
> Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres?
>
> On 12/06/2016 07:20 AM, Yang Gao wrote:
>> You are most likely correct about local log in.
>>
>> This is the string that I'm using:
>>
>> sqlalchemy.create_engine("postgresql+psycopg2://fqdn_of_host/db_name"
>> ) I tried adding "? krbsrvname=postgres" at the end as well, but got
>> the error aforementioned.
>
> It would have been nice to know you where using SQLAlchemy above
> psycopg2 at the beginning of this conversation.
>
> So again can you show the actual complete string you are using?
>
> Not sure if this a cut and paste error but you have:
>
> ? krbsrvname=postgres
>
> Pretty sure the space between ? and krbsrvname is not good.
>
> Also what happens if you bypass SQLAlchemy and try to connect using
> psycopg2 directly?
>
>
>>
>> Thanks.
>>
>> Yang
>>
>>
>
>
>
--
Adrian Klaver
adrian.klaver@aklaver.com
Got it, so psycopq uses the Postgres c-lib directly. Then, what is the recommended way of managing libpq? Looks like whenI installed psycopq from PyPi, it also pulled down this dependency which has no Kerberos support. Is there better wayto overwrite this dependency rather than manually swap it out? Thanks. Yang -----Original Message----- From: Adrian Klaver [mailto:adrian.klaver@aklaver.com] Sent: Tuesday, December 06, 2016 12:15 PM To: Yang Gao; 'psycopg@postgresql.org' Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres? On 12/06/2016 08:28 AM, Yang Gao wrote: > That's a very good question. I pulled out the 2 libqp libraries. One in the default python installation that my psycopg2uses and the other used in my pyodbc experiment where Kerberos works. > They are of the same name, but different builds. Looks like I was using a lib without Kerberos support! > > If I replace the lib with the Kerberos supporting one, everything starts to work. > > Thank you very much! > > A further question, does psycopg2 respect the unixODBC's odbcinst.ini and odbc.ini (didn't find it in documentation)? Isthere way to explicitly specify the driver library? psycopg2 knows nothing about ODBC, so it will not look at those files. In other words psycopg2 != psqlodbc. > > Yang > > yangg@home:/proc/24196$ ldd > /home/yangg/.conda/envs/py2/lib/libpq.so.5.8 > shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory > linux-vdso.so.1 => (0x00007ffd38db5000) > libssl.so.1.0.0 => /home/yangg/.conda/envs/py2/lib/./libssl.so.1.0.0 (0x00007fcecf7e7000) > libcrypto.so.1.0.0 => /home/yangg/.conda/envs/py2/lib/./libcrypto.so.1.0.0 (0x00007fcecf3b0000) > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fcecf178000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fcecedeb000) > libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fcecebe7000) > /lib64/ld-linux-x86-64.so.2 (0x0000559f3b33b000) > yangg@home:/proc/24196$ ldd > /home/yangg/vats_code/.base_universe/current-architecture/ext/public/p > sqlodbc/09/05/0100/dist/lib/libpq.so.5.8 > shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory > linux-vdso.so.1 => (0x00007ffee39e1000) > libssl-opt.so.1.0.0 => /opt/openssl/1.0/lib/libssl-opt.so.1.0.0 (0x00007f0b7e1e9000) > libcrypto-opt.so.1.0.0 => /opt/openssl/1.0/lib/libcrypto-opt.so.1.0.0 (0x00007f0b7dda7000) > libgssapi_krb5.so.2 => /opt/mitkrb5/lib/libgssapi_krb5.so.2 (0x00007f0b7db61000) > libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0b7d92a000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0b7d59d000) > libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f0b7d398000) > libkrb5.so.3 => /opt/mitkrb5/lib/libkrb5.so.3 (0x00007f0b7d0d6000) > libk5crypto.so.3 => /opt/mitkrb5/lib/libk5crypto.so.3 (0x00007f0b7cea7000) > libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f0b7cca2000) > libkrb5support.so.0 => /opt/mitkrb5/lib/libkrb5support.so.0 (0x00007f0b7ca97000) > libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f0b7c881000) > /lib64/ld-linux-x86-64.so.2 (0x000055ec0f899000) > -- Adrian Klaver adrian.klaver@aklaver.com
On Tue, Dec 6, 2016 at 6:09 PM, Yang Gao <Yang.Gao@twosigma.com> wrote: > Got it, so psycopq uses the Postgres c-lib directly. Then, what is the recommended way of managing libpq? Looks like whenI installed psycopq from PyPi, it also pulled down this dependency which has no Kerberos support. Is there better wayto overwrite this dependency rather than manually swap it out? No, installing psycopg2 from PyPI doesn't install the libpq: if not found already installed in the system it will give the dreaded error "pg_config" (well, it's sort of unrelated, but libpq and pg_config are regularly packaged together. If not you'd have a different error at build time). I'm sorry I've jumping on the thread only now, and thank you very much for your support Adrian: I don't know much about kerberos support. Yang, I see your libpq is at: /home/yangg/.conda/envs/py2/lib/libpq.so.5.8, so ISTM you haven't installed psycopg from PyPI but from Conda. It seems Conda ships a libpq build without kerberos support: you may want to open a bug with them and ask for a more complete build. AFAICS the libpq version shipped with stock ubuntu does include kerberos, so 'apt-get install python-dev libpq-dev && pip install psycopg2' should actually work. -- Daniele
Hi, Daniele, Thanks for pointing this out. You are absolutely right that conda installs libpq as a dependency of psycopg2. In https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6.2-py27_0.tar.bz2, the info/index.json as below contains adependency to the latest libpq which does not have Kerberos support(https://repo.continuum.io/pkgs/free/linux-64/libpq-9.5.4-0.tar.bz2). However, this dependency does not exist forpython3 builds such as psycopg2-2.6.2-py34_0.tar.bz2 and psycopg2-2.6.2-py35_0.tar.bz2. It actually makes sense as theshared libs are often specified by LD_LIBRARY_PATH. Would you agree? Not sure who packages and uploads psycopg2 to the central conda repo, but it would be nice not to have the dependency declaredfor the python2.7 builds. Thanks. Yang ----- Info/Index.json: { "arch": "x86_64", "build": "py27_0", "build_number": 0, "depends": [ "libpq", "openssl", "python 2.7*" ], "license": "LGPL, BSD-like, ZPL", "license_family": "LGPL", "name": "psycopg2", "platform": "linux", "subdir": "linux-64", "version": "2.6.2" } -----Original Message----- From: Daniele Varrazzo [mailto:daniele.varrazzo@gmail.com] Sent: Tuesday, December 06, 2016 8:55 PM To: Yang Gao Cc: Adrian Klaver; psycopg@postgresql.org Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres? On Tue, Dec 6, 2016 at 6:09 PM, Yang Gao <Yang.Gao@twosigma.com> wrote: > Got it, so psycopq uses the Postgres c-lib directly. Then, what is the recommended way of managing libpq? Looks like whenI installed psycopq from PyPi, it also pulled down this dependency which has no Kerberos support. Is there better wayto overwrite this dependency rather than manually swap it out? No, installing psycopg2 from PyPI doesn't install the libpq: if not found already installed in the system it will give thedreaded error "pg_config" (well, it's sort of unrelated, but libpq and pg_config are regularly packaged together. If notyou'd have a different error at build time). I'm sorry I've jumping on the thread only now, and thank you very much for your support Adrian: I don't know much about kerberossupport. Yang, I see your libpq is at: /home/yangg/.conda/envs/py2/lib/libpq.so.5.8, so ISTM you haven't installed psycopg from PyPI but from Conda. It seems Condaships a libpq build without kerberos support: you may want to open a bug with them and ask for a more complete build.AFAICS the libpq version shipped with stock ubuntu does include kerberos, so 'apt-get install python-dev libpq-dev&& pip install psycopg2' should actually work. -- Daniele
On 12/08/2016 11:01 AM, Yang Gao wrote: > Hi, Daniele, > > Thanks for pointing this out. You are absolutely right that conda installs libpq as a dependency of psycopg2. > In https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6.2-py27_0.tar.bz2, the info/index.json as below containsa dependency to the latest libpq which does not have Kerberos support(https://repo.continuum.io/pkgs/free/linux-64/libpq-9.5.4-0.tar.bz2). However, this dependency does not exist forpython3 builds such as psycopg2-2.6.2-py34_0.tar.bz2 and psycopg2-2.6.2-py35_0.tar.bz2. It actually makes sense as theshared libs are often specified by LD_LIBRARY_PATH. Would you agree? You sure about that?: { "build": "py35_0", "build_number": 0, "channel": "defaults", "date": "2016-08-25", "depends": [ "libpq", "openssl", "python 3.5*" ], "extracted": true, "features": [], "fn": "psycopg2-2.6.2-py35_0.tar.bz2", "full_channel": "https://repo.continuum.io/pkgs/free/linux-64", "installed": false, "license": "LGPL, BSD-like, ZPL", "license_family": "LGPL", "md5": "5ee356151c0049ae30f3c0afd723775b", "name": "psycopg2", "priority": 0, "requires": [], "schannel": "defaults", "size": 417719, "type": null, "url": "https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6.2-py35_0.tar.bz2", "version": "2.6.2" } It sees the dependency git added in the change from psycopg2 2.6.1 --> 2.6.1, not based on Python version: { "build": "py27_1", "build_number": 1, "channel": "defaults", "date": "2016-02-22", "depends": [ "openssl 1.0.2*", "python 2.7*" ], "extracted": false, "features": [], "fn": "psycopg2-2.6.1-py27_1.tar.bz2", "full_channel": "https://repo.continuum.io/pkgs/free/linux-64", "installed": false, "license": "LGPL, BSD-like, ZPL", "license_family": "LGPL", "md5": "cfc47131fc25f954e852658ed902f945", "name": "psycopg2", "priority": 0, "requires": [], "schannel": "defaults", "size": 442528, "type": null, "url": "https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6.1-py27_1.tar.bz2", "version": "2.6.1" }, { "build": "py35_0", "build_number": 0, "channel": "defaults", "date": "2015-09-29", "depends": [ "openssl 1.0.1*", "python 3.5*" ], "extracted": false, "features": [], "fn": "psycopg2-2.6.1-py35_0.tar.bz2", "full_channel": "https://repo.continuum.io/pkgs/free/linux-64", "installed": false, "license": "LGPL, BSD-like, ZPL", "license_family": "LGPL", "md5": "c81233e8c3ded76a6f25c9567d2a2c91", "name": "psycopg2", "priority": 0, "requires": [], "schannel": "defaults", "size": 467021, "type": null, "url": "https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6.1-py35_0.tar.bz2", "version": "2.6.1" }, > > Not sure who packages and uploads psycopg2 to the central conda repo, but it would be nice not to have the dependencydeclared for the python2.7 builds. > > Thanks. > > Yang > > ----- > Info/Index.json: > > { > "arch": "x86_64", > "build": "py27_0", > "build_number": 0, > "depends": [ > "libpq", > "openssl", > "python 2.7*" > ], > "license": "LGPL, BSD-like, ZPL", > "license_family": "LGPL", > "name": "psycopg2", > "platform": "linux", > "subdir": "linux-64", > "version": "2.6.2" > } > > > -----Original Message----- > From: Daniele Varrazzo [mailto:daniele.varrazzo@gmail.com] > Sent: Tuesday, December 06, 2016 8:55 PM > To: Yang Gao > Cc: Adrian Klaver; psycopg@postgresql.org > Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres? > > On Tue, Dec 6, 2016 at 6:09 PM, Yang Gao <Yang.Gao@twosigma.com> wrote: >> Got it, so psycopq uses the Postgres c-lib directly. Then, what is the recommended way of managing libpq? Looks like whenI installed psycopq from PyPi, it also pulled down this dependency which has no Kerberos support. Is there better wayto overwrite this dependency rather than manually swap it out? > > No, installing psycopg2 from PyPI doesn't install the libpq: if not found already installed in the system it will givethe dreaded error "pg_config" (well, it's sort of unrelated, but libpq and pg_config are regularly packaged together.If not you'd have a different error at build time). > > I'm sorry I've jumping on the thread only now, and thank you very much for your support Adrian: I don't know much aboutkerberos support. > > Yang, I see your libpq is at: > /home/yangg/.conda/envs/py2/lib/libpq.so.5.8, so ISTM you haven't installed psycopg from PyPI but from Conda. It seemsConda ships a libpq build without kerberos support: you may want to open a bug with them and ask for a more completebuild. AFAICS the libpq version shipped with stock ubuntu does include kerberos, so 'apt-get install python-dev libpq-dev&& pip install psycopg2' should actually work. > > -- Daniele > -- Adrian Klaver adrian.klaver@aklaver.com
On 12/08/2016 11:01 AM, Yang Gao wrote: > Hi, Daniele, > > Thanks for pointing this out. You are absolutely right that conda installs libpq as a dependency of psycopg2. > In https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6.2-py27_0.tar.bz2, the info/index.json as below containsa dependency to the latest libpq which does not have Kerberos support(https://repo.continuum.io/pkgs/free/linux-64/libpq-9.5.4-0.tar.bz2). However, this dependency does not exist forpython3 builds such as psycopg2-2.6.2-py34_0.tar.bz2 and psycopg2-2.6.2-py35_0.tar.bz2. It actually makes sense as theshared libs are often specified by LD_LIBRARY_PATH. Would you agree? You sure about that?: { "build": "py35_0", "build_number": 0, "channel": "defaults", "date": "2016-08-25", "depends": [ "libpq", "openssl", "python 3.5*" ], "extracted": true, "features": [], "fn": "psycopg2-2.6.2-py35_0.tar.bz2", "full_channel": "https://repo.continuum.io/pkgs/free/linux-64", "installed": false, "license": "LGPL, BSD-like, ZPL", "license_family": "LGPL", "md5": "5ee356151c0049ae30f3c0afd723775b", "name": "psycopg2", "priority": 0, "requires": [], "schannel": "defaults", "size": 417719, "type": null, "url": "https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6.2-py35_0.tar.bz2", "version": "2.6.2" } It sees the dependency git added in the change from psycopg2 2.6.1 --> 2.6.1, not based on Python version: { "build": "py27_1", "build_number": 1, "channel": "defaults", "date": "2016-02-22", "depends": [ "openssl 1.0.2*", "python 2.7*" ], "extracted": false, "features": [], "fn": "psycopg2-2.6.1-py27_1.tar.bz2", "full_channel": "https://repo.continuum.io/pkgs/free/linux-64", "installed": false, "license": "LGPL, BSD-like, ZPL", "license_family": "LGPL", "md5": "cfc47131fc25f954e852658ed902f945", "name": "psycopg2", "priority": 0, "requires": [], "schannel": "defaults", "size": 442528, "type": null, "url": "https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6.1-py27_1.tar.bz2", "version": "2.6.1" }, { "build": "py35_0", "build_number": 0, "channel": "defaults", "date": "2015-09-29", "depends": [ "openssl 1.0.1*", "python 3.5*" ], "extracted": false, "features": [], "fn": "psycopg2-2.6.1-py35_0.tar.bz2", "full_channel": "https://repo.continuum.io/pkgs/free/linux-64", "installed": false, "license": "LGPL, BSD-like, ZPL", "license_family": "LGPL", "md5": "c81233e8c3ded76a6f25c9567d2a2c91", "name": "psycopg2", "priority": 0, "requires": [], "schannel": "defaults", "size": 467021, "type": null, "url": "https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6.1-py35_0.tar.bz2", "version": "2.6.1" }, > > Not sure who packages and uploads psycopg2 to the central conda repo, but it would be nice not to have the dependencydeclared for the python2.7 builds. > > Thanks. > > Yang > > ----- > Info/Index.json: > > { > "arch": "x86_64", > "build": "py27_0", > "build_number": 0, > "depends": [ > "libpq", > "openssl", > "python 2.7*" > ], > "license": "LGPL, BSD-like, ZPL", > "license_family": "LGPL", > "name": "psycopg2", > "platform": "linux", > "subdir": "linux-64", > "version": "2.6.2" > } > > > -----Original Message----- > From: Daniele Varrazzo [mailto:daniele.varrazzo@gmail.com] > Sent: Tuesday, December 06, 2016 8:55 PM > To: Yang Gao > Cc: Adrian Klaver; psycopg@postgresql.org > Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres? > > On Tue, Dec 6, 2016 at 6:09 PM, Yang Gao <Yang.Gao@twosigma.com> wrote: >> Got it, so psycopq uses the Postgres c-lib directly. Then, what is the recommended way of managing libpq? Looks like whenI installed psycopq from PyPi, it also pulled down this dependency which has no Kerberos support. Is there better wayto overwrite this dependency rather than manually swap it out? > > No, installing psycopg2 from PyPI doesn't install the libpq: if not found already installed in the system it will givethe dreaded error "pg_config" (well, it's sort of unrelated, but libpq and pg_config are regularly packaged together.If not you'd have a different error at build time). > > I'm sorry I've jumping on the thread only now, and thank you very much for your support Adrian: I don't know much aboutkerberos support. > > Yang, I see your libpq is at: > /home/yangg/.conda/envs/py2/lib/libpq.so.5.8, so ISTM you haven't installed psycopg from PyPI but from Conda. It seemsConda ships a libpq build without kerberos support: you may want to open a bug with them and ask for a more completebuild. AFAICS the libpq version shipped with stock ubuntu does include kerberos, so 'apt-get install python-dev libpq-dev&& pip install psycopg2' should actually work. > > -- Daniele > -- Adrian Klaver adrian.klaver@aklaver.com
Sorry Adrian for the confusion. I was actually referring the builds below which have no libpq dependency. https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6-py34_1.tar.bz2 https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6.1-py27_0.tar.bz2 Do you know why the libpq is added as dependency? Thanks. Yang -----Original Message----- From: Adrian Klaver [mailto:adrian.klaver@aklaver.com] Sent: Thursday, December 08, 2016 4:04 PM To: Yang Gao; 'Daniele Varrazzo' Cc: psycopg@postgresql.org Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres? On 12/08/2016 11:01 AM, Yang Gao wrote: > Hi, Daniele, > > Thanks for pointing this out. You are absolutely right that conda installs libpq as a dependency of psycopg2. > In https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6.2-py27_0.tar.bz2, the info/index.json as below containsa dependency to the latest libpq which does not have Kerberos support(https://repo.continuum.io/pkgs/free/linux-64/libpq-9.5.4-0.tar.bz2). However, this dependency does not exist forpython3 builds such as psycopg2-2.6.2-py34_0.tar.bz2 and psycopg2-2.6.2-py35_0.tar.bz2. It actually makes sense as theshared libs are often specified by LD_LIBRARY_PATH. Would you agree? You sure about that?: { "build": "py35_0", "build_number": 0, "channel": "defaults", "date": "2016-08-25", "depends": [ "libpq", "openssl", "python 3.5*" ], "extracted": true, "features": [], "fn": "psycopg2-2.6.2-py35_0.tar.bz2", "full_channel": "https://repo.continuum.io/pkgs/free/linux-64", "installed": false, "license": "LGPL, BSD-like, ZPL", "license_family": "LGPL", "md5": "5ee356151c0049ae30f3c0afd723775b", "name": "psycopg2", "priority": 0, "requires": [], "schannel": "defaults", "size": 417719, "type": null, "url": "https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6.2-py35_0.tar.bz2", "version": "2.6.2" } It sees the dependency git added in the change from psycopg2 2.6.1 --> 2.6.1, not based on Python version: { "build": "py27_1", "build_number": 1, "channel": "defaults", "date": "2016-02-22", "depends": [ "openssl 1.0.2*", "python 2.7*" ], "extracted": false, "features": [], "fn": "psycopg2-2.6.1-py27_1.tar.bz2", "full_channel": "https://repo.continuum.io/pkgs/free/linux-64", "installed": false, "license": "LGPL, BSD-like, ZPL", "license_family": "LGPL", "md5": "cfc47131fc25f954e852658ed902f945", "name": "psycopg2", "priority": 0, "requires": [], "schannel": "defaults", "size": 442528, "type": null, "url": "https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6.1-py27_1.tar.bz2", "version": "2.6.1" }, { "build": "py35_0", "build_number": 0, "channel": "defaults", "date": "2015-09-29", "depends": [ "openssl 1.0.1*", "python 3.5*" ], "extracted": false, "features": [], "fn": "psycopg2-2.6.1-py35_0.tar.bz2", "full_channel": "https://repo.continuum.io/pkgs/free/linux-64", "installed": false, "license": "LGPL, BSD-like, ZPL", "license_family": "LGPL", "md5": "c81233e8c3ded76a6f25c9567d2a2c91", "name": "psycopg2", "priority": 0, "requires": [], "schannel": "defaults", "size": 467021, "type": null, "url": "https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6.1-py35_0.tar.bz2", "version": "2.6.1" }, > > Not sure who packages and uploads psycopg2 to the central conda repo, but it would be nice not to have the dependencydeclared for the python2.7 builds. > > Thanks. > > Yang > > ----- > Info/Index.json: > > { > "arch": "x86_64", > "build": "py27_0", > "build_number": 0, > "depends": [ > "libpq", > "openssl", > "python 2.7*" > ], > "license": "LGPL, BSD-like, ZPL", > "license_family": "LGPL", > "name": "psycopg2", > "platform": "linux", > "subdir": "linux-64", > "version": "2.6.2" > } > > > -----Original Message----- > From: Daniele Varrazzo [mailto:daniele.varrazzo@gmail.com] > Sent: Tuesday, December 06, 2016 8:55 PM > To: Yang Gao > Cc: Adrian Klaver; psycopg@postgresql.org > Subject: Re: [psycopg] Does psycopg2 support Kerberos for Postgres? > > On Tue, Dec 6, 2016 at 6:09 PM, Yang Gao <Yang.Gao@twosigma.com> wrote: >> Got it, so psycopq uses the Postgres c-lib directly. Then, what is the recommended way of managing libpq? Looks like whenI installed psycopq from PyPi, it also pulled down this dependency which has no Kerberos support. Is there better wayto overwrite this dependency rather than manually swap it out? > > No, installing psycopg2 from PyPI doesn't install the libpq: if not found already installed in the system it will givethe dreaded error "pg_config" (well, it's sort of unrelated, but libpq and pg_config are regularly packaged together.If not you'd have a different error at build time). > > I'm sorry I've jumping on the thread only now, and thank you very much for your support Adrian: I don't know much aboutkerberos support. > > Yang, I see your libpq is at: > /home/yangg/.conda/envs/py2/lib/libpq.so.5.8, so ISTM you haven't installed psycopg from PyPI but from Conda. It seemsConda ships a libpq build without kerberos support: you may want to open a bug with them and ask for a more completebuild. AFAICS the libpq version shipped with stock ubuntu does include kerberos, so 'apt-get install python-dev libpq-dev&& pip install psycopg2' should actually work. > > -- Daniele > -- Adrian Klaver adrian.klaver@aklaver.com
On 12/09/2016 06:14 AM, Yang Gao wrote: > Sorry Adrian for the confusion. I was actually referring the builds below which have no libpq dependency. > > https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6-py34_1.tar.bz2 > https://repo.continuum.io/pkgs/free/linux-64/psycopg2-2.6.1-py27_0.tar.bz2 > > Do you know why the libpq is added as dependency? No, that would be something to ask here: https://www.continuum.io/anaconda-community Probably best bet would be: https://groups.google.com/a/continuum.io/forum/#!forum/anaconda > > Thanks. > > Yang > > -- Adrian Klaver adrian.klaver@aklaver.com