Re: Stuff for 2.4.1 - Mailing list psycopg

On Sunday, March 27, 2011 11:25:20 am Daniele Varrazzo wrote:

> On Sun, Mar 27, 2011 at 6:23 PM, David Blewett <david@dawninglight.net> wrote:

> > On Sun, Mar 27, 2011 at 9:46 AM, Daniele Varrazzo

> >

> > <daniele.varrazzo@gmail.com> wrote:

> >> On Sun, Mar 27, 2011 at 12:51 PM, Harald Armin Massa

> >>

> >> <harald@2ndquadrant.com> wrote:

> >>> Are you really sure that psycopg2 should go the road of having own

> >>> parsers in addition to libpq-s routines? As much as I am happy about

> >>> the robustness when having other libpqs, and about the performance

> >>> benefit, as much I fear to have some new areas for possible bugs -

> >>> especially security-relevant things like SQL-injections.

> >>

> >> The will to stick as much as possible to the libpq functions has been

> >> the reason I had not written the above parser before (releasing

> >> 2.4.0). Unfortunately the bytea problem has proven trickier to handle

> >> for many psycopg users. I've changed my mind as I think psycopg has

> >> the responsibility to provide a set of feature in a robust way, and if

> >> the libpq is just not reliable for bytea parsing (for me the hex

> >> format should have been backported to the the client libraries of the

> >> previous versions) I think we have to provide a solution, not just to

> >> propagate the problem.

> >

> > I think I agree with Harald here. In my opinion, this shouldn't be

> > done at the driver level.

>

> No? At which level do you think this should be done?

>

> > There never has been a guarantee from the

> > database side that applications compiled against older libpq will be

> > able to communicate with newer versions. Emulating this in the driver

> > only propagates this mis-conception. What has been the problem in the

> > past?

>

> There have been several examples in the not so long story of this

> mailing list, other were in the previous ml. Here's a couple. You can

> google yourself to discover that GnuMed, Drupal and who knows how many

> not public projects have been bitten by the transition to hex.

This is fixed by:

bytea_output = escape

Major upgrades are just that, major. Failure to read the docs is on the user. I know, I got bit by the change in behavior of pl/python with regards to 't' or 'f'.

>

> http://archives.postgresql.org/psycopg/2011-02/msg00020.php

> http://archives.postgresql.org/pgsql-general/2010-10/msg00146.php

> (from Harald, interesting)

>

> > Maybe the documentation should be improved so that people are

> > sure to build against the appropriate version of libpq for the version

> > of the server they intend to communicate with?

>

> The documentation does a pretty good job in describing the problem.

> Unfortunately not everybody is willing/able/allowed to install an

> up-to-date client library. PG 9 is not so widespread on the clients

> yet (e.g. Natty still deploys 8.4, so you will have to wait at least

> Oct 2011 for a libpq 9 on the most widespread linux distro).

I am not entirely against your argument, but if you are going to make the above statement at least compare apples to apples. In Natty the released version of Psycopg2 is 2.2.1 , so 2.4.1 is not an option by your rules. Also, it is possible for people to run a Pg 9.0 on newer distributions:

https://launchpad.net/~pitti/+archive/postgresql

>

> Half of Psycopg job is to convert the textual representation of

> postgres data type into python object. The libpq provides no

> assistance in doing that for any other format than bytea. There is no

> PQunescapeDateTime whatsoever: should we ask Python users to parse the

> dates from string themselves? Not to mention composite types, hstore

> and other delicacies, for which we provide parsers to Python objects.

> The bytea is in a sense "blessed" for having a dedicated unescape

> function, but if using it causes more trouble than it solves I think a

> less problematic parser is a good answer.

Like I said I am not entirely against what you are trying to do. I am concerned though that hiding a implementation detail does not really serve the user. Sooner or later they will have to come to grips with the change in behavior, whether via Psycopg or some other interface.

>

> -- Daniele

--

Adrian Klaver

adrian.klaver@gmail.com