Re: libpq and prepared statements progress for 8.0 - Mailing list pgsql-hackers

Tom Lane <tgl@sss.pgh.pa.us> writes:

> Greg Stark <gsstark@mit.edu> writes:
> > Is there anything technically hard in adding this functionality to libpq? It
> > looks like it's just mechanically adding more entry points to existing code.
> 
> Well, (a) I ran out of time, and (b) I wasn't sure what the most
> convenient API would be.  Should we create something that packages
> together a Parse and a Describe Statement, or expose those as
> separate things?

I don't know either. For my purposes it seems like a waste to be handling this
if I don't need it. On the other hand another round trip when it is needed
would be poor. Does the protocol provide it immediately or does it require
another message to the server?

> > The describe statement part could be much trickier but DBD::Pg doesn't really
> > need that for basic functionality.
> 
> Doesn't it?  I thought the problem was that they couldn't find out what
> datatypes the parameters got resolved as.  That may not be important
> if they are happy with always shipping text strings, but if they want to
> move over to using binary transmission of parameter values then they
> gotta know the parameter types.

Well personally I'm happy always shipping text strings. I'm sure someone else
will have different opinions but I don't really see why it would be any faster
to marshal data into Postgres's preferred binary representation than it would
to marshal it into a string. Neither are going to match Perl's internal
representation anyways.

I'm assuming the data type is always known to the programmer anyways and he
can ensure the data is provided in the appropriate Perl representation the
driver expects.

It could be useful for fancier situations like marshalling a timestamp from an
ambiguous perl datastructure that could represent an integer or integer array
into a Postgres string or binary representation of a timestamp.

Or perhaps for things like GUI tools that will display a user dialog box for
prompting for parameters of the appropriate type.

But these seem like refinements. The basic functionality is to offer the
equivalent functionality to what exists already, where the provided parameters
are simply interpolated into the query string.

> > I do wonder whether DBD::Pg is really best off using libpq.
> 
> I was wondering that myself.  Would they be able to implement a
> pure-Perl driver if they ginned up their own protocol code?
> It'd be a lot of work, of course, so I can understand that they
> might not feel it's worth the trouble.

Well a pure-Perl driver or a driver written in C with perl bindings wouldn't
really be an unreasonable amount of work I don't think. What I'm worried about
is whether it's the right strategy.

I was pretty shocked when I heard that JDBC implements the low level protocol
itself. It seems like a dead-end strategy to me. Any new protocol innovations
need to be implemented in every driver. Every implementation gets the chance
to reimplement the same bugs and same inefficiencies over and over again.

I had thought it was a better idea to have a library that handled the low
level protocol details. It should provide a 1-1 mapping for everything you can
do with the protocol. But at least that way you're guaranteed to never be
sending garbage down the wire getting the state machine out of sync with the
server.

> > From what I'm reading now it seems the "read every record before
> > returning" behaviour is rooted in the libpq interface.
> 
> Right.  Again that's probably something that could be handled by
> exposing more/different API, but no one has stepped up to design it.

It sort of seems to me that libpq's problem is trying to do too much. It tries
to be an abstract easy-to-use API for C programmers. But there's a need for a
low level API that just handles the protocol syntax and state machine and not
much more.

-- 
greg