Thread: Proposal - libpq Type System beta-0.8a (was PGparam)
We previously tried to send this proposal/patch, but it never showed up. We tried twice in a 16 hour period. It might besomething with the attachment, who knows. This time, we are providing a URL instead of attaching the patch. Please disregard previous emails if they come back from never-never-land. http://www.esilo.com/projects/postgresql/libpq/typesys-beta-0.8a.tar.gz What was previously called PGparam is now called libpq Type System. PGparam is just one structure within the Type System. This is an updated proposal/patch for a Type System in libpq. Basically, it allows applications to send binary formatted paramters "put" and receive text or binary formatted results "get" through a printf-style interface. It also adds the ability to register user-defined types, sub-classes of existing types as well as composites. There is full support for arrays, composites, composite arrays and nested composites. There are four documents included within the tar: type-system-api.txt - Documents the API functions type-specifiers.txt - This explains the type specifier syntax (printf-style). It also documents how to construct parameters and get result values for every supported type. composites-arrays.txt - Documents the use of arrays, composites and arrays of composites. This proposed API has full support for nested arrays or composites. type-handlers.txt - Explains how to register and implement a libpq type handler. Types can be sub-classed. The patch is named typesys.patch. There is a regression test named regression-test.c and a makefile for it named makefile.typesys. Andrew & Merlin eSilo
Hi, Andrew Chernow wrote: > It might be something with the > attachment, who knows. Most probably that was the case, yes. The -hackers list is limited, please use -patches to send patches. ;-) Regards Markus
Markus Schiltknecht wrote: > Hi, > > Andrew Chernow wrote: >> It might be something with the attachment, who knows. > > Most probably that was the case, yes. The -hackers list is limited, > please use -patches to send patches. ;-) > > Regards > > Markus > > Noted. In our case, its a little ambiguos whether -hackers or -patches is the correct place. We are really posting a proposal that happens to have a working implementation. We are looking for feedback and/or a discussion. andrew
Andrew Chernow wrote: > Markus Schiltknecht wrote: >> Hi, >> >> Andrew Chernow wrote: >>> It might be something with the attachment, who knows. >> >> Most probably that was the case, yes. The -hackers list is limited, >> please use -patches to send patches. ;-) >> >> Regards >> >> Markus >> >> > > Noted. > > In our case, its a little ambiguos whether -hackers or -patches is the > correct place. We are really posting a proposal that happens to have > a working implementation. We are looking for feedback and/or a > discussion. > > You should split it and send the proposal to -hackers. Ideally, you would have had a proposal discussed before you wrote a line of code. cheers andrew
Andrew Dunstan wrote: > > > Andrew Chernow wrote: >> Markus Schiltknecht wrote: >>> Hi, >>> >>> Andrew Chernow wrote: >>>> It might be something with the attachment, who knows. >>> >>> Most probably that was the case, yes. The -hackers list is limited, >>> please use -patches to send patches. ;-) >>> >>> Regards >>> >>> Markus >>> >>> >> >> Noted. >> >> In our case, its a little ambiguos whether -hackers or -patches is the >> correct place. We are really posting a proposal that happens to have >> a working implementation. We are looking for feedback and/or a >> discussion. >> >> > > You should split it and send the proposal to -hackers. Ideally, you > would have had a proposal discussed before you wrote a line of code. > > cheers > > andrew > > >>proposal discussed before you wrote a line of code Yeah, we realize that. In our situation, we use this code internally which is why it exists. Back in Aug 2007, we packaged it up and proposed it because we thought it would be useful to others. Since then, we have submitted several versions. Feedback was minimal. In fact, only Tom has made suggestions, which we have taken into consideration and adjusted the spec accordingly. We are interested in having a discussion about the beta-0.8a proposal and concept, not the implementation or submitting procedures. We provided the code in case someone wants to take a test drive. Andrew
On Jan 8, 2008 12:57 PM, Andrew Dunstan <andrew@dunslane.net> wrote: > You should split it and send the proposal to -hackers. Ideally, you > would have had a proposal discussed before you wrote a line of code. This is the latest in a long series of submissions...check the archives. We are (and have been) sensitive to the other pressures resulting from the 8.3 release. That said, we have been a little dismayed in the lack of comment. We attributed this to either lack of interest or just general business (we perhaps optimistically guessed the latter). We needed our extensions for our own projects and are willing to maintain them outside of the project if we have to...that is for the community to decide. For the record, we are extremely excited about the libpq changes and think others well be as well. Tom noted the lack of documentation as well as a number of technical issues. We addressed those issues (within the scope of what we wanted to accomplish). merlin
Merlin, > That said, we have been a little dismayed in the lack of comment. I think most people can't really follow what functionality this would allow users & driver authors to access, and what the spec for that functionality would. I know I'm not clear on it. A high-level proposal would arouse more general interest. Otherwise, you'll just get a critique and eventually it'll either get applied or rejected without much comment. -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco
"Merlin Moncure" <mmoncure@gmail.com> writes:
> That said, we have been a little dismayed in the lack of comment.
Personally, I'm ignoring this thread, as well as the one on
partitioning, because I'm up to my arse in 8.3-release alligators.
I'm going to try hard not to think about any 8.4 development issues
until 8.3 is actually out the door and we branch for 8.4 development.
I can't say how many other people are equally pressed for time ...
but seeing the minimal attention that seems to be getting paid to
open 8.3 issues, it doesn't look to me like the community as a whole
has a lot of spare cycles right now.
regards, tom lane
On Jan 8, 2008 4:31 PM, Josh Berkus <josh@agliodbs.com> wrote:
> Merlin,
>
> > That said, we have been a little dismayed in the lack of comment.
>
> I think most people can't really follow what functionality this would allow
> users & driver authors to access, and what the spec for that functionality
> would. I know I'm not clear on it.
>
> A high-level proposal would arouse more general interest. Otherwise,
> you'll just get a critique and eventually it'll either get applied or
> rejected without much comment.
There are over 1500 lines of documentation attached in the patch.
That's a good place to start learning (or the attached regression
test, if you want to get right to it). There's a lot of material to
cover. Here's a very high level overview of the functionality:
Goal:
The ultimate goal is to be able to put parameters for command
execution and get results in a consistent way. The wire format
(either text or binary) is completely abstracted. We added many
functions to facilitate this, but the core functionality comes from
two varargs style functions, PQgetf and PQputf, and the PGparam, which
complements PGresult.
Features:
*) Binary transfer of all built in types is supported in both
directions. Basically, we marshal backend wire format to/from C types
(some native, some were introduced). We also support text results so
you can pull data in a consistent interface.
*) For user types (with custom send/recv functions), functions can be
registered to marshal them through a type registration interface.
*) Arrays and composites are supported automatically (composites have
to be 'registered').
*) Client side handlers can be aliased (domains) or subclassed...type
handlers can chained together for special handling and/or conversion
to exotic application types.
Here is a short example which demonstrates some of the major features.There are many other examples and discussions of
minutiain the
documentation.
int resfmt = 1; /* binary format */
/* Put an int4 and a text */
PGparam *param = PQparamCreate(conn);
PQputf(param, "%int4 %text", 2000, "foobar");
PQparamExec(conn, param, resfmt, "insert into foo(id, user) values ($1, $2)");
/* The above as a one liner, internally 'puts' for you */
res = PQexecParamsf(conn, resfmt, "insert into foo(id, user) values (%int4, %text)", 2000, "foobar");
int i4;
char *text;
PGresult *res = PQparamExec(conn, NULL, resfmt, "select * from foo limit 1");
/* From tuple 0, get an int4 at field 0 and a text* from the user field* '%' denotes by field num, '#' by field name*/
PQgetf(res, 0, "%int4 #text*", 0, &i4, "user", &text);
note the above line is not wired to binary, text results would be fine as well.
/* let's get an array */
PGresult *res = PQparamExec(conn, NULL, resfmt, "select current_schemas(true)");
/* pop an array object out of the result. it creates a new result* with one field and one 'tuple' for each array
element.**arrays of composites return one field for each attribute of the composite.
*/
PGarray array;
PQgetf(res, 0, "%name[]", 0, &array);
PQclear(res);
for (i = 0; i < PQntuples(array.res); i++)
{ char *name; PQgetf(array.res, i, "%name*", 0, &name); printf("%s\n", name);
}
PQclear(array.res);
/* return data from composite type which we create and register */
CREATE TYPE merlin as (a int, t text); -- on server
PGresult *merlin;
PQregisterTypeHandler(conn, "merlin", NULL, NULL);
res = PQparamExec(conn, NULL, resfmt, "select (1, 'abc')::merlin");
PQgetf(res, 0, "%merlin", 0, &merlin);
PQclear(res);
PQgetf(merlin, 0, "%int4 #text*", 0, &i4, "t", &text);
PQclear(merlin);
merlin
Tom Lane wrote: > "Merlin Moncure" <mmoncure@gmail.com> writes: >> That said, we have been a little dismayed in the lack of comment. > > Personally, I'm ignoring this thread, as well as the one on > partitioning, because I'm up to my arse in 8.3-release alligators. > I'm going to try hard not to think about any 8.4 development issues > until 8.3 is actually out the door and we branch for 8.4 development. > > I can't say how many other people are equally pressed for time ... > but seeing the minimal attention that seems to be getting paid to > open 8.3 issues, it doesn't look to me like the community as a whole > has a lot of spare cycles right now. > > regards, tom lane > > Thanks for the heads up. We had a feeling the 8.3 crunch was a factor. We don't want to slow that down because we wereare waiting for 8.3 to perform yet another mysql to postgresql migration :) andrew
On Tue, Jan 08, 2008 at 05:33:51PM -0500, Merlin Moncure wrote: > Here is a short example which demonstrates some of the major features. > There are many other examples and discussions of minutia in the > documentation. I havn't looked at the source but FWIW I think it's an awesome idea. Have a nice day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > Those who make peaceful revolution impossible will make violent revolution inevitable. > -- John F Kennedy