Thread: plpython3
Here's my latest patch. I'm going to submit to the open commitfest. (hrm, it's 189K compressed, should I just let peopleuse git in the future?) Known issues: - Documentation has gaps. - Code formatting/ws is not consistent with PG's. Any feedback would be appreciated. Excepting trusted and DB-API, this patch fulfills all of the current PL/Python TODOs for Python 3, and adds *many* new features(Seethe documentation, and the WIP wiki page). plpython3 is intentionally incompatible with plpython as Python 3 is intentionally incompatible with Python 2. Be sure tosee the recent discussion in the other Python 3 thread[1]. =) New Docs: http://python.projects.postgresql.org/pldocs/plpython3.html Overview: http://wiki.postgresql.org/wiki/WIP:plpython3 git(plpython3 branch): http://git.postgresql.org/gitweb?p=plpython3.git;a=summary Past threads on the subject: http://archives.postgresql.org/pgsql-hackers/2009-05/msg01376.php http://archives.postgresql.org/pgsql-hackers/2009-07/msg01519.php http://archives.postgresql.org/pgsql-hackers/2009-08/msg01505.php [1] http://archives.postgresql.org/pgsql-hackers/2009-11/msg00547.php
Attachment
On Nov 19, 2009, at 5:41 PM, James Pye wrote: > Here's my latest patch. Fixed a lot of memory/reference leaks, added some minor features(mostly around Arrays), and filled in more documentation. At this point, I don't have any more minor features in mind(save extending Postgres.notify when the payload patch hits),so I'm just doing finish work(improvements/clarifications to docs, message strings, and maybe some makefile work).
Attachment
On Sun, Dec 13, 2009 at 5:02 PM, James Pye <lists@jwp.name> wrote: > On Nov 19, 2009, at 5:41 PM, James Pye wrote: >> Here's my latest patch. > > Fixed a lot of memory/reference leaks, added some minor features(mostly around Arrays), and filled in more documentation. > > At this point, I don't have any more minor features in mind(save extending Postgres.notify when the payload patch hits),so I'm just doing finish work(improvements/clarifications to docs, message strings, and maybe some makefile work). I'm almost afraid to write anything at all about this patch for fear of being branded a nattering nabob of negativity (see other thread: damage control mode) but hopefully if I'm full of it (or not) others will write in and set me straight (or confirm my thinking, whichever is appropriate). Anyhow, I started by reviewing the past threads on this patch, to which the author helpfully provided links: > Past threads on the subject: > > http://archives.postgresql.org/pgsql-hackers/2009-05/msg01376.php > http://archives.postgresql.org/pgsql-hackers/2009-07/msg01519.php > http://archives.postgresql.org/pgsql-hackers/2009-08/msg01505.php I think we should be clear that this patch doesn't have a great deal to do with Python 3, since Peter Eisentraut has already patched the existing code to support Python 3. It is, rather, a reimplementation of PL/python, and accordingly it ought to be called pl/newpython or perhaps pl/pye-thon (sorry, couldn't resist). Peter Eisentraut has made it pretty clear that he would prefer to see us maintain and enhance the existing implementation rather than starting over, and even if we did start over, it seems from the above threads that we'd still need to maintain the existing code for quite a while (if not forever). So it seems to me that the threshold question for this patch is - do we think it's a good idea to maintain two implementations of PL/python in core? ...Robert
> So it seems to me that the threshold question for this patch is - do > we think it's a good idea to maintain two implementations of PL/python > in core? Not really, no. This is why we need PGAN ;-) If the new implementation is *better* that the existing PL/python, I could see eventually replacing it. It wouldn't be the first time that a rewrite exceeded the original tool. However, I'm not in a position to judge quality. --Josh Berkus
On Tue, 2010-01-12 at 20:06 -0800, Josh Berkus wrote: > > So it seems to me that the threshold question for this patch is - do > > we think it's a good idea to maintain two implementations of PL/python > > in core? > > Not really, no. This is why we need PGAN ;-) > > If the new implementation is *better* that the existing PL/python, I > could see eventually replacing it. It wouldn't be the first time that a > rewrite exceeded the original tool. I think it is important to remember that the current version of PL/python is pretty weak compared to its counter parts (Specifically PL/Perl). If the new version, is adequately written to community standards and increases PL/Python's capabilities we need to seriously consider it. If we can address any issues with this module, let's commit it as Pl/pythonng3 or something. Anyway, I am +1 on reviewing this patch for viability. I would love to never touch plPerl for advanced procedures again. Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On ons, 2010-01-13 at 09:47 -0800, Joshua D. Drake wrote: > I think it is important to remember that the current version of > PL/python is pretty weak compared to its counter parts (Specifically > PL/Perl). How so?
On Wed, 2010-01-13 at 19:53 +0200, Peter Eisentraut wrote: > On ons, 2010-01-13 at 09:47 -0800, Joshua D. Drake wrote: > > I think it is important to remember that the current version of > > PL/python is pretty weak compared to its counter parts (Specifically > > PL/Perl). > > How so? O.k. you may have just called me on an unintentional bluff. My knowledge of plpython is dated. I just tested some of the things (like in/out) and they appear to work at least on 8.4). My argument would be now, what is the benefit of the James Pye version over our version. James can you illustrate succinctly why we should be supporting a new version? If there is, I am still all for it, but I am a python bigot. Sincerely, Joshua D. Drake > > -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
> My argument would be now, what is the benefit of the James Pye version > over our version. James can you illustrate succinctly why we should be > supporting a new version? > > If there is, I am still all for it, but I am a python bigot. Yeah, it's just my viewpoint that we don't want 2 python procedural languages in core. One should be in core, and one should go on pgFoundry/PGAN. Which ever one is "better" by some clear definition should go in core. As I said, I have no opinion about whether Pye's PLpython should replace the existing because I'm in no position to judge. --Josh Berkus
On Wed, Jan 13, 2010 at 1:16 PM, Josh Berkus <josh@agliodbs.com> wrote: > >> My argument would be now, what is the benefit of the James Pye version >> over our version. James can you illustrate succinctly why we should be >> supporting a new version? >> >> If there is, I am still all for it, but I am a python bigot. > > Yeah, it's just my viewpoint that we don't want 2 python procedural > languages in core. One should be in core, and one should go on > pgFoundry/PGAN. Which ever one is "better" by some clear definition > should go in core. That was my thinking also. There are a couple of reasons to think that throwing over the current implementation for an new one may not be the right thing to do. 1. It's not just a rewrite, it's an incompatible rewrite that will present significant user-visible behavioral differences. So replacing the current implementation wholesale would produce massive breakage for anyone actually using PL/python in production. 2. Peter Eisentraut, who has put significant time into improving PL/python for this release (see the commit logs), and who is the ONLY committer to work on improving PL/python for this release, has made it clear that he prefers the current implementation. Given these two facts, it's hard for me to see how we could decide to REMOVE the current implementation and replace it with the new one. So the most we could do is maintain them side by side, and then you have to ask, why? It will actually be much easier for James Pye to maintain his code outside core, and if it turns out to be popular and people come back to us and say "hey, why isn't this the default?" then we can revisit the issue. Sure, his code won't get as much exposure that way, but it's been posted to the mailing list several times now over a period of 8 months and nobody has said "oh, wow, this is great". The most we've gotten is several versions of this exchange: Somebody: We should really consider this code, the current code isn't very good. Peter: What's wrong with it? Somebody: <describes some problem, usually fairly vaguely> Peter: Why can't we fix that in the existing code base? <thread ends> I'm not saying a complete rewrite isn't the way to go - I'm just saying that there isn't much evidence at this point that it's really necessary, and I don't think we want to maintain two copies of PL/python unless we're really sure that the new implementation is an improvement. ...Robert
On Jan 13, 2010, at 11:08 AM, Joshua D. Drake wrote: > My argument would be now, what is the benefit of the James Pye version > over our version. James can you illustrate succinctly why we should be > supporting a new version? Doing so, succinctly, is unfortunately difficult. It is primarily a matter of comparing features, AFAICT. And, furthermore, some features may not be useful to some users. It exposes additional functionality that should *not* be incrementally developed in plpython as it would break applications.This was the point of trying to move forward with it for Python 3. Function Modules:- Does away with the need for GD/SD (more natural Python environment).- Allows tracebacks (tracebacks areuseful, right?) to implemented easily.- Does *not* expose a bastardized variant of the language by pretending that "modules/scriptfiles" can return and yield.- Helps to promote the Python tenet of being explicit. Native Typing:- Provides PG type introspection not available in any other PL, AFAIK.- Improves efficiency in some cases (conversionmust be _explicitly_ called for) - MD Array support.- Composites are a sequence and a mapping. Other features: http://wiki.postgresql.org/wiki/WIP:plpython3 Aside from function modules and native typing, many of plpython3's features could be implemented incrementally. However,I had a chance to sprint and they are available now in a new implementation. I did so, rather than improving plpython,because I believe that native typing and function modules are very useful. I'm not sure this fulfills your request, but, hopefully, it's a start.
On Wed, 2010-01-13 at 13:06 -0700, James William Pye wrote: > On Jan 13, 2010, at 11:08 AM, Joshua D. Drake wrote: > > My argument would be now, what is the benefit of the James Pye version > > over our version. James can you illustrate succinctly why we should be > > supporting a new version? > > > Doing so, succinctly, is unfortunately difficult. > It is primarily a matter of comparing features, AFAICT. And, furthermore, some features may not be useful to some users. > > It exposes additional functionality that should *not* be incrementally developed in plpython as it would break applications.This was the point of trying to move forward with it for Python 3. > > Function Modules: > - Does away with the need for GD/SD (more natural Python environment). > - Allows tracebacks (tracebacks are useful, right?) to implemented easily. > - Does *not* expose a bastardized variant of the language by pretending that "modules/script files" can return and yield. > - Helps to promote the Python tenet of being explicit. > > Native Typing: > - Provides PG type introspection not available in any other PL, AFAIK. > - Improves efficiency in some cases (conversion must be _explicitly_ called for) > - MD Array support. > - Composites are a sequence and a mapping. > > Other features: http://wiki.postgresql.org/wiki/WIP:plpython3 > > > Aside from function modules and native typing, many of plpython3's features could be implemented incrementally. However,I had a chance to sprint and they are available now in a new implementation. I did so, rather than improving plpython,because I believe that native typing and function modules are very useful. > > I'm not sure this fulfills your request, but, hopefully, it's a start. It does actually. Now, hackers... as a Python guy I can say these things are truly useful, to a Python programmer trying to use Python as a procedural language with PostgreSQL. What do we think? Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On ons, 2010-01-13 at 12:12 -0800, Joshua D. Drake wrote: > On Wed, 2010-01-13 at 13:06 -0700, James William Pye wrote: > > Function Modules: > > - Does away with the need for GD/SD (more natural Python environment). > > - Allows tracebacks (tracebacks are useful, right?) to implemented easily. > > - Does *not* expose a bastardized variant of the language by pretending that "modules/script files" can return and yield. > > - Helps to promote the Python tenet of being explicit. > > > > Native Typing: > > - Provides PG type introspection not available in any other PL, AFAIK. > > - Improves efficiency in some cases (conversion must be _explicitly_ called for) > > - MD Array support. > > - Composites are a sequence and a mapping. > > > > Other features: http://wiki.postgresql.org/wiki/WIP:plpython3 > > > > > > Aside from function modules and native typing, many of plpython3's features could be implemented incrementally. However,I had a chance to sprint and they are available now in a new implementation. I did so, rather than improving plpython,because I believe that native typing and function modules are very useful. > > > > I'm not sure this fulfills your request, but, hopefully, it's a start. > > It does actually. Now, hackers... as a Python guy I can say these things > are truly useful, to a Python programmer trying to use Python as a > procedural language with PostgreSQL. The problem I'm having with this discussion is that every time someone asks what the supposed advantages of this new Python PL are, a feature list like the above is dumped, 75% of which is subjective and tends to use semi-buzzwords, such that then someone else who by his own admission isn't completely up to date on things says, sure, that sounds great. Who wouldn't like a "more natural Python environment", "native typing" and "efficiency", and maybe even "explicitness"? The current PL/Python also has, arguably, a more natural Python environment, native typing, efficiency, and explicitness. So there you go. Now what?
On Wed, 2010-01-13 at 23:27 +0200, Peter Eisentraut wrote: > The problem I'm having with this discussion is that every time someone > asks what the supposed advantages of this new Python PL are, a feature > list like the above is dumped, 75% of which is subjective and tends to > use semi-buzzwords, such that then someone else who by his own admission > isn't completely up to date on things says, sure, that sounds great. > Who wouldn't like a "more natural Python environment", "native typing" > and "efficiency", and maybe even "explicitness"? The current PL/Python > also has, arguably, a more natural Python environment, native typing, > efficiency, and explicitness. So there you go. Now what? Peter, are you interested in the benefit of the project, or your own ego? Take it easy and let's see if we can be productive here. I was just stating that James's explanation was good and looking for more information from a reviewer. Secondly nobody (at least I am not) is suggesting we dump the current PL/Python, so your hard work is not going to be lost. The only thing I am currently looking for is an objective review of the patch based on the benefits it provides. I can tell you that if the Pye patch provides stack trace capability and the current PL does not. That right there is enough for me to push a +1 for review and possible inclusion. If you feel what James is saying is actually trite and more buzzword compliant I welcome the opportunity to see a comparative of the current status of PL/Python versus the Pye patch from you. If nothing else it is a learning opportunity for all involved. Sincerely, Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On ons, 2010-01-13 at 13:33 -0800, Joshua D. Drake wrote: > The only thing I am currently looking for is an objective review of the > patch based on the benefits it provides. Right, but I was opining that such a vague feature listing is not adequate for that. > I can tell you that if the Pye > patch provides stack trace capability and the current PL does not. That > right there is enough for me to push a +1 for review and possible > inclusion. That is the case. > If you feel what James is saying is actually trite and more buzzword > compliant I welcome the opportunity to see a comparative of the current > status of PL/Python versus the Pye patch from you. If nothing else it is > a learning opportunity for all involved. There was extensive discussion about some of the design decisions upthread, so any reviewer should review those. It may end up being a agree-to-disagree situation.
On Tue, 2010-01-12 at 20:06 -0800, Josh Berkus wrote: > > So it seems to me that the threshold question for this patch is - do > > we think it's a good idea to maintain two implementations of PL/python > > in core? > > Not really, no. This is why we need PGAN ;-) > > If the new implementation is *better* that the existing PL/python, I > could see eventually replacing it. It wouldn't be the first time that a > rewrite exceeded the original tool. I think it is important to remember that the current version of PL/python is pretty weak compared to its counter parts (Specifically PL/Perl). If the new version, is adequately written to community standards and increases PL/Python's capabilities we need to seriously consider it. If we can address any issues with this module, let's commit it as Pl/pythonng3 or something. Anyway, I am +1 on reviewing this patch for viability. I would love to never touch plPerl for advanced procedures again. Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Wed, 2010-01-13 at 19:53 +0200, Peter Eisentraut wrote: > On ons, 2010-01-13 at 09:47 -0800, Joshua D. Drake wrote: > > I think it is important to remember that the current version of > > PL/python is pretty weak compared to its counter parts (Specifically > > PL/Perl). > > How so? O.k. you may have just called me on an unintentional bluff. My knowledge of plpython is dated. I just tested some of the things (like in/out) and they appear to work at least on 8.4). My argument would be now, what is the benefit of the James Pye version over our version. James can you illustrate succinctly why we should be supporting a new version? If there is, I am still all for it, but I am a python bigot. Sincerely, Joshua D. Drake > > -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Wed, 2010-01-13 at 13:06 -0700, James William Pye wrote: > On Jan 13, 2010, at 11:08 AM, Joshua D. Drake wrote: > > My argument would be now, what is the benefit of the James Pye version > > over our version. James can you illustrate succinctly why we should be > > supporting a new version? > > > Doing so, succinctly, is unfortunately difficult. > It is primarily a matter of comparing features, AFAICT. And, furthermore, some features may not be useful to some users. > > It exposes additional functionality that should *not* be incrementally developed in plpython as it would break applications.This was the point of trying to move forward with it for Python 3. > > Function Modules: > - Does away with the need for GD/SD (more natural Python environment). > - Allows tracebacks (tracebacks are useful, right?) to implemented easily. > - Does *not* expose a bastardized variant of the language by pretending that "modules/script files" can return and yield. > - Helps to promote the Python tenet of being explicit. > > Native Typing: > - Provides PG type introspection not available in any other PL, AFAIK. > - Improves efficiency in some cases (conversion must be _explicitly_ called for) > - MD Array support. > - Composites are a sequence and a mapping. > > Other features: http://wiki.postgresql.org/wiki/WIP:plpython3 > > > Aside from function modules and native typing, many of plpython3's features could be implemented incrementally. However,I had a chance to sprint and they are available now in a new implementation. I did so, rather than improving plpython,because I believe that native typing and function modules are very useful. > > I'm not sure this fulfills your request, but, hopefully, it's a start. It does actually. Now, hackers... as a Python guy I can say these things are truly useful, to a Python programmer trying to use Python as a procedural language with PostgreSQL. What do we think? Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Jan 13, 2010, at 2:27 PM, Peter Eisentraut wrote: > The problem I'm having with this discussion is that every time someone > asks what the supposed advantages of this new Python PL are, a feature > list like the above is dumped, I agree that this is unfortunate, but how else can we to discuss the advantages? It boils down to comparing a couple featurelists, and *maybe* some implementation details. No? > 75% of which is subjective and tends to use semi-buzzwords, You say "semi-buzzwords", I say "names". I have to call "it" something. > such that then someone else who by his own admission > isn't completely up to date on things says, sure, that sounds great. Which is why we need to get some more experienced Python users involved in this. Well, even the mileage of inexperienced users is quite useful for detecting what level of obviousness has been achieved bythe features, so I'm not trying to exclude anyone. > The current PL/Python also has, arguably, a more natural Python environment, No, it doesn't. GD/SD are contrived in order to compensate for the very absence of that. Additionally, "modules / scriptfiles" don't return or yield. > native typing, Okay, that's arguably subjective, but when I write "native typing", it's wrt PG, not Python's built-in types. If that wasn'tthe case, I wouldn't call it anything--save "[data] conversion" when necessary--as it wouldn't be much of a featureto clamor about. > efficiency, Yes, as discussed in the thread before there are trade-offs here wrt how PG data is handled. It depends pretty heavily onhow the parameters / query results are used. Although, as stated before, the difference in efficiency can be rather significant in situations where conversion to built-inPython types is *not* desired. > and explicitness. Hrm? What is explicit about munging the source code of the procedure to make it into a function body? Perhaps you could givesome examples where plpython helps promote explicitness? And sure, I understand that other PLs do this. That may be fine for those languages, but for Python it's not, IMO.
On Wed, 2010-01-13 at 23:27 +0200, Peter Eisentraut wrote: > The problem I'm having with this discussion is that every time someone > asks what the supposed advantages of this new Python PL are, a feature > list like the above is dumped, 75% of which is subjective and tends to > use semi-buzzwords, such that then someone else who by his own admission > isn't completely up to date on things says, sure, that sounds great. > Who wouldn't like a "more natural Python environment", "native typing" > and "efficiency", and maybe even "explicitness"? The current PL/Python > also has, arguably, a more natural Python environment, native typing, > efficiency, and explicitness. So there you go. Now what? Peter, are you interested in the benefit of the project, or your own ego? Take it easy and let's see if we can be productive here. I was just stating that James's explanation was good and looking for more information from a reviewer. Secondly nobody (at least I am not) is suggesting we dump the current PL/Python, so your hard work is not going to be lost. The only thing I am currently looking for is an objective review of the patch based on the benefits it provides. I can tell you that if the Pye patch provides stack trace capability and the current PL does not. That right there is enough for me to push a +1 for review and possible inclusion. If you feel what James is saying is actually trite and more buzzword compliant I welcome the opportunity to see a comparative of the current status of PL/Python versus the Pye patch from you. If nothing else it is a learning opportunity for all involved. Sincerely, Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Jan 13, 2010, at 12:15 PM, Robert Haas wrote: > 1. It's not just a rewrite, it's an incompatible rewrite that will > present significant user-visible behavioral differences. So replacing > the current implementation wholesale would produce massive breakage > for anyone actually using PL/python in production. Right. That was the point of trying to leverage Python 3 to make the distinction. Most people will need to update their functionsif they are moving to Python 3. And for the larger chunks of code, the hard stuff, the amount of change requiredis likely significant already. <snip some> > Given these two facts, it's hard for me to see how we could decide to > REMOVE the current implementation and replace it with the new one. So > the most we could do is maintain them side by side, and then you have > to ask, why? My original hope was that plpython would be maintained for several years to come and when Python 3 started picking up steam,we would deprecate plpython. If people still wanted to use it, they could continue using the older version of PG and/orsomeone could continue to maintain plpython out of core for legacy support. <snip/maintaining out of core> > Sure, his code won't get as much exposure that way, =) Try next to none. The existence of core's implementation makes competing *very* difficult, IMO. Thinking of something alongthe lines: "Why would I use/contribute to your implementation when core has one?" And, all I can say in response is,"Check out my features." Subsequently, they will probably weigh the added risk of choosing the loner's implementationand come to the conclusion that using core's would be safer in the long term. I can't fault that line of reasoning,so ISTM that it would be difficult to "sell". > but it's been posted to the mailing list several times now > over a period of 8 months and nobody has said "oh, wow, this is > great". Yeah. :( In the past, one person showed interest in function modules(Stuart, see the first WIP message), and two others showed interestin native typing(Nathan and Tino). Mr. Drake has also shown some interest in this thread. But, yes, you are correct. There has been no "wow, this is great" message.
James William Pye wrote: <blockquote cite="mid:76769544-E8EF-47F3-9CC7-5EBA7587200B@jwp.name" type="cite"><pre wrap="">OnJan 13, 2010, at 2:27 PM, Peter Eisentraut wrote: </pre><blockquote type="cite"><pre wrap="">The problem I'm havingwith this discussion is that every time someone asks what the supposed advantages of this new Python PL are, a feature list like the above is dumped, </pre></blockquote><pre wrap=""> I agree that this is unfortunate, but how else can we to discuss the advantages? It boils down to comparing a couple featurelists, and *maybe* some implementation details. No? </pre></blockquote><br /> Code samples. You're trying to unseata well established incumbent here, and you're not ever going to do that with documentation. Maybe your plpython3 hassome compelling features to it. I don't know, because even with several thousand lines of basic Python code to my creditI cannot understand a single one of the arguments you presented for why your implementation is better--except agreeingthat, yes, tracebacks are useful And even on that one, I'm not going to take your word on the superiority of yourimplementation. You're writing way over people's heads here. (Doesn't help that your docs link at the bottom of <aclass="moz-txt-link-freetext" href="http://wiki.postgresql.org/wiki/WIP:plpython3">http://wiki.postgresql.org/wiki/WIP:plpython3</a>is broken either). If one has to be a Python expert to understand your position, you've already lost.<br /><br /> Python code is easyto read though. If you'd said "here's a great example of how Function Modules are an improvement over what you can dowith the current pl/python," that would be infinitely more useful than the list of language trivia related to them. Youshould be aiming to put Peter on the spot to respond to claims you make like "you can't do this easily with the currentimplementation" after showing an elegant bit of code.<br /><br /> One of the things I'm increasingly frustrated by(and don't take this personally, this is a general comment coming more from the last CF rather than something I mean tosingle you out for) is how many patch submissions we get that don't have *compelling* examples showing their value. Havea better programming approach to something? Show me the old way and how the new way is better. Performance improvement? Provide a complete, self-contained example showing how to demonstrate it. New type of feature? Cut and pastea whole session showing how it's used, with every single command you typed after initdb.<br /><br /> Basically, if areviewer can't confirm your patch is doing something useful in five minutes and be excited that they've watched somethinginteresting happen, you're decreasing the chances that your patch will ever go anywhere dramatically. I hope thateveryone submitting patches reads <a class="moz-txt-link-freetext" href="http://www.depesz.com/">http://www.depesz.com/</a>at least once in a while. One of the things I really enjoy abouthis blog is how he shows complete working examples of so many patches. To pick a standout recent entry, <a class="moz-txt-link-freetext" href="http://www.depesz.com/index.php/2010/01/03/waiting-for-8-5-exclusion-constraints/">http://www.depesz.com/index.php/2010/01/03/waiting-for-8-5-exclusion-constraints/</a> takes"exclusion constraints"--a feature I didn't follow a bit of the discussion about--and works through the whole featurewith a series of examples that, while still complicated, are completely self-contained and possible to follow alonguntil you understand how it all fits together. Patch submitters should consider it a goal to make life that easy forthe reviewer stuck with checking their patch out.<br /><br /><pre class="moz-signature" cols="72">-- Greg Smith 2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support <a class="moz-txt-link-abbreviated" href="mailto:greg@2ndQuadrant.com">greg@2ndQuadrant.com</a> <a class="moz-txt-link-abbreviated"href="http://www.2ndQuadrant.com">www.2ndQuadrant.com</a> </pre>
* Greg Smith <greg@2ndquadrant.com> [100114 02:17]: > One of the things I'm increasingly frustrated by (and don't take this > personally, this is a general comment coming more from the last CF > rather than something I mean to single you out for) is how many patch > submissions we get that don't have *compelling* examples showing their > value. Have a better programming approach to something? Show me the old > way and how the new way is better. Performance improvement? Provide a > complete, self-contained example showing how to demonstrate it. New type > of feature? Cut and paste a whole session showing how it's used, with > every single command you typed after initdb. Wow, I can't agree more... I've seen *so* many patches fly by that don't mean *anything* to me with the description sent to -hackers, until I find out what they actually do, or could do, until I find it in my RSS reader, via: > I hope that everyone submitting patches > reads http://www.depesz.com/ at least once in a while. One of the things > I really enjoy about his blog is how he shows complete working examples > of so many patches. To pick a standout recent entry, > http://www.depesz.com/index.php/2010/01/03/waiting-for-8-5-exclusion-constraints/ > takes "exclusion constraints"--a feature I didn't follow a bit of the > discussion about--and works through the whole feature with a series of > examples that, while still complicated, are completely self-contained > and possible to follow along until you understand how it all fits > together. <what he said>++ > Patch submitters should consider it a goal to make life that > easy for the reviewer stuck with checking their patch out. Yes, submitters, please specifically try to make Hubert's life easier, because we *all* will appreciate it... -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
On Jan 14, 2010, at 12:17 AM, Greg Smith wrote: > Code samples. Okay. > I don't know, because even with several thousand lines of basic Python code to my credit I cannot understand a single oneof the arguments you presented for why your implementation is better--except agreeing that, yes, tracebacks are useful > And even on that one, I'm not going to take your word on the superiority of your implementation. Sure, that's what review is about. No? > You're writing way over people's heads here. Okay. I guess I hoped the documentation would help clarify a lot of this, and make the advantages self-evident. On that: > (Doesn't help that your docs link at the bottom of http://wiki.postgresql.org/wiki/WIP:plpython3 is broken either). Ouch. Thanks, that's fixed now. Please take a look again: http://python.projects.postgresql.org/pldocs/plpython3.html > If one has to be a Python expert to understand your position, you've already lost. Function modules should be pretty obvious. "native typing" is a bit more difficult as a solid understanding of PG's typesystem is fairly important for a firm grasp. > Python code is easy to read though. If you'd said "here's a great example of how Function Modules are an improvement overwhat you can do with the current pl/python," that would be infinitely more useful than the list of language trivia relatedto them. You should be aiming to put Peter on the spot to respond to claims you make like "you can't do this easilywith the current implementation" after showing an elegant bit of code. Okay. So, some examples would help. The documentation is back up, so please be sure to look at the numerous examples providedtherein. In addition to that, I'll try to get some contrasting examples posted as a follow-up to an earlier message."In plpython you do X whereas in plpython3 you do Y." Thanks.
On Thu, 2010-01-14 at 05:39 -0700, James William Pye wrote: > > > Python code is easy to read though. If you'd said "here's a great example of how Function Modules are an improvementover what you can do with the current pl/python," that would be infinitely more useful than the list of languagetrivia related to them. You should be aiming to put Peter on the spot to respond to claims you make like "you can'tdo this easily with the current implementation" after showing an elegant bit of code. > > Okay. So, some examples would help. The documentation is back up, so please be sure to look at the numerous examples providedtherein. In addition to that, I'll try to get some contrasting examples posted as a follow-up to an earlier message."In plpython you do X whereas in plpython3 you do Y." > The documentation is very thorough, thank you. I am still a fan of getting this reviewed for potential inclusion but I firmly agree with what Greg has already said. What I would (as a non hacker) would look for is: (1) Generalized benchmarks between plpython(core) and plpython3u I know a lot of these are subjective, but it is still good to see if there are any curves or points that bring the performance of either to light. (2) Example of the traceback facility, I know it is silly but I don't have time to actually download head, apply the patch and test this. This type of thing, showing debugging facilities within the function would be killer. (3) A distinct real world comparison where the core plpython falls down (if it does) against the plpython3u implementation I can't speak to your code quality, that is going to have to be someone else. Sincerely, Joshua D. Drake > > Thanks. -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
James William Pye wrote: > The documentation is back up, so please be sure to look at the numerous examples provided therein. In addition to that,I'll try to get some contrasting examples posted as a follow-up to an earlier message. "In plpython you do X whereasin plpython3 you do Y." > I had read the documentation before so it wasn't that bad that I couldn't see it. Basically, the issue I had is that it's not really clear which features are unique to this implementation that make it compelling. So more targeted examples like you're considering now would help. -- Greg Smith 2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support greg@2ndQuadrant.com www.2ndQuadrant.com
On Thu, 2010-01-14 at 05:39 -0700, James William Pye wrote: > > > Python code is easy to read though. If you'd said "here's a great example of how Function Modules are an improvementover what you can do with the current pl/python," that would be infinitely more useful than the list of languagetrivia related to them. You should be aiming to put Peter on the spot to respond to claims you make like "you can'tdo this easily with the current implementation" after showing an elegant bit of code. > > Okay. So, some examples would help. The documentation is back up, so please be sure to look at the numerous examples providedtherein. In addition to that, I'll try to get some contrasting examples posted as a follow-up to an earlier message."In plpython you do X whereas in plpython3 you do Y." > The documentation is very thorough, thank you. I am still a fan of getting this reviewed for potential inclusion but I firmly agree with what Greg has already said. What I would (as a non hacker) would look for is: (1) Generalized benchmarks between plpython(core) and plpython3u I know a lot of these are subjective, but it is still good to see if there are any curves or points that bring the performance of either to light. (2) Example of the traceback facility, I know it is silly but I don't have time to actually download head, apply the patch and test this. This type of thing, showing debugging facilities within the function would be killer. (3) A distinct real world comparison where the core plpython falls down (if it does) against the plpython3u implementation I can't speak to your code quality, that is going to have to be someone else. Sincerely, Joshua D. Drake > > Thanks. -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Jan 14, 2010, at 2:03 PM, Joshua D. Drake wrote: > What I would (as a non hacker) would look for is: > > (1) Generalized benchmarks between plpython(core) and plpython3u > > I know a lot of these are subjective, but it is still good to see if > there are any curves or points that bring the performance of either to > light. I guess I could do some simple function I/O tests to identify invocation overhead(take a single parameter and return it).This should give a somewhat reasonable view of the trade-offs of "native typing" vs conversion performance-wise. Onething to keep in mind is that *three* tests would need to be done per parameter set: 1. plpython's2. plpython3's (raw data objects/"native typing")3. plpython3's + @pytypes The third should show degraded performance in comparison to plpythonu's whereas the second should show improvement or nearequivalence. @pytypes is actually implemented in pure-Python, so the impact should be quite visible. http://python.projects.postgresql.org/pldocs/plpython3-postgres-pytypes.html I'm not sure there's anything else worth measuring. SRFs, maybe? > (2) Example of the traceback facility, I know it is silly but I don't > have time to actually download head, apply the patch and test this. Well, if you ever do find some time, the *easiest* way would probably be to download a branch snapshot from git.pg.org: http://git.postgresql.org/gitweb?p=plpython3.git;a=snapshot;h=refs/heads/plpython3;sf=tgz It requires Python 3.1. 3.0 has been abandoned by python.org. > This > type of thing, showing debugging facilities within the function would be > killer. The test output has a *lot* of tracebacks, so I'll just copy and paste one here. This one shows the traceback output of a chained exception. -- suffocates a pg error, and attempts to enter a protected area CREATE OR REPLACE FUNCTION pg_failure_suf_IFTE() RETURNS VOID LANGUAGE plpython3u AS $python$ import Postgres rp = Postgres.Type(Postgres.CONST['REGPROCEDUREOID']) def main(): try: fun = rp('nosuchfunc(int17,zzz)') except: # Should be valid, but the protection of # PL_DB_IN_ERROR should keep it from getting called. rp('pg_x_failure_suf()') $python$; SELECT pg_failure_suf_IFTE(); ERROR: database action attempted while in failed transaction CONTEXT: [exception from Python] Traceback (most recent call last): File "public.pg_failure_suf_ifte()", line 8, in main fun = rp('nosuchfunc(int17,zzz)')Postgres.Exception:type "int17" does not exist CODE: 42704 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "public.pg_failure_suf_ifte()", line 12, in main rp('pg_x_failure_suf()')Postgres.Exception [public.pg_failure_suf_ifte()] > (3) A distinct real world comparison where the core plpython falls down > (if it does) against the plpython3u implementation Hrm. Are you looking for something that plpython3 can do that plpython can't? Or are you looking for something where plpythonmakes the user work a lot harder?
On Fri, 2010-01-15 at 13:26 -0700, James William Pye wrote: > On Jan 14, 2010, at 2:03 PM, Joshua D. Drake wrote: > > What I would (as a non hacker) would look for is: > > > > (1) Generalized benchmarks between plpython(core) and plpython3u > > > > I know a lot of these are subjective, but it is still good to see if > > there are any curves or points that bring the performance of either to > > light. > > I guess I could do some simple function I/O tests to identify invocation overhead(take a single parameter and return it).This should give a somewhat reasonable view of the trade-offs of "native typing" vs conversion performance-wise. Onething to keep in mind is that *three* tests would need to be done per parameter set: > > 1. plpython's > 2. plpython3's (raw data objects/"native typing") > 3. plpython3's + @pytypes > > The third should show degraded performance in comparison to plpythonu's whereas the second should show improvement or nearequivalence. > > @pytypes is actually implemented in pure-Python, so the impact should be quite visible. > > http://python.projects.postgresql.org/pldocs/plpython3-postgres-pytypes.html > > > I'm not sure there's anything else worth measuring. SRFs, maybe? > > > > (2) Example of the traceback facility, I know it is silly but I don't > > have time to actually download head, apply the patch and test this. > > Well, if you ever do find some time, the *easiest* way would probably be to download a branch snapshot from git.pg.org: > > http://git.postgresql.org/gitweb?p=plpython3.git;a=snapshot;h=refs/heads/plpython3;sf=tgz > > It requires Python 3.1. 3.0 has been abandoned by python.org. > > > This > > type of thing, showing debugging facilities within the function would be > > killer. > > The test output has a *lot* of tracebacks, so I'll just copy and paste one here. > > This one shows the traceback output of a chained exception. > > -- suffocates a pg error, and attempts to enter a protected area > CREATE OR REPLACE FUNCTION pg_failure_suf_IFTE() RETURNS VOID LANGUAGE plpython3u AS > $python$ > import Postgres > > rp = Postgres.Type(Postgres.CONST['REGPROCEDUREOID']) > > def main(): > try: > fun = rp('nosuchfunc(int17,zzz)') > except: > # Should be valid, but the protection of > # PL_DB_IN_ERROR should keep it from getting called. > rp('pg_x_failure_suf()') > $python$; > > > SELECT pg_failure_suf_IFTE(); > ERROR: database action attempted while in failed transaction > CONTEXT: [exception from Python] > Traceback (most recent call last): > File "public.pg_failure_suf_ifte()", line 8, in main > fun = rp('nosuchfunc(int17,zzz)') > Postgres.Exception: type "int17" does not exist > CODE: 42704 > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File "public.pg_failure_suf_ifte()", line 12, in main > rp('pg_x_failure_suf()') > Postgres.Exception > > [public.pg_failure_suf_ifte()] > > > > (3) A distinct real world comparison where the core plpython falls down > > (if it does) against the plpython3u implementation > > Hrm. Are you looking for something that plpython3 can do that plpython can't? Or are you looking for something where plpythonmakes the user work a lot harder? I think both apply. This is great stuff, thank you for taking the effort. Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Fri, 2010-01-15 at 13:26 -0700, James William Pye wrote: > On Jan 14, 2010, at 2:03 PM, Joshua D. Drake wrote: > > What I would (as a non hacker) would look for is: > > > > (1) Generalized benchmarks between plpython(core) and plpython3u > > > > I know a lot of these are subjective, but it is still good to see if > > there are any curves or points that bring the performance of either to > > light. > > I guess I could do some simple function I/O tests to identify invocation overhead(take a single parameter and return it).This should give a somewhat reasonable view of the trade-offs of "native typing" vs conversion performance-wise. Onething to keep in mind is that *three* tests would need to be done per parameter set: > > 1. plpython's > 2. plpython3's (raw data objects/"native typing") > 3. plpython3's + @pytypes > > The third should show degraded performance in comparison to plpythonu's whereas the second should show improvement or nearequivalence. > > @pytypes is actually implemented in pure-Python, so the impact should be quite visible. > > http://python.projects.postgresql.org/pldocs/plpython3-postgres-pytypes.html > > > I'm not sure there's anything else worth measuring. SRFs, maybe? > > > > (2) Example of the traceback facility, I know it is silly but I don't > > have time to actually download head, apply the patch and test this. > > Well, if you ever do find some time, the *easiest* way would probably be to download a branch snapshot from git.pg.org: > > http://git.postgresql.org/gitweb?p=plpython3.git;a=snapshot;h=refs/heads/plpython3;sf=tgz > > It requires Python 3.1. 3.0 has been abandoned by python.org. > > > This > > type of thing, showing debugging facilities within the function would be > > killer. > > The test output has a *lot* of tracebacks, so I'll just copy and paste one here. > > This one shows the traceback output of a chained exception. > > -- suffocates a pg error, and attempts to enter a protected area > CREATE OR REPLACE FUNCTION pg_failure_suf_IFTE() RETURNS VOID LANGUAGE plpython3u AS > $python$ > import Postgres > > rp = Postgres.Type(Postgres.CONST['REGPROCEDUREOID']) > > def main(): > try: > fun = rp('nosuchfunc(int17,zzz)') > except: > # Should be valid, but the protection of > # PL_DB_IN_ERROR should keep it from getting called. > rp('pg_x_failure_suf()') > $python$; > > > SELECT pg_failure_suf_IFTE(); > ERROR: database action attempted while in failed transaction > CONTEXT: [exception from Python] > Traceback (most recent call last): > File "public.pg_failure_suf_ifte()", line 8, in main > fun = rp('nosuchfunc(int17,zzz)') > Postgres.Exception: type "int17" does not exist > CODE: 42704 > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File "public.pg_failure_suf_ifte()", line 12, in main > rp('pg_x_failure_suf()') > Postgres.Exception > > [public.pg_failure_suf_ifte()] > > > > (3) A distinct real world comparison where the core plpython falls down > > (if it does) against the plpython3u implementation > > Hrm. Are you looking for something that plpython3 can do that plpython can't? Or are you looking for something where plpythonmakes the user work a lot harder? I think both apply. This is great stuff, thank you for taking the effort. Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Jan 14, 2010, at 7:08 PM, Greg Smith wrote: > So more targeted examples like you're considering now would help. So far, I have three specific examples in mind: The first will illustrate the advantages of function modules wrt setup code in the module body. Primarily this is about convenience.(I'm going to send this example when I send this message) The second is a generic after trigger that does manipulation logging for some simple replication purposes. This example willillustrate one application of "native typing" as it provides direct access to a PG type's typoutput. The third one is a fairly old plpythonu example written by Elein that exercises SD to keep state for an aggregate. I'm expectingthis to be a good candidate for showing off stateful functions. Other things I plan to cover, but nothing specific in mind yet: Direct function callsInternal subtransactions, "with xact():" (something plpython can't do, save calling plpgsql =)
On Jan 14, 2010, at 7:08 PM, Greg Smith wrote: > So more targeted examples like you're considering now would help. Here's the first example. This covers an advantage of function modules. This is a conversion of a plpythonu function published to the wiki: http://wiki.postgresql.org/wiki/Google_Translate In the above link, the code is executed in the body of a Python function. Please see plpython's documentation if you don't understand what I mean by that. The effect of this is that every time the FUNCTION is called from PG, the import statements are ran, a new class object,UrlOpener, is created, and a new function object, translate, is created. Granted, a minor amount of overhead in thiscase, but the point is that in order to avoid it the author would have to use SD: if "urlopener" in SD:UrlOpener = SD["urlopener"] else:class UrlOpener(urllib.UrlOpener): ...SD["urlopener"] = UrlOpener While some may consider this a minor inconvenience, the problem is that *setup code is common*, so it's, at least, a ratherfrequent, minor inconvenience. With function modules, users have a module body to run any necessary setup code. Now, WRT the actual example code, I'm not suggesting that either example is ideal. Only that it should *help* identify oneparticular advantage of function modules. CREATE OR REPLACE FUNCTION public.gtranslate(src text, target text, phrase text)RETURNS textLANGUAGE plpython3u AS $function$ from urllib.request import URLopener from urllib.parse import quote_plus import json base_uri = "http://ajax.googleapis.com/ajax/services/language/translate?" class UrlOpener(URLopener): version = "py-gtranslate/1.0" urlopen = UrlOpener().open equal_fmt = '{0}={1}'.format @pytypes def main(src, to, phrase): args = ( ('v', '1.0'), ('langpair', quote_plus(src + '|' + to)), ('q', quote_plus(phrase)), ) argstring = '&'.join([equal_fmt(k,v) for (k,v) in args]) resp = urlopen(base_uri + argstring).read() resp = json.loads(resp.decode('utf-8')) try: return resp['responseData']['translatedText'] except: # should probably warn about failed translation return phrase $function$; pl_regression=# SELECT gtranslate('en', 'es', 'i like coffee'); gtranslate ------------------Me gusta el café (1 row) pl_regression=# SELECT gtranslate('en', 'de', 'i like coffee'); gtranslate ----------------Ich mag Kaffee (1 row)
On Sun, Jan 17, 2010 at 4:07 PM, James William Pye <lists@jwp.name> wrote: > The effect of this is that every time the FUNCTION is called from PG, the import statements are ran, a new class object,UrlOpener, is created, and a new function object, translate, is created. Granted, a minor amount of overhead in thiscase, but the point is that in order to avoid it the author would have to use SD: > > if "urlopener" in SD: > UrlOpener = SD["urlopener"] > else: > class UrlOpener(urllib.UrlOpener): > ... > SD["urlopener"] = UrlOpener > > While some may consider this a minor inconvenience, the problem is that *setup code is common*, so it's, at least, a ratherfrequent, minor inconvenience. > > > With function modules, users have a module body to run any necessary setup code. Coming from a Python developer perspective, this is indeed an improvement. I always thought the whole SD/GD thing was a little odd. Doing the setup at the module level and relying on the interpreter to keep it cached is much more "Pythonic" and is a common idiom. David
On Jan 14, 2010, at 2:03 PM, Joshua D. Drake wrote:
> What I would (as a non hacker) would look for is:
>
> (1) Generalized benchmarks between plpython(core) and plpython3u
>
> I know a lot of these are subjective, but it is still good to see if
> there are any curves or points that bring the performance of either to
> light.
k, it was pretty much as expected. However, the surprise for me was that @pytypes didn't perform as terribly as I
expectedit to. I imagine it's impact may become more noticeable with more parameters, but, nonetheless, I was
surprised.I didn't do any SRF tests, but the installations are still setup, so if anyone really wants to see that, it
shouldn'ttake long to do.
Apologies ahead of time for the lack pretty graphs. =)
I used two different builds/installations of PG to test as the PL names conflict. Both were compiled with the following
CFLAGS(pg_configoutput):
-O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing
-fwrapv
Both PLs were built against the same build of Python(recent svn update of release31-maint):
Python 3.1.1+ (release31-maint:77585M, Jan 17 2010, 10:29:13)
[GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin
I ran each of the test files a few times against the target installation, but I only attached one of each to this
message.(Primarily, multiple runs to filter out any spurious spikes.)
The source SQL and output files are attached.
rawtest.sql.out is the output for raw data objects(native typing).
pytypestest.sql.out is the output of the @pytypes test(native typing with conversion overhead).
plpythontest.sql.out is the output for core's plpython(conversion).
A few samples from the output files are included inline below.
Each volatile function is called 100,000 times from a COUNT() aggregate, and the duration is measured using psql's
\timing.Most of the functions simply return the first parameter given to it. The functions are ran inside a transaction
becauseplpython3 does some cache clears(linecache) and GC at the end of transactions.
The parameter type, if any, is indicated by the label:
noparams:
raw: 125ms
pytypes: 372ms (base overhead, it would appear)
plpython: 309ms
oneint2:
raw: 140ms
pytypes: 684ms
plpython: 750ms
oneint8:
raw: 145ms
pytypes: 676ms
plpython: 718ms
text_large:
raw: 271ms
pytypes: 2766ms
plpython: 2310ms
composite:
raw: 235ms
pytypes: 795ms (N/A, no conversion done, but takes a bit of a hit anyways)
plpython: 1654ms
Attachment
On Jan 14, 2010, at 7:08 PM, Greg Smith wrote: > So more targeted examples like you're considering now would help. Here's the trigger example which should help reveal some of the advantages of "native typing". This is a generic triggerthat constructs and logs manipulation statements for simple replication purposes. The original plpython version is located here: http://ar.pycon.org/common/2009/talkdata/PyCon2009/020/plpython.txt [You'll need to scroll down to the very bottom of that page.] There are three points in this example that need to be highlighted: 1. There is no need for a "mogrify" function (see original in the above link). 2. Attributes/columns of the records (new/old) are extracted when referenced. 3. The comparisons in after_update uses the data type's actual inequality operator. The first point is true because "native typing" gives the user direct access to a given type's typoutput via ``str(ob)``.This makes constructing the PG string representation of a given object *much* easier--quote_nullable, and done.The original plpython example will need to be updated to compensate for any changes in conversion: arrays will now needspecial handling and MD arrays will not work at all. It also relies heavily on the Python object representation matchingPG's; where that fails, special cases need to be implemented(composites, notably). All of that compensation performedin the original version is unnecessary in the plpython3 version. The second point touches on the "efficiency" that was referenced in an earlier message. No cycles are spent converting thecontents of a container object unless the user chooses to. Naturally, there is no advantage performance-wise if you arealways converting everything. I'd wager that with triggers, it's rare that everything needs to be converted. The third point reveals that Postgres.Object instances--a component of native typing--use the data type's operator for inequality.It's not limited to comparisons as all available Python operators are mapped to corresponding operators in PG.For many or all primitives, there is no added value over conversion. However, this provides a lot of convenience whenworking with UDTs, datetime types, and geometric types. ...ISTM that the primary advantage of "native typing" is that we get to define the Python interface to a given Postgres datatype. Three files are attached: afterlog.py - the trigger returning function afterlog.sql - the sql exercising the TRF (creates the replica_log table as well) afterlog.out - the contents of the replica_log table after executing afterlog.sql To replay: \i afterlog.py \i afterlog.sql SELECT * FROM replica_log;
Attachment
On Wed, 2010-01-20 at 19:32 -0700, James William Pye wrote: > Apologies ahead of time for the lack pretty graphs. =) > > I used two different builds/installations of PG to test as the PL names conflict. Both were compiled with the followingCFLAGS(pg_config output): > > -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing -fwrapv > > Both PLs were built against the same build of Python(recent svn update of release31-maint): > > Python 3.1.1+ (release31-maint:77585M, Jan 17 2010, 10:29:13) > [GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin > This is great stuff. Thanks for your diligence. > I ran each of the test files a few times against the target installation, but I only attached one of each to this message.(Primarily, multiple runs to filter out any spurious spikes.) > > The source SQL and output files are attached. > > rawtest.sql.out is the output for raw data objects(native typing). > pytypestest.sql.out is the output of the @pytypes test(native typing with conversion overhead). > plpythontest.sql.out is the output for core's plpython(conversion). > > A few samples from the output files are included inline below. > > Each volatile function is called 100,000 times from a COUNT() aggregate, and the duration is measured using psql's \timing.Most of the functions simply return the first parameter given to it. The functions are ran inside a transaction becauseplpython3 does some cache clears(linecache) and GC at the end of transactions. > > The parameter type, if any, is indicated by the label: > > noparams: > > raw: 125ms > pytypes: 372ms (base overhead, it would appear) > plpython: 309ms > > oneint2: > > raw: 140ms > pytypes: 684ms > plpython: 750ms > > oneint8: > > raw: 145ms > pytypes: 676ms > plpython: 718ms > > text_large: > > raw: 271ms > pytypes: 2766ms > plpython: 2310ms > > composite: > > raw: 235ms > pytypes: 795ms (N/A, no conversion done, but takes a bit of a hit anyways) > plpython: 1654ms > > -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Wed, 2010-01-20 at 19:32 -0700, James William Pye wrote: > Apologies ahead of time for the lack pretty graphs. =) > > I used two different builds/installations of PG to test as the PL names conflict. Both were compiled with the followingCFLAGS(pg_config output): > > -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing -fwrapv > > Both PLs were built against the same build of Python(recent svn update of release31-maint): > > Python 3.1.1+ (release31-maint:77585M, Jan 17 2010, 10:29:13) > [GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin > This is great stuff. Thanks for your diligence. > I ran each of the test files a few times against the target installation, but I only attached one of each to this message.(Primarily, multiple runs to filter out any spurious spikes.) > > The source SQL and output files are attached. > > rawtest.sql.out is the output for raw data objects(native typing). > pytypestest.sql.out is the output of the @pytypes test(native typing with conversion overhead). > plpythontest.sql.out is the output for core's plpython(conversion). > > A few samples from the output files are included inline below. > > Each volatile function is called 100,000 times from a COUNT() aggregate, and the duration is measured using psql's \timing.Most of the functions simply return the first parameter given to it. The functions are ran inside a transaction becauseplpython3 does some cache clears(linecache) and GC at the end of transactions. > > The parameter type, if any, is indicated by the label: > > noparams: > > raw: 125ms > pytypes: 372ms (base overhead, it would appear) > plpython: 309ms > > oneint2: > > raw: 140ms > pytypes: 684ms > plpython: 750ms > > oneint8: > > raw: 145ms > pytypes: 676ms > plpython: 718ms > > text_large: > > raw: 271ms > pytypes: 2766ms > plpython: 2310ms > > composite: > > raw: 235ms > pytypes: 795ms (N/A, no conversion done, but takes a bit of a hit anyways) > plpython: 1654ms > > -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Sat, 2010-01-23 at 13:28 -0700, James William Pye wrote: > On Jan 14, 2010, at 7:08 PM, Greg Smith wrote: > > So more targeted examples like you're considering now would help. So is there any more movement on this? Peter, what do you think? I mean... he has put in quite a bit of effort here. How do we get this reviewed? I asked my guys to do it, but my C guys are very Perl centric... Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Sat, 2010-01-23 at 13:28 -0700, James William Pye wrote: > On Jan 14, 2010, at 7:08 PM, Greg Smith wrote: > > So more targeted examples like you're considering now would help. So is there any more movement on this? Peter, what do you think? I mean... he has put in quite a bit of effort here. How do we get this reviewed? I asked my guys to do it, but my C guys are very Perl centric... Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Sat, Jan 23, 2010 at 3:28 PM, James William Pye <lists@jwp.name> wrote: > On Jan 14, 2010, at 7:08 PM, Greg Smith wrote: >> So more targeted examples like you're considering now would help. > > Here's the trigger example which should help reveal some of the advantages of "native typing". This is a generic triggerthat constructs and logs manipulation statements for simple replication purposes. > > The original plpython version is located here: > > http://ar.pycon.org/common/2009/talkdata/PyCon2009/020/plpython.txt > [You'll need to scroll down to the very bottom of that page.] > > > There are three points in this example that need to be highlighted: > > 1. There is no need for a "mogrify" function (see original in the above link). > 2. Attributes/columns of the records (new/old) are extracted when referenced. > 3. The comparisons in after_update uses the data type's actual inequality operator. > > The first point is true because "native typing" gives the user direct access to a given type's typoutput via ``str(ob)``.This makes constructing the PG string representation of a given object *much* easier--quote_nullable, and done.The original plpython example will need to be updated to compensate for any changes in conversion: arrays will now needspecial handling and MD arrays will not work at all. It also relies heavily on the Python object representation matchingPG's; where that fails, special cases need to be implemented(composites, notably). All of that compensation performedin the original version is unnecessary in the plpython3 version. > > The second point touches on the "efficiency" that was referenced in an earlier message. No cycles are spent convertingthe contents of a container object unless the user chooses to. Naturally, there is no advantage performance-wiseif you are always converting everything. > I'd wager that with triggers, it's rare that everything needs to be converted. > > The third point reveals that Postgres.Object instances--a component of native typing--use the data type's operator forinequality. It's not limited to comparisons as all available Python operators are mapped to corresponding operators inPG. For many or all primitives, there is no added value over conversion. However, this provides a lot of convenience whenworking with UDTs, datetime types, and geometric types. > > ...ISTM that the primary advantage of "native typing" is that we get to define the Python interface to a given Postgresdata type. When I initially read the description of this patch, I had really no idea what it was about, and I have to say that over the last several weeks, the subsequent posts have increased my interest substantially, and I think it could potentially have a future in core, or in contrib.The performance gains from native typing are particularly interesting. That having been said, we don't have a reviewer for this patch, and more to the point, even if we did, I don't believe we have a COMMITTER for this patch. The only committer who has done any significant work on the existing PL/python implementation for this release is Peter Eisentraut, and he's made it pretty clear that he's not interested in this reimplementation. I personally do not know Python even as a user, let alone as a hacker, so I'm poorly qualified to review it myself. Even were it otherwise, I know that if I were to commit this I would to some degree be on the hook to maintain it forever, which is more than I really want to take on. I suspect other committers feel similarly; or if not, they haven't chimed in on any of the relevant threads to say so. To recap the votes I've seen on this thread and elsewhere: - JD is very enthusiastic about this patch - So is the OP - Josh Berkus and I are both dubious about having two in-core PL/pythons - Peter Eisentraut prefers the original implementation - Greg Smith thinks (if I'm not putting words into his mouth) that this might be worth considering, but not for 9.0 On the basis of all of the foregoing, I don't think we can consider this patch further for this CommitFest and will update commitfest.postgresql.org accordingly. If the user community grows or if one of the committers takes an interest in this down the road, I think we could consider it for a future release. ...Robert
On Mon, 2010-02-01 at 13:20 -0500, Robert Haas wrote: > On the basis of all of the foregoing, I don't think we can consider > this patch further for this CommitFest and will update > commitfest.postgresql.org accordingly. If the user community grows or > if one of the committers takes an interest in this down the road, I > think we could consider it for a future release. > I spoke with James offline about this as well. My understanding (correct me James) is that he is working on an implementation that can be installed via PGXS. Joshua D. Drake > ...Robert > -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Mon, Feb 1, 2010 at 1:29 PM, Joshua D. Drake <jd@commandprompt.com> wrote: > On Mon, 2010-02-01 at 13:20 -0500, Robert Haas wrote: >> On the basis of all of the foregoing, I don't think we can consider >> this patch further for this CommitFest and will update >> commitfest.postgresql.org accordingly. If the user community grows or >> if one of the committers takes an interest in this down the road, I >> think we could consider it for a future release. > > I spoke with James offline about this as well. My understanding (correct > me James) is that he is working on an implementation that can be > installed via PGXS. Sounds good! ...Robert
Robert Haas <robertmhaas@gmail.com> writes:
> To recap the votes I've seen on this thread and elsewhere:
> - JD is very enthusiastic about this patch
> - So is the OP
> - Josh Berkus and I are both dubious about having two in-core PL/pythons
> - Peter Eisentraut prefers the original implementation
> - Greg Smith thinks (if I'm not putting words into his mouth) that
> this might be worth considering, but not for 9.0
One other problem with accepting this to be parallel with the existing
plpython is that there's a name conflict: Peter's work to allow the
existing PL to use Python 3 has already claimed the name "plpython3".
Whether it's to be distributed in core or separately, I think something
needs to be done about that.
The first thought that comes to mind is "plpythonng", following a
tradition established by the tcl client rewrite among others ... but
that double n doesn't read very well.
regards, tom lane
Tom Lane escribió: > The first thought that comes to mind is "plpythonng", following a > tradition established by the tcl client rewrite among others ... but > that double n doesn't read very well. plpythoNG perhaps? -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
On Mon, Feb 1, 2010 at 2:00 PM, Alvaro Herrera <alvherre@commandprompt.com> wrote: > Tom Lane escribió: > >> The first thought that comes to mind is "plpythonng", following a >> tradition established by the tcl client rewrite among others ... but >> that double n doesn't read very well. > > plpythoNG perhaps? ROFL. It's a scantily-clad implementation of PL/py... ...Robert
On Feb 1, 2010, at 10:53 AM, Tom Lane wrote: > The first thought that comes to mind is "plpythonng", following a > tradition established by the tcl client rewrite among others ... but > that double n doesn't read very well. And without it, you have a thong. Who's going to wear that? Best, David
On Feb 1, 2010, at 11:29 AM, Joshua D. Drake wrote: > On Mon, 2010-02-01 at 13:20 -0500, Robert Haas wrote: > >> On the basis of all of the foregoing, I don't think we can consider >> this patch further for this CommitFest and will update >> commitfest.postgresql.org accordingly. If the user community grows or >> if one of the committers takes an interest in this down the road, I >> think we could consider it for a future release. >> > > I spoke with James offline about this as well. My understanding (correct > me James) is that he is working on an implementation that can be > installed via PGXS. yep, mostly done: http://github.com/jwp/pg-python The tests that can pass are passing on 8.3 and 8.4 now, save optimized Cursor returns in 8.3(the materialize preferred flag). Also made some other improvements like getting rid of the ugly `__func__.stateful = True` in favor of a decorator, @Stateful.(Thanks to Harald for a push in that direction.) Right now, I'm trying to trim some of the easy issues[1] and getting a project web page up. I expect to be able to make arelease soon, and I'll follow-up to this thread when I do. However, the lack of release files shouldn't stop anyone from trying it out. =) Snapshot:http://github.com/jwp/pg-python/zipball/master [1] http://github.com/jwp/pg-python/issues
> On the basis of all of the foregoing, I don't think we can consider > this patch further for this CommitFest and will update > commitfest.postgresql.org accordingly. FWIW, I am very excited about this patch and would be happy to review it but have been very busy over the past month. If I can promise a review by Thursday morning could we keep it active? Hopefully, at the very least, I can provide some useful feedback and spawn some community interest. I am worried that there is a bit of a chicken and an egg problem with this patch. I code nearly exclusively in python and C, but I have often found pl/python to be very unwieldy. For this reason I often use pl/perl or pl/pgsql for problems that, outside of postgres, I would always use python. From the documentation, this patch seems like an enormous step in the right direction. -Nathan
On Mon, 2010-02-01 at 12:01 -0800, Nathan Boley wrote: > > On the basis of all of the foregoing, I don't think we can consider > > this patch further for this CommitFest and will update > > commitfest.postgresql.org accordingly. > > FWIW, I am very excited about this patch and would be happy to review > it but have been very busy over the past month. If I can promise a > review by Thursday morning could we keep it active? Hopefully, at the > very least, I can provide some useful feedback and spawn some > community interest. > > I am worried that there is a bit of a chicken and an egg problem with > this patch. I code nearly exclusively in python and C, but I have > often found pl/python to be very unwieldy. For this reason I often > use pl/perl or pl/pgsql for problems that, outside of postgres, I > would always use python. From the documentation, this patch seems like > an enormous step in the right direction. I am a +1 Joshua D. Drake > > -Nathan > -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Mon, Feb 1, 2010 at 3:01 PM, Nathan Boley <npboley@gmail.com> wrote: >> On the basis of all of the foregoing, I don't think we can consider >> this patch further for this CommitFest and will update >> commitfest.postgresql.org accordingly. > > FWIW, I am very excited about this patch and would be happy to review > it but have been very busy over the past month. If I can promise a > review by Thursday morning could we keep it active? Hopefully, at the > very least, I can provide some useful feedback and spawn some > community interest. > > I am worried that there is a bit of a chicken and an egg problem with > this patch. I code nearly exclusively in python and C, but I have > often found pl/python to be very unwieldy. For this reason I often > use pl/perl or pl/pgsql for problems that, outside of postgres, I > would always use python. From the documentation, this patch seems like > an enormous step in the right direction. I think it would be great for you to review it... I doubt that will cause it to get committed for 9.0, but my doubt is no reason for you to hold off reviewing it. ...Robert
> I think it would be great for you to review it... I doubt that will > cause it to get committed for 9.0, but my doubt is no reason for you > to hold off reviewing it. I assumed so, but the pretense of a chance will probably help to motivate me :-) I'll have something by Thursday, and then 'Returned with Feedback' will at least be factual. Best, Nathan
On mån, 2010-02-01 at 12:01 -0800, Nathan Boley wrote: > I code nearly exclusively in python and C, but I have > often found pl/python to be very unwieldy. For this reason I often > use pl/perl or pl/pgsql for problems that, outside of postgres, I > would always use python. I find that curious, because much of the criticism about the current PL/Python can be traced back to the fact that the implementation used to be an exact copy of PL/Perl.
On Mon, 2010-02-01 at 22:35 +0200, Peter Eisentraut wrote: > On mån, 2010-02-01 at 12:01 -0800, Nathan Boley wrote: > > I code nearly exclusively in python and C, but I have > > often found pl/python to be very unwieldy. For this reason I often > > use pl/perl or pl/pgsql for problems that, outside of postgres, I > > would always use python. > > I find that curious, because much of the criticism about the current > PL/Python can be traced back to the fact that the implementation used to > be an exact copy of PL/Perl. Well my guess is, if you want to code python, you don't want to feel like you are coding perl and thus you might as well just code perl? Joshua D. Drake > -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
Peter Eisentraut escribió: > On mån, 2010-02-01 at 12:01 -0800, Nathan Boley wrote: > > I code nearly exclusively in python and C, but I have > > often found pl/python to be very unwieldy. For this reason I often > > use pl/perl or pl/pgsql for problems that, outside of postgres, I > > would always use python. > > I find that curious, because much of the criticism about the current > PL/Python can be traced back to the fact that the implementation used to > be an exact copy of PL/Perl. Perhaps the problem is that PL/Perl used to be unwieldy back when PL/Python was created. PL/Perl has definitely seen a lot more activity. -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Alvaro Herrera wrote: > Peter Eisentraut escribi?: > > On m?n, 2010-02-01 at 12:01 -0800, Nathan Boley wrote: > > > I code nearly exclusively in python and C, but I have > > > often found pl/python to be very unwieldy. For this reason I often > > > use pl/perl or pl/pgsql for problems that, outside of postgres, I > > > would always use python. > > > > I find that curious, because much of the criticism about the current > > PL/Python can be traced back to the fact that the implementation used to > > be an exact copy of PL/Perl. > > Perhaps the problem is that PL/Perl used to be unwieldy back when > PL/Python was created. PL/Perl has definitely seen a lot more activity. I would love to know why PL/Python can't be incrementally improved like the rest of our code. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
On Mon, 2010-02-01 at 16:13 -0500, Bruce Momjian wrote: > Alvaro Herrera wrote: > > Peter Eisentraut escribi?: > > > On m?n, 2010-02-01 at 12:01 -0800, Nathan Boley wrote: > > > > I code nearly exclusively in python and C, but I have > > > > often found pl/python to be very unwieldy. For this reason I often > > > > use pl/perl or pl/pgsql for problems that, outside of postgres, I > > > > would always use python. > > > > > > I find that curious, because much of the criticism about the current > > > PL/Python can be traced back to the fact that the implementation used to > > > be an exact copy of PL/Perl. > > > > Perhaps the problem is that PL/Perl used to be unwieldy back when > > PL/Python was created. PL/Perl has definitely seen a lot more activity. > > I would love to know why PL/Python can't be incrementally improved like > the rest of our code. It has been. That is exactly what PeterE has been doing. However, if you look at this whole thread, you will see the James has a very different view of the implementation. One that at least appears to be more advanced and "pythonic" than our version. Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
Joshua D. Drake wrote: > On Mon, 2010-02-01 at 16:13 -0500, Bruce Momjian wrote: > > Alvaro Herrera wrote: > > > Peter Eisentraut escribi?: > > > > On m?n, 2010-02-01 at 12:01 -0800, Nathan Boley wrote: > > > > > I code nearly exclusively in python and C, but I have > > > > > often found pl/python to be very unwieldy. For this reason I often > > > > > use pl/perl or pl/pgsql for problems that, outside of postgres, I > > > > > would always use python. > > > > > > > > I find that curious, because much of the criticism about the current > > > > PL/Python can be traced back to the fact that the implementation used to > > > > be an exact copy of PL/Perl. > > > > > > Perhaps the problem is that PL/Perl used to be unwieldy back when > > > PL/Python was created. PL/Perl has definitely seen a lot more activity. > > > > I would love to know why PL/Python can't be incrementally improved like > > the rest of our code. > > It has been. That is exactly what PeterE has been doing. > > However, if you look at this whole thread, you will see the James has a > very different view of the implementation. One that at least appears to > be more advanced and "pythonic" than our version. More "pythonic" in its internal processing or in its user interface? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
On Mon, 2010-02-01 at 16:31 -0500, Bruce Momjian wrote: > > > > I would love to know why PL/Python can't be incrementally improved like > > > the rest of our code. > > > > It has been. That is exactly what PeterE has been doing. > > > > However, if you look at this whole thread, you will see the James has a > > very different view of the implementation. One that at least appears to > > be more advanced and "pythonic" than our version. > > More "pythonic" in its internal processing or in its user interface? User interface and also internal processing (see the types discussion). Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Feb 1, 2010, at 2:13 PM, Bruce Momjian wrote: > I would love to know why PL/Python can't be incrementally improved like > the rest of our code. AFAICT, there are two primary, perhaps identifying, parts to a PL extension: code management (compilation, execution, etc)and type I/O (conversion in most PLs). (well, aside from the language itself =) My proposed extension chooses a different design for both of those parts. It didn't make sense to try and incrementally change PL/Python because I would have been rewriting the whole thing anyways.Not to mention breaking user code in the process for the mentioned parts--thus the Python 3 target. Hope this clears things up.
On 2/1/10 1:39 PM, Joshua D. Drake wrote: > On Mon, 2010-02-01 at 16:31 -0500, Bruce Momjian wrote: >> >>>> I would love to know why PL/Python can't be incrementally improved like >>>> the rest of our code. >>> It has been. That is exactly what PeterE has been doing. >>> >>> However, if you look at this whole thread, you will see the James has a >>> very different view of the implementation. One that at least appears to >>> be more advanced and "pythonic" than our version. >> More "pythonic" in its internal processing or in its user interface? > > User interface and also internal processing (see the types discussion). Yeah, from the sound of it, we should put this in pgfoundry (or elsewhere) and have people try it out for 9.0. If the python folks love it, we can consider adding it to core, and then we can have the argument about whether to depreciate the older version. --Josh Berkus
On Mon, Feb 1, 2010 at 4:30 PM, Joshua D. Drake <jd@commandprompt.com> wrote: > On Mon, 2010-02-01 at 16:13 -0500, Bruce Momjian wrote: >> Alvaro Herrera wrote: >> > Peter Eisentraut escribi?: >> > > On m?n, 2010-02-01 at 12:01 -0800, Nathan Boley wrote: >> > > > I code nearly exclusively in python and C, but I have >> > > > often found pl/python to be very unwieldy. For this reason I often >> > > > use pl/perl or pl/pgsql for problems that, outside of postgres, I >> > > > would always use python. >> > > >> > > I find that curious, because much of the criticism about the current >> > > PL/Python can be traced back to the fact that the implementation used to >> > > be an exact copy of PL/Perl. >> > >> > Perhaps the problem is that PL/Perl used to be unwieldy back when >> > PL/Python was created. PL/Perl has definitely seen a lot more activity. >> >> I would love to know why PL/Python can't be incrementally improved like >> the rest of our code. > > It has been. That is exactly what PeterE has been doing. > > However, if you look at this whole thread, you will see the James has a > very different view of the implementation. One that at least appears to > be more advanced and "pythonic" than our version. I don't know if the native typing stuff is "more advanced" than our current code or not; that's kind of fuzzy terminology if you think about it. It is, however, a lot different than what we do in the existing PL/python, or, to the best of my knowledge, any of the other PLs with, perhaps, the exception of PL/pgsql. So conceivably someone could submit a PL/perlNG, a PL/lolcodeNG, etc. taking a similar approach. It's worth thinking about how we feel about that. ...Robert
On Feb 1, 2010, at 1:23 PM, Nathan Boley wrote: >> I think it would be great for you to review it... I doubt that will >> cause it to get committed for 9.0, but my doubt is no reason for you >> to hold off reviewing it. > > I assumed so, but the pretense of a chance will probably help to motivate me :-) > > I'll have something by Thursday, and then 'Returned with Feedback' > will at least be factual. I haven't updated the plpython3 branch in a while, so you may want to hit the github repo with the PGXS build: http://github.com/jwp/pg-python ...Should probably get the updated docs published too, but they are available in src/documentation as ReST files. If youhave sphinx installed, running `make html` in the root project directory should build them into src/sphinx/html.
On Mon, 2010-02-01 at 13:20 -0500, Robert Haas wrote: > On the basis of all of the foregoing, I don't think we can consider > this patch further for this CommitFest and will update > commitfest.postgresql.org accordingly. If the user community grows or > if one of the committers takes an interest in this down the road, I > think we could consider it for a future release. > I spoke with James offline about this as well. My understanding (correct me James) is that he is working on an implementation that can be installed via PGXS. Joshua D. Drake > ...Robert > -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Mon, 2010-02-01 at 12:01 -0800, Nathan Boley wrote: > > On the basis of all of the foregoing, I don't think we can consider > > this patch further for this CommitFest and will update > > commitfest.postgresql.org accordingly. > > FWIW, I am very excited about this patch and would be happy to review > it but have been very busy over the past month. If I can promise a > review by Thursday morning could we keep it active? Hopefully, at the > very least, I can provide some useful feedback and spawn some > community interest. > > I am worried that there is a bit of a chicken and an egg problem with > this patch. I code nearly exclusively in python and C, but I have > often found pl/python to be very unwieldy. For this reason I often > use pl/perl or pl/pgsql for problems that, outside of postgres, I > would always use python. From the documentation, this patch seems like > an enormous step in the right direction. I am a +1 Joshua D. Drake > > -Nathan > -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Mon, 2010-02-01 at 22:35 +0200, Peter Eisentraut wrote: > On mån, 2010-02-01 at 12:01 -0800, Nathan Boley wrote: > > I code nearly exclusively in python and C, but I have > > often found pl/python to be very unwieldy. For this reason I often > > use pl/perl or pl/pgsql for problems that, outside of postgres, I > > would always use python. > > I find that curious, because much of the criticism about the current > PL/Python can be traced back to the fact that the implementation used to > be an exact copy of PL/Perl. Well my guess is, if you want to code python, you don't want to feel like you are coding perl and thus you might as well just code perl? Joshua D. Drake > -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Mon, 2010-02-01 at 16:13 -0500, Bruce Momjian wrote: > Alvaro Herrera wrote: > > Peter Eisentraut escribi?: > > > On m?n, 2010-02-01 at 12:01 -0800, Nathan Boley wrote: > > > > I code nearly exclusively in python and C, but I have > > > > often found pl/python to be very unwieldy. For this reason I often > > > > use pl/perl or pl/pgsql for problems that, outside of postgres, I > > > > would always use python. > > > > > > I find that curious, because much of the criticism about the current > > > PL/Python can be traced back to the fact that the implementation used to > > > be an exact copy of PL/Perl. > > > > Perhaps the problem is that PL/Perl used to be unwieldy back when > > PL/Python was created. PL/Perl has definitely seen a lot more activity. > > I would love to know why PL/Python can't be incrementally improved like > the rest of our code. It has been. That is exactly what PeterE has been doing. However, if you look at this whole thread, you will see the James has a very different view of the implementation. One that at least appears to be more advanced and "pythonic" than our version. Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Mon, 2010-02-01 at 16:31 -0500, Bruce Momjian wrote: > > > > I would love to know why PL/Python can't be incrementally improved like > > > the rest of our code. > > > > It has been. That is exactly what PeterE has been doing. > > > > However, if you look at this whole thread, you will see the James has a > > very different view of the implementation. One that at least appears to > > be more advanced and "pythonic" than our version. > > More "pythonic" in its internal processing or in its user interface? User interface and also internal processing (see the types discussion). Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.
On Feb 1, 2010, at 12:18 PM, James William Pye wrote: > Right now, I'm trying to trim some of the easy issues[1] and getting a project web page up. I expect to be able to makea release soon, and I'll follow-up to this thread when I do. Well, I ended up doing some others things at that point in time, but I have managed to get a release out: http://python.projects.postgresql.org/backend/