Thread: Pl/Python -- current maintainer?
I'm interested in poking though and taking a shot at getting my feet wet with pl/python. I see the file is copyright Andrew Bosma -- is he still around perhance? Is anyone currently the 'owner' ? ---- James Robinson Socialserve.com
James Robinson wrote: > I'm interested in poking though and taking a shot at getting my feet > wet with pl/python. I see the file is copyright Andrew Bosma -- is he > still around perhance? Is anyone currently the 'owner' ? To my knowledge there is no current maintainer of plPython and it definitely needs some love. > > ---- > James Robinson > Socialserve.com > > > ---------------------------(end of broadcast)--------------------------- > TIP 1: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly > -- The PostgreSQL Company - Command Prompt, Inc. 1.503.667.4564 PostgreSQL Replication, Consulting, Custom Development, 24x7 support Managed Services, Shared and Dedicated Hosting Co-Authors: PLphp, PLperl - http://www.commandprompt.com/
I see neilc has hacked on it very recently to reduce memory leaks. I take that as both good and bad signs. We're a [ small ] python shop, and would be most interested in being able to simplify our life through doing some things in plpython instead of pl/pgsql where appropriate. Keeping our constants and so forth in the appropriate python module would make things ever so much simpler here and there at the very least. But we've never hacked on the backend, nor at the C python API level. But I see no reason why not to start now -- lurked here for many a year. For example, I see that plpython functions cannot be declared to return void. That can't be too tough to remedy. Implementing the DBI 2.0 API interface to SPI can wait another day. On Feb 24, 2006, at 11:08 PM, Joshua D. Drake wrote: > James Robinson wrote: >> I'm interested in poking though and taking a shot at getting my >> feet wet with pl/python. I see the file is copyright Andrew Bosma >> -- is he still around perhance? Is anyone currently the 'owner' ? > To my knowledge there is no current maintainer of plPython and it > definitely needs some love. > >> >> ---- >> James Robinson >> Socialserve.com >> >> >> ---------------------------(end of >> broadcast)--------------------------- >> TIP 1: if posting/reading through Usenet, please send an appropriate >> subscribe-nomail command to majordomo@postgresql.org so that >> your >> message can get through to the mailing list cleanly >> > > > -- > The PostgreSQL Company - Command Prompt, Inc. 1.503.667.4564 > PostgreSQL Replication, Consulting, Custom Development, 24x7 support > Managed Services, Shared and Dedicated Hosting > Co-Authors: PLphp, PLperl - http://www.commandprompt.com/ > ---- James Robinson Socialserve.com
James Robinson schrieb: > I see neilc has hacked on it very recently to reduce memory leaks. I > take that as both good and bad signs. > > We're a [ small ] python shop, and would be most interested in being > able to simplify our life through doing some things in plpython instead > of pl/pgsql where appropriate. Keeping our constants and so forth in > the appropriate python module would make things ever so much simpler > here and there at the very least. > > But we've never hacked on the backend, nor at the C python API level. > But I see no reason why not to start now -- lurked here for many a > year. For example, I see that plpython functions cannot be declared to > return void. That can't be too tough to remedy. Implementing the DBI > 2.0 API interface to SPI can wait another day. Also have a look at: http://python.projects.postgresql.org/ it needs some more love too but has high potential. Maybe it can become next generation pl/pythonu? Would be nice. And with even more love the restricted python from zope could be ported so there could be a pl/python again :-) Ok, just haluzinating ;) Tino
Ühel kenal päeval, L, 2006-02-25 kell 10:09, kirjutas Tino Wildenhain: > James Robinson schrieb: > > I see neilc has hacked on it very recently to reduce memory leaks. I > > take that as both good and bad signs. > > > > We're a [ small ] python shop, and would be most interested in being > > able to simplify our life through doing some things in plpython instead > > of pl/pgsql where appropriate. Keeping our constants and so forth in > > the appropriate python module would make things ever so much simpler > > here and there at the very least. > > > > But we've never hacked on the backend, nor at the C python API level. > > But I see no reason why not to start now -- lurked here for many a > > year. For example, I see that plpython functions cannot be declared to > > return void. That can't be too tough to remedy. Implementing the DBI > > 2.0 API interface to SPI can wait another day. > > Also have a look at: http://python.projects.postgresql.org/ > it needs some more love too but has high potential. Yes, this one seems to be the pl/python done right :) But it also suffers a little from trying to do too much at one time, and so moves a little slow :( > Maybe it can become next generation pl/pythonu? Would be nice. I guess that it would need to be at least somewhat backwards compatible to replace current pl/pythonu. > And with even more love the restricted python from zope could > be ported so there could be a pl/python again :-) That would be nice, but actually not very high on my list of wishes, as I mostly want to use plpythony as a replacement for writing C funtions, and we probably will never have "restricted C" -------------- Hannu
On Sat, Feb 25, 2006 at 10:09:52AM +0100, Tino Wildenhain wrote:
> And with even more love the restricted python from zope could
> be ported so there could be a pl/python again :-)
>
> Ok, just haluzinating ;)
Not necessarily. ;)
From what I have seen of zope's restricted python, it does, or can, force its
restrictions by checking bytecode. I imagine a simple PL sitting on top of the
untrusted varient that merely implements a custom validator that checks the
bytecode produced by the untrusted PL's validator. The language handler would
remain the same:
<Create untrusted plpy...>
CREATE FUNCTION zope_restrict(oid) RETURNS VOID LANGUAGE python AS
$$
getcode = Postgres.CachedQuery("SELECT probin FROM pg_proc WHERE oid = $1")
getlangval = Postgres.CachedQuery("""SELECT lanvalidator FROM pg_language WHERE oid = (SELECT prolang FROM pg_proc
WHEREoid = $1)
""")
langvaloid = getlangval(self.function.procid)
langval = Postgres.Function(langvaloid)
# produce the function's bytecode for checking
langval(args[0])
# get the function's newly created bytecode
code = getcode(args[0]).next()[0]
#
# XXX: do the zope restriction stuff, raise exception if necessary
#
Postgres.NOTICE(repr(code))
$$;
CREATE TRUSTED LANGUAGE plpyr HANDLER python.handler VALIDATOR zope_restrict;
[This almost works in plpy head, but I think I just found a bug ;]
--
Regards, James William Pye
James William Pye wrote: > On Sat, Feb 25, 2006 at 10:09:52AM +0100, Tino Wildenhain wrote: > > And with even more love the restricted python from zope could > > be ported so there could be a pl/python again :-) > > > > Ok, just haluzinating ;) > > Not necessarily. ;) > > From what I have seen of zope's restricted python, it does, or can, force its > restrictions by checking bytecode. I imagine a simple PL sitting on top of the > untrusted varient that merely implements a custom validator that checks the > bytecode produced by the untrusted PL's validator. I'm not sure it's an issue now that we have pg_pltemplate, but in older versions it's possible to create a language without setting a validator. This would make the validator an unsuitable place for checking the restrictions. But the call handler can access the bytecode just the same, so it's just a matter of moving the checks there, just before the execution. -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
On Sat, Feb 25, 2006 at 06:36:19PM -0300, Alvaro Herrera wrote: > I'm not sure it's an issue now that we have pg_pltemplate, but in older > versions it's possible to create a language without setting a validator. > This would make the validator an unsuitable place for checking the > restrictions. Hrm. I think this would only be an issue in PL/Py is if the user had the ability to alter probin. The handler will never directly execute code in prosrc; it relies on a validator to fill in probin. Whether a regular user could take advantage of this or not, I'm not sure as I have yet to test it or to give it much thought. -- Regards, James William Pye
On Sat, Feb 25, 2006 at 01:21:34PM -0700, I wrote: > From what I have seen of zope's restricted python, it does, or can, force its > restrictions by checking bytecode. I imagine a simple PL sitting on top of the > untrusted varient that merely implements a custom validator that checks the > bytecode produced by the untrusted PL's validator. The language handler would > remain the same: [ugh, Correcting my assumptions...] Zope's RestrictedPython is a custom bytecode generator that compiles Python code specially, as opposed to a bytecode processor that validates against some rule set as I had thought for some (wishful? ;) reason. The bytecode then needs to be executed in an special environment that then imposes some specified restrictions at runtime(I'm not really clear on all the details here as I am having a very difficult time finding documentation). This doesn't mean that it couldn't be used. However, it does mean that some munging of the handler would be required(Something that I desired to avoid). -- Regards, James William Pye
James William Pye <pgsql@jwp.name> writes:
> On Sat, Feb 25, 2006 at 06:36:19PM -0300, Alvaro Herrera wrote:
>> I'm not sure it's an issue now that we have pg_pltemplate, but in older
>> versions it's possible to create a language without setting a validator.
>> This would make the validator an unsuitable place for checking the
>> restrictions.
> Hrm. I think this would only be an issue in PL/Py is if the user had the ability
> to alter probin. The handler will never directly execute code in prosrc; it
> relies on a validator to fill in probin.
That design is broken on its face, as the system does not guarantee to
call the validator.
regards, tom lane
On Sun, Feb 26, 2006 at 01:08:52PM -0500, Tom Lane wrote: > That design is broken on its face, as the system does not guarantee to > call the validator. Hrm. Other than language creations that do not specify a validator, at what times will Postgres not call the validator upon function creation? -- Regards, James William Pye
James William Pye wrote: > On Sun, Feb 26, 2006 at 01:08:52PM -0500, Tom Lane wrote: >> That design is broken on its face, as the system does not guarantee to >> call the validator. > > Hrm. Other than language creations that do not specify a validator, at > what > times will Postgres not call the validator upon function creation? SET check_function_bodies = off; Best Regards, Michael Paesold
James William Pye schrieb: > On Sat, Feb 25, 2006 at 01:21:34PM -0700, I wrote: > >>From what I have seen of zope's restricted python, it does, or can, force its >>restrictions by checking bytecode. I imagine a simple PL sitting on top of the >>untrusted varient that merely implements a custom validator that checks the >>bytecode produced by the untrusted PL's validator. The language handler would >>remain the same: > > [ugh, Correcting my assumptions...] > > Zope's RestrictedPython is a custom bytecode generator that compiles Python > code specially, as opposed to a bytecode processor that validates against some > rule set as I had thought for some (wishful? ;) reason. The bytecode then needs The key point is: it replaces dangerous elements while it compiles the bytecode - in theory you could also walk the tree after the python bytecode compiler (not sure if it even works this way) for example eval() open() file() import, ... are/can be replaced in this step. > to be executed in an special environment that then imposes some specified > restrictions at runtime(I'm not really clear on all the details here as I > am having a very difficult time finding documentation). The special environment is there for the fine grained security only zope would need in this case. (It protects object attributes individually while maintaining acquisition and all that stuff) > This doesn't mean that it couldn't be used. However, it does mean that some > munging of the handler would be required(Something that I desired to avoid). You should be able to use most of that technique in the stage where you create the bytecode - just the step pl/plsql does too. Regards Tino
On Sun, Feb 26, 2006 at 07:36:01PM +0100, Michael Paesold wrote: > SET check_function_bodies = off; Hrm, thanks for pointing that out. However, this, thankfully, does not appear to inhibit the validator at all. Rather, the PL's validator is left with the job to respect it if need be: jwp=# select version(); version ------------------------------------------------------------------------------------ -------------PostgreSQL 8.1.3 on i386-unknown-freebsd6.1, compiled by GCC gcc (GCC) 3.4.4 [FreeB SD] 20050518 (1 row) jwp=# set check_function_bodies = off; SET jwp=# SHOW check_function_bodies ;check_function_bodies -----------------------off (1 row) jwp=# CREATE FUNCTION test() RETURNS int LANGUAGE python AS 'return 1'; CREATE FUNCTION jwp=# SELECT prolang, probin FROM pg_proc WHERE proname = 'test';prolang | probin ---------+-------------------------------------------------------------------------- ------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------ -------------------------------------------------------------------------------- 16390 | c\002\000\000\000\002\000\000\000\001\000\000\000C\000\000\000s\004\000\0 00\000d\001\000S(\002\000\000\000Ni\001\000\000\000(\000\000\000\000(\002\000\000\00 0s\004\000\000\000selfs\004\000\000\000args(\000\000\000\000(\000\000\000\000s\005\0 00\000\00069121s\006\000\000\000test()\002\000\000\000s\002\000\000\000\000\001 (1 row) jwp=# SELECT test();test ------ 1 (1 row) ^D This is a good thing, IMO. Python primarily does a syntax check so the cited[1] reasons for the use of check_function_bodies do not seem to apply here. (Not sure if this is different on 8.0 or earlier.) [1] http://www.postgresql.org/docs/8.1/static/runtime-config-client.html -- Regards, James William Pye