Thread: Re: PlPython
Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:
> Attached is a patch that removes all of the RExec code from plpython from
> the current PostgreSQL CVS. In addition, plpython needs to be changed to an
> untrusted language in createlang.
I am inclined to rename plpython to plpythonu, by analogy to pltclu.
The advantage of doing so is that (a) the name change makes it somewhat
more obvious that there's a fundamental behavioral change, and (b)
assuming that the Python folk someday figure out a secure version of
RExec, we'd want to reinstitute the trusted version of plpython, but
perhaps not take away the untrusted one.
On the other hand, this would create headaches for people who are trying
to load dump files that declare plpython or contain plpython-language
functions. I can't think of any non-kluge solution to this (kluge
solutions would include putting special-case code into CREATE FUNCTION
to change 'plpython' to 'plpythonu' ...)
Comments?
regards, tom lane
elein <elein@varlena.com> writes:
> For 7.4 (which I expect is the patch's target) it might be
> best to make both names point to the same thing with a
> clear release note that says that they are the same thing
> and that plpython[u] is now untrusted.
I don't know any way to actually do that, though. If we put two entries
in pg_language then functions created in plpython will stay associated
with that entry. That'd probably be the worst of all possible worlds,
since a person looking at pg_language would quite reasonably assume that
plpython was still trusted and the untrusted plpythonu was just an
addition. (Especially if he happened to know that such an addition was
planned long ago.) You could shoot yourself in the foot pretty badly
with such a misunderstanding :-(
The behavior that I think would be most useful would be to automatically
transpose CREATE FUNCTION ... LANGUAGE "plpython" into CREATE FUNCTION
... LANGUAGE "plpythonu". Which we could do with an ugly hack in CREATE
FUNCTION (ugly, but no worse than things we've done to index opclass
names, for example). But it could be too confusing.
regards, tom lane
Tom Lane wrote: > elein <elein@varlena.com> writes: > > For 7.4 (which I expect is the patch's target) it might be > > best to make both names point to the same thing with a > > clear release note that says that they are the same thing > > and that plpython[u] is now untrusted. > > I don't know any way to actually do that, though. If we put two entries > in pg_language then functions created in plpython will stay associated > with that entry. That'd probably be the worst of all possible worlds, > since a person looking at pg_language would quite reasonably assume that > plpython was still trusted and the untrusted plpythonu was just an > addition. (Especially if he happened to know that such an addition was > planned long ago.) You could shoot yourself in the foot pretty badly > with such a misunderstanding :-( > > The behavior that I think would be most useful would be to automatically > transpose CREATE FUNCTION ... LANGUAGE "plpython" into CREATE FUNCTION > ... LANGUAGE "plpythonu". Which we could do with an ugly hack in CREATE > FUNCTION (ugly, but no worse than things we've done to index opclass > names, for example). But it could be too confusing. You mean in gram.y? Yes, I think that is our only choice. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Tom Lane wrote:
>> The behavior that I think would be most useful would be to automatically
>> transpose CREATE FUNCTION ... LANGUAGE "plpython" into CREATE FUNCTION
>> ... LANGUAGE "plpythonu". Which we could do with an ugly hack in CREATE
>> FUNCTION (ugly, but no worse than things we've done to index opclass
>> names, for example). But it could be too confusing.
> You mean in gram.y? Yes, I think that is our only choice.
Actually I think it should be in functioncmds.c. I moved the special
kluges for opclass names out of gram.y and into indexcmds.c awhile ago.
But that's just an implementation detail --- we really need to still be
thinking about whether this is the behavior we want or not. Someone
else made a fair point that such a kluge might not actually make life
any easier for reloading dump files. If we do it that way, then if a
non-superuser tries to CREATE FUNCTION ... LANGUAGE "plpython" it will
fail (not being trusted) and so he's got no hope of loading the dump
without editing anyway. If that's true, there's not much point in
introducing a hidden kluge.
regards, tom lane
Tom Lane wrote: > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > Tom Lane wrote: > >> The behavior that I think would be most useful would be to automatically > >> transpose CREATE FUNCTION ... LANGUAGE "plpython" into CREATE FUNCTION > >> ... LANGUAGE "plpythonu". Which we could do with an ugly hack in CREATE > >> FUNCTION (ugly, but no worse than things we've done to index opclass > >> names, for example). But it could be too confusing. > > > You mean in gram.y? Yes, I think that is our only choice. > > Actually I think it should be in functioncmds.c. I moved the special > kluges for opclass names out of gram.y and into indexcmds.c awhile ago. > But that's just an implementation detail --- we really need to still be > thinking about whether this is the behavior we want or not. Someone > else made a fair point that such a kluge might not actually make life > any easier for reloading dump files. If we do it that way, then if a > non-superuser tries to CREATE FUNCTION ... LANGUAGE "plpython" it will > fail (not being trusted) and so he's got no hope of loading the dump > without editing anyway. If that's true, there's not much point in > introducing a hidden kluge. Well, it does fix the super-user case, so we have to tell non-super users to get their administrator to install it, which actually is the right solution anyway for non-super-user installs of plpython language modules anyway. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Hi Tom, > I am inclined to rename plpython to plpythonu, by analogy to pltclu. > The advantage of doing so is that (a) the name change makes it somewhat > more obvious that there's a fundamental behavioral change, and (b) > assuming that the Python folk someday figure out a secure version of > RExec, we'd want to reinstitute the trusted version of plpython, but > perhaps not take away the untrusted one. Sounds good. > On the other hand, this would create headaches for people who are trying > to load dump files that declare plpython or contain plpython-language > functions. But since plpython is untrusted now this probably would not work anyway... > I can't think of any non-kluge solution to this (kluge > solutions would include putting special-case code into CREATE FUNCTION > to change 'plpython' to 'plpythonu' ...) I think this would only make it more confusing. Changing plpython from trusted to untrusted is a big change, so some extra attention from the db-admin and users is good IMHO. Bye, Sander.
On Sun, 22 Jun 2003, Tom Lane wrote: > Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes: > > Attached is a patch that removes all of the RExec code from plpython from > > the current PostgreSQL CVS. In addition, plpython needs to be changed to an > > untrusted language in createlang. > > I am inclined to rename plpython to plpythonu, by analogy to pltclu. In 7.4, I would do this. In the next 7.3.x release, I would leave the name as is, but still make plpython untrusted. It will cause some problems, but safety and integrity are more important here. After all, the alternative is to drop the PL entirely. > On the other hand, this would create headaches for people who are trying > to load dump files that declare plpython or contain plpython-language > functions. I can't think of any non-kluge solution to this (kluge > solutions would include putting special-case code into CREATE FUNCTION > to change 'plpython' to 'plpythonu' ...) Allowing both plpython and plpythonu to be added as separate languages should be sufficient. This way, we effectively deprecate plpython, or at least defer its final removal until the restricted execution problem can be solved. I hope to start looking into exactly what needs to be done to restore that functionality in the next few weeks. Thanks, -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com
Tom Lane kirjutas E, 23.06.2003 kell 01:29: > Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes: > > Attached is a patch that removes all of the RExec code from plpython from > > the current PostgreSQL CVS. In addition, plpython needs to be changed to an > > untrusted language in createlang. > > I am inclined to rename plpython to plpythonu, by analogy to pltclu. ... > Comments? could we not just make sure that plpython uses python ver < 2.x and use plpythonu for python versions >= 2.x until a secure regex solution comes from Guido and folks ? I guess most plpython users would be much happier with plpython with some minor limitations due to older version than with being forced to use an untrusted pl altogether. IIRC python 1.5.2 has a perfectly good RExec. Or is there a requirement that only latest language versions are used in pg 74 ;) -------------- Hannu
Hannu Krosing <hannu@tm.ee> writes:
> could we not just make sure that plpython uses python ver < 2.x and use
> plpythonu for python versions >= 2.x until a secure regex solution comes
> from Guido and folks ?
We'd still have to mark it untrusted, so what's the point?
regards, tom lane
Tom Lane kirjutas E, 30.06.2003 kell 00:18: > Hannu Krosing <hannu@tm.ee> writes: > > could we not just make sure that plpython uses python ver < 2.x and use > > plpythonu for python versions >= 2.x until a secure regex solution comes > > from Guido and folks ? > > We'd still have to mark it untrusted, so what's the point? No we don't! The old version of plpython was perfectly OK when used with python 1.5.x and will be so. The RExec security holes were only introduced with new class mechanisms in python 2.x. The version with patch which removes RExec (as Python 2.x is not supporting it ) is the right thoing to do FOR PYTHON 2.X, but there is no reason to remove safe execution when using python 1.5.x. Thus my proposition for using the old version as plpython and the new version as plpython-u, but allowing the non-u version to be compuled only for python v < 2.x. ----------------- Hannu
Hannu Krosing <hannu@tm.ee> writes:
> The version with patch which removes RExec (as Python 2.x is not
> supporting it ) is the right thoing to do FOR PYTHON 2.X, but there is
> no reason to remove safe execution when using python 1.5.x.
Who's still using 1.5, I guess is the question? And are they likely
to be updating their PG installation when they're not updating Python?
regards, tom lane
Tom Lane kirjutas E, 30.06.2003 kell 01:21: > Hannu Krosing <hannu@tm.ee> writes: > > The version with patch which removes RExec (as Python 2.x is not > > supporting it ) is the right thoing to do FOR PYTHON 2.X, but there is > > no reason to remove safe execution when using python 1.5.x. > > Who's still using 1.5, I guess is the question? And are they likely > to be updating their PG installation when they're not updating Python? Python is designed such that one can install and use different versions in parallel - for example the deafult directopries for libraries are /usr/lib/python1.5/ and /usr/lib/python2.2/ if you have installed python 1.5.x and 2.2.x, also executables are python2, python2.2 and python1.5, with plain python being a link to one of these. I guess that anyone who needs safe Restricted Execution will be using 1.5 at least for that purpose until RExec is fixed in 2.x. ------------------ Hannu
Hannu Krosing <hannu@tm.ee> writes:
> Tom Lane kirjutas E, 30.06.2003 kell 01:21:
>> Who's still using 1.5, I guess is the question? And are they likely
>> to be updating their PG installation when they're not updating Python?
> I guess that anyone who needs safe Restricted Execution will be using
> 1.5 at least for that purpose until RExec is fixed in 2.x.
I don't find that real compelling ...
The bottom line is that this has to get done. I have the time to
convert plpython to untrusted status tomorrow. I do not have the time,
the infrastructure, nor the interest to build a conditional setup.
Unless someone else wants to volunteer to make it happen in a timely
fashion, untrusted is what it will be.
regards, tom lane
Hannu Krosing wrote: > could we not just make sure that plpython uses python ver < 2.x and use > plpythonu for python versions >= 2.x until a secure regex solution comes > from Guido and folks ? > > I guess most plpython users would be much happier with plpython with > some minor limitations due to older version than with being forced to > use an untrusted pl altogether. But if rexec isn't safe they're just fooling themselves. There's only two kinds of safety for restricted environments: absolute and not. That's why the Python developers were honest and disabled rexec for now. If you want to fool yourself, that's easy enough: ship a modified rexec.py with the 'raise RuntimeError, "This code is not secure ..."' removed ;-) > IIRC python 1.5.2 has a perfectly good RExec. You are likely mistaken. Because I was interested in getting this problem solved for plpython and because most rexec problems are because of the new-style classes in Python 2.2 and later, I asked on comp.lang.python wether it was safe with 2.1 and earlier. Martin von Löwis told me it probably wasn't in http://groups.google.com/groups?selm=m3y8ztib79.fsf%40mira.informatik.hu-berlin.de > Or is there a requirement that only latest language versions are used in > pg 74 ;) No, but I find it hard to believe that PL/python is used by untrusted users at all, so making it untrusted might not really be a problem in real life. -- Gerhard PS: Thanks Kevin for submitting the PL/Python patch. I intended to make it available at least as an untrusted language, but you beat me :)
Gerhard Häring <gh@ghaering.de> writes: > ... because most rexec problems are because > of the new-style classes in Python 2.2 and later, I asked on > comp.lang.python wether it was safe with 2.1 and earlier. > Martin von L�wis told me it probably wasn't in > http://groups.google.com/groups?selm=m3y8ztib79.fsf%40mira.informatik.hu-berlin.de Hmm. I don't think the second bug mentioned there applies to us (if our use of rexec allows the Python code to write any file anywhere, it's already lost its right to trusted status IMHO). But the first one looks like a killer: http://sourceforge.net/tracker/?group_id=5470&atid=105470&func=detail&aid=577530 According to that discussion, this bug does exist in Python 1.5 :-( ... so Hannu's argument rests on a false premise anyway. regards, tom lane
Tom Lane kirjutas E, 30.06.2003 kell 06:39: > Hannu Krosing <hannu@tm.ee> writes: > > Tom Lane kirjutas E, 30.06.2003 kell 01:21: > >> Who's still using 1.5, I guess is the question? And are they likely > >> to be updating their PG installation when they're not updating Python? > > > I guess that anyone who needs safe Restricted Execution will be using > > 1.5 at least for that purpose until RExec is fixed in 2.x. > > I don't find that real compelling ... > > The bottom line is that this has to get done. I have the time to > convert plpython to untrusted status tomorrow. I do not have the time, > the infrastructure, nor the interest to build a conditional setup. > Unless someone else wants to volunteer to make it happen in a timely > fashion, untrusted is what it will be. Fine with me. Just don't put in any hacks to pretend it is trusted. --------------- Hannu
Gerhard Häring kirjutas E, 30.06.2003 kell 07:39: > Hannu Krosing wrote: > > IIRC python 1.5.2 has a perfectly good RExec. > > You are likely mistaken. Because I was interested in getting this > problem solved for plpython and because most rexec problems are because > of the new-style classes in Python 2.2 and later, I asked on > comp.lang.python wether it was safe with 2.1 and earlier. > > Martin von Löwis told me it probably wasn't in > http://groups.google.com/groups?selm=m3y8ztib79.fsf%40mira.informatik.hu-berlin.de Uups! I got that impression from the emails which claimed that new-style classes make it near impossilble to fix. > > Or is there a requirement that only latest language versions are used in > > pg 74 ;) > > No, but I find it hard to believe that PL/python is used by untrusted > users at all, so making it untrusted might not really be a problem in > real life. That is most likely true. And trusted users like to use untrusted languages, so they can send email, get xml via http and write files ;) ------------- Hannu
Hannu Krosing writes: > Tom Lane kirjutas E, 23.06.2003 kell 01:29: > > Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes: > > > Attached is a patch that removes all of the RExec code from plpython > > > from the current PostgreSQL CVS. In addition, plpython needs to be > > > changed to an untrusted language in createlang. > > > > I am inclined to rename plpython to plpythonu, by analogy to pltclu. > > ... > > > Comments? > > could we not just make sure that plpython uses python ver < 2.x and use > plpythonu for python versions >= 2.x until a secure regex solution comes > from Guido and folks ? > > I guess most plpython users would be much happier with plpython with > some minor limitations due to older version than with being forced to > use an untrusted pl altogether. Actually, there are also people (me, for example :-), who would love to see an untrusted PlPython to use the full Python power as Pl. So I'd be very happy with a solution, where I could choose between plpython and plpythonu. But I don't know if that means having twice the effort compared to only having plpython _or_ plpythonu. Regards, Tilo
For 7.4 (which I expect is the patch's target) it might be best to make both names point to the same thing with a clear release note that says that they are the same thing and that plpython[u] is now untrusted. That will give people a bit a time to reload their existing functions. elein On Sunday 22 June 2003 15:29, Tom Lane wrote: > Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes: > > Attached is a patch that removes all of the RExec code from plpython from > > the current PostgreSQL CVS. In addition, plpython needs to be changed to an > > untrusted language in createlang. > > I am inclined to rename plpython to plpythonu, by analogy to pltclu. > The advantage of doing so is that (a) the name change makes it somewhat > more obvious that there's a fundamental behavioral change, and (b) > assuming that the Python folk someday figure out a secure version of > RExec, we'd want to reinstitute the trusted version of plpython, but > perhaps not take away the untrusted one. > > On the other hand, this would create headaches for people who are trying > to load dump files that declare plpython or contain plpython-language > functions. I can't think of any non-kluge solution to this (kluge > solutions would include putting special-case code into CREATE FUNCTION > to change 'plpython' to 'plpythonu' ...) > > Comments? > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 9: the planner will ignore your desire to choose an index scan if your > joining column's datatypes do not match > > -- ============================================================= elein@varlena.com Database Consulting www.varlena.com PostgreSQL General Bits http:/www.varlena.com/GeneralBits/ "Free your mind the rest will follow" -- en vogue
I thought there would be a relatively clear way to alias them both to the same language library for a release or two. But I see your point on transitioning. Clear notice is really important. plpython should be phased out if it is not replaced within a release or two. If only the change could be transparent to those people who are not using any untrusted features. Maybe it is better to break everyone's plpython functions in 7.4. That would certainly make sure everyone heard of the change from trusted to untrusted. But explanations of what exactly that means should also be included in the release notes. elein PS: I've built and tested the plpython patch against 7.3.2 and am happy it does not affect the features I count on. I'd do it against the development release but I can't get it to build (for other reasons). On Monday 23 June 2003 13:53, Tom Lane wrote: > elein <elein@varlena.com> writes: > > For 7.4 (which I expect is the patch's target) it might be > > best to make both names point to the same thing with a > > clear release note that says that they are the same thing > > and that plpython[u] is now untrusted. > > I don't know any way to actually do that, though. If we put two entries > in pg_language then functions created in plpython will stay associated > with that entry. That'd probably be the worst of all possible worlds, > since a person looking at pg_language would quite reasonably assume that > plpython was still trusted and the untrusted plpythonu was just an > addition. (Especially if he happened to know that such an addition was > planned long ago.) You could shoot yourself in the foot pretty badly > with such a misunderstanding :-( > > The behavior that I think would be most useful would be to automatically > transpose CREATE FUNCTION ... LANGUAGE "plpython" into CREATE FUNCTION > ... LANGUAGE "plpythonu". Which we could do with an ugly hack in CREATE > FUNCTION (ugly, but no worse than things we've done to index opclass > names, for example). But it could be too confusing. > > regards, tom lane > > -- ============================================================= elein@varlena.com Database Consulting www.varlena.com PostgreSQL General Bits http:/www.varlena.com/GeneralBits/ "Free your mind the rest will follow" -- en vogue
On 30 Jun 2003, Hannu Krosing wrote: > Tom Lane kirjutas E, 30.06.2003 kell 01:21: > > Hannu Krosing <hannu@tm.ee> writes: > > > The version with patch which removes RExec (as Python 2.x is not > > > supporting it ) is the right thoing to do FOR PYTHON 2.X, but there is > > > no reason to remove safe execution when using python 1.5.x. > > > > Who's still using 1.5, I guess is the question? And are they likely > > to be updating their PG installation when they're not updating Python? > > Python is designed such that one can install and use different versions > in parallel - for example the deafult directopries for libraries are > /usr/lib/python1.5/ and /usr/lib/python2.2/ if you have installed python > 1.5.x and 2.2.x, also executables are python2, python2.2 and python1.5, > with plain python being a link to one of these. > > I guess that anyone who needs safe Restricted Execution will be using > 1.5 at least for that purpose until RExec is fixed in 2.x. Unless a someone steps forward to actively maintain such a configuration, then trusted PlPython is not going to exist anymore. The cost of getting even the most minute detail wrong is just too high tobe left to chance. -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com
elein kirjutas T, 24.06.2003 kell 00:42: There is a realtively clean hack one can use to convert plpython functions to plpythonu manually - just rename the language for the time of loading functions - do as superuser update pg_language set lanname = 'plpython' where lanname = 'plpythonu'; LOAD YOUR Pl/Python FUNCTIONS ; update pg_language set lanname = 'plpythonu' where lanname = 'plpython'; > PS: I've built and tested the plpython patch against > 7.3.2 and am happy it does not affect the features I count > on. As it should. The untrusted language gives you *more* power, not less. The untrusted status means that the user has to be trusted to use that much power. ---------------- Hannu