Thread: Re: PlPython

Re: PlPython

From
Tom Lane
Date:
Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:
> Attached is a patch that removes all of the RExec code from plpython from
> the current PostgreSQL CVS.  In addition, plpython needs to be changed to an
> untrusted language in createlang.

I am inclined to rename plpython to plpythonu, by analogy to pltclu.
The advantage of doing so is that (a) the name change makes it somewhat
more obvious that there's a fundamental behavioral change, and (b)
assuming that the Python folk someday figure out a secure version of
RExec, we'd want to reinstitute the trusted version of plpython, but
perhaps not take away the untrusted one.

On the other hand, this would create headaches for people who are trying
to load dump files that declare plpython or contain plpython-language
functions.  I can't think of any non-kluge solution to this (kluge
solutions would include putting special-case code into CREATE FUNCTION
to change 'plpython' to 'plpythonu' ...)

Comments?

            regards, tom lane

Re: [GENERAL] PlPython

From
Tom Lane
Date:
elein  <elein@varlena.com> writes:
> For 7.4 (which I expect is the patch's target) it might be
> best to make both names point to the same thing with a
> clear release note that says that they are the same thing
> and that plpython[u] is now untrusted.

I don't know any way to actually do that, though.  If we put two entries
in pg_language then functions created in plpython will stay associated
with that entry.  That'd probably be the worst of all possible worlds,
since a person looking at pg_language would quite reasonably assume that
plpython was still trusted and the untrusted plpythonu was just an
addition.  (Especially if he happened to know that such an addition was
planned long ago.)  You could shoot yourself in the foot pretty badly
with such a misunderstanding :-(

The behavior that I think would be most useful would be to automatically
transpose CREATE FUNCTION ... LANGUAGE "plpython" into CREATE FUNCTION
... LANGUAGE "plpythonu".  Which we could do with an ugly hack in CREATE
FUNCTION (ugly, but no worse than things we've done to index opclass
names, for example).  But it could be too confusing.

            regards, tom lane

Re: [GENERAL] PlPython

From
Bruce Momjian
Date:
Tom Lane wrote:
> elein  <elein@varlena.com> writes:
> > For 7.4 (which I expect is the patch's target) it might be
> > best to make both names point to the same thing with a
> > clear release note that says that they are the same thing
> > and that plpython[u] is now untrusted.
>
> I don't know any way to actually do that, though.  If we put two entries
> in pg_language then functions created in plpython will stay associated
> with that entry.  That'd probably be the worst of all possible worlds,
> since a person looking at pg_language would quite reasonably assume that
> plpython was still trusted and the untrusted plpythonu was just an
> addition.  (Especially if he happened to know that such an addition was
> planned long ago.)  You could shoot yourself in the foot pretty badly
> with such a misunderstanding :-(
>
> The behavior that I think would be most useful would be to automatically
> transpose CREATE FUNCTION ... LANGUAGE "plpython" into CREATE FUNCTION
> ... LANGUAGE "plpythonu".  Which we could do with an ugly hack in CREATE
> FUNCTION (ugly, but no worse than things we've done to index opclass
> names, for example).  But it could be too confusing.

You mean in gram.y?  Yes, I think that is our only choice.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: [GENERAL] PlPython

From
Tom Lane
Date:
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Tom Lane wrote:
>> The behavior that I think would be most useful would be to automatically
>> transpose CREATE FUNCTION ... LANGUAGE "plpython" into CREATE FUNCTION
>> ... LANGUAGE "plpythonu".  Which we could do with an ugly hack in CREATE
>> FUNCTION (ugly, but no worse than things we've done to index opclass
>> names, for example).  But it could be too confusing.

> You mean in gram.y?  Yes, I think that is our only choice.

Actually I think it should be in functioncmds.c.  I moved the special
kluges for opclass names out of gram.y and into indexcmds.c awhile ago.
But that's just an implementation detail --- we really need to still be
thinking about whether this is the behavior we want or not.  Someone
else made a fair point that such a kluge might not actually make life
any easier for reloading dump files.  If we do it that way, then if a
non-superuser tries to CREATE FUNCTION ... LANGUAGE "plpython" it will
fail (not being trusted) and so he's got no hope of loading the dump
without editing anyway.  If that's true, there's not much point in
introducing a hidden kluge.

            regards, tom lane

Re: [GENERAL] PlPython

From
Bruce Momjian
Date:
Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > Tom Lane wrote:
> >> The behavior that I think would be most useful would be to automatically
> >> transpose CREATE FUNCTION ... LANGUAGE "plpython" into CREATE FUNCTION
> >> ... LANGUAGE "plpythonu".  Which we could do with an ugly hack in CREATE
> >> FUNCTION (ugly, but no worse than things we've done to index opclass
> >> names, for example).  But it could be too confusing.
>
> > You mean in gram.y?  Yes, I think that is our only choice.
>
> Actually I think it should be in functioncmds.c.  I moved the special
> kluges for opclass names out of gram.y and into indexcmds.c awhile ago.
> But that's just an implementation detail --- we really need to still be
> thinking about whether this is the behavior we want or not.  Someone
> else made a fair point that such a kluge might not actually make life
> any easier for reloading dump files.  If we do it that way, then if a
> non-superuser tries to CREATE FUNCTION ... LANGUAGE "plpython" it will
> fail (not being trusted) and so he's got no hope of loading the dump
> without editing anyway.  If that's true, there's not much point in
> introducing a hidden kluge.

Well, it does fix the super-user case, so we have to tell non-super
users to get their administrator to install it, which actually is the
right solution anyway for non-super-user installs of plpython language
modules anyway.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: PlPython

From
"Sander Steffann"
Date:
Hi Tom,

> I am inclined to rename plpython to plpythonu, by analogy to pltclu.
> The advantage of doing so is that (a) the name change makes it somewhat
> more obvious that there's a fundamental behavioral change, and (b)
> assuming that the Python folk someday figure out a secure version of
> RExec, we'd want to reinstitute the trusted version of plpython, but
> perhaps not take away the untrusted one.

Sounds good.

> On the other hand, this would create headaches for people who are trying
> to load dump files that declare plpython or contain plpython-language
> functions.

But since plpython is untrusted now this probably would not work anyway...

> I can't think of any non-kluge solution to this (kluge
> solutions would include putting special-case code into CREATE FUNCTION
> to change 'plpython' to 'plpythonu' ...)

I think this would only make it more confusing. Changing plpython from trusted
to untrusted is a big change, so some extra attention from the db-admin and
users is good IMHO.

Bye,
Sander.



Re: PlPython

From
Kevin Jacobs
Date:
On Sun, 22 Jun 2003, Tom Lane wrote:
> Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:
> > Attached is a patch that removes all of the RExec code from plpython from
> > the current PostgreSQL CVS.  In addition, plpython needs to be changed to an
> > untrusted language in createlang.
> 
> I am inclined to rename plpython to plpythonu, by analogy to pltclu.

In 7.4, I would do this.  In the next 7.3.x release, I would leave the name
as is, but still make plpython untrusted.  It will cause some problems, but
safety and integrity are more important here.  After all, the alternative is
to drop the PL entirely.

> On the other hand, this would create headaches for people who are trying
> to load dump files that declare plpython or contain plpython-language
> functions.  I can't think of any non-kluge solution to this (kluge
> solutions would include putting special-case code into CREATE FUNCTION
> to change 'plpython' to 'plpythonu' ...)

Allowing both plpython and plpythonu to be added as separate languages
should be sufficient.  This way, we effectively deprecate plpython, or at
least defer its final removal until the restricted execution problem can be
solved.  I hope to start looking into exactly what needs to be done to
restore that functionality in the next few weeks.

Thanks,
-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com



Re: PlPython

From
Hannu Krosing
Date:
Tom Lane kirjutas E, 23.06.2003 kell 01:29:
> Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:
> > Attached is a patch that removes all of the RExec code from plpython from
> > the current PostgreSQL CVS.  In addition, plpython needs to be changed to an
> > untrusted language in createlang.
>
> I am inclined to rename plpython to plpythonu, by analogy to pltclu.

...

> Comments?

could we not just make sure that plpython uses python ver < 2.x and use
plpythonu for python versions >= 2.x until a secure regex solution comes
from Guido and folks ?

I guess most plpython users would be much happier with plpython with
some minor limitations due to older version than with being forced to
use an untrusted pl altogether.

IIRC python 1.5.2 has a perfectly good RExec.

Or is there a requirement that only latest language versions are used in
pg 74 ;)

--------------
Hannu


Re: PlPython

From
Tom Lane
Date:
Hannu Krosing <hannu@tm.ee> writes:
> could we not just make sure that plpython uses python ver < 2.x and use
> plpythonu for python versions >= 2.x until a secure regex solution comes
> from Guido and folks ?

We'd still have to mark it untrusted, so what's the point?

            regards, tom lane

Re: PlPython

From
Hannu Krosing
Date:
Tom Lane kirjutas E, 30.06.2003 kell 00:18:
> Hannu Krosing <hannu@tm.ee> writes:
> > could we not just make sure that plpython uses python ver < 2.x and use
> > plpythonu for python versions >= 2.x until a secure regex solution comes
> > from Guido and folks ?
>
> We'd still have to mark it untrusted, so what's the point?

No we don't! The old version of plpython was perfectly OK when used with
python 1.5.x and will be so. The RExec security holes were only
introduced with new class mechanisms in python 2.x.

The version with patch which removes RExec (as Python 2.x is not
supporting it ) is the right thoing to do FOR PYTHON 2.X, but there is
no reason to remove safe execution when using python 1.5.x.

Thus my proposition for using the old version as plpython and the new
version as plpython-u, but allowing the non-u version to be compuled
only for python v < 2.x.

-----------------
Hannu


Re: PlPython

From
Tom Lane
Date:
Hannu Krosing <hannu@tm.ee> writes:
> The version with patch which removes RExec (as Python 2.x is not
> supporting it ) is the right thoing to do FOR PYTHON 2.X, but there is
> no reason to remove safe execution when using python 1.5.x.

Who's still using 1.5, I guess is the question?  And are they likely
to be updating their PG installation when they're not updating Python?

            regards, tom lane

Re: PlPython

From
Hannu Krosing
Date:
Tom Lane kirjutas E, 30.06.2003 kell 01:21:
> Hannu Krosing <hannu@tm.ee> writes:
> > The version with patch which removes RExec (as Python 2.x is not
> > supporting it ) is the right thoing to do FOR PYTHON 2.X, but there is
> > no reason to remove safe execution when using python 1.5.x.
>
> Who's still using 1.5, I guess is the question?  And are they likely
> to be updating their PG installation when they're not updating Python?

Python is designed such that one can install and use different versions
in parallel - for example the deafult directopries for libraries are
/usr/lib/python1.5/ and /usr/lib/python2.2/ if you have installed python
1.5.x and 2.2.x, also executables are python2, python2.2 and python1.5,
with plain python being a link to one of these.

I guess that anyone who needs safe Restricted Execution will be using
1.5 at least for that purpose until RExec is fixed in 2.x.

------------------
Hannu


Re: PlPython

From
Tom Lane
Date:
Hannu Krosing <hannu@tm.ee> writes:
> Tom Lane kirjutas E, 30.06.2003 kell 01:21:
>> Who's still using 1.5, I guess is the question?  And are they likely
>> to be updating their PG installation when they're not updating Python?

> I guess that anyone who needs safe Restricted Execution will be using
> 1.5 at least for that purpose until RExec is fixed in 2.x.

I don't find that real compelling ...

The bottom line is that this has to get done.  I have the time to
convert plpython to untrusted status tomorrow.  I do not have the time,
the infrastructure, nor the interest to build a conditional setup.
Unless someone else wants to volunteer to make it happen in a timely
fashion, untrusted is what it will be.

            regards, tom lane

Re: PlPython

From
Gerhard Häring
Date:
Hannu Krosing wrote:
> could we not just make sure that plpython uses python ver < 2.x and use
> plpythonu for python versions >= 2.x until a secure regex solution comes
> from Guido and folks ?
> 
> I guess most plpython users would be much happier with plpython with
> some minor limitations due to older version than with being forced to
> use an untrusted pl altogether.

But if rexec isn't safe they're just fooling themselves. There's only 
two kinds of safety for restricted environments: absolute and not. 
That's why the Python developers were honest and disabled rexec for now.

If you want to fool yourself, that's easy enough: ship a modified 
rexec.py with the 'raise RuntimeError, "This code is not secure ..."' 
removed ;-)

> IIRC python 1.5.2 has a perfectly good RExec.

You are likely mistaken. Because I was interested in getting this 
problem solved for plpython and because most rexec problems are because 
of the new-style classes in Python 2.2 and later, I asked on 
comp.lang.python wether it was safe with 2.1 and earlier.

Martin von Löwis told me it probably wasn't in 
http://groups.google.com/groups?selm=m3y8ztib79.fsf%40mira.informatik.hu-berlin.de

> Or is there a requirement that only latest language versions are used in
> pg 74 ;)

No, but I find it hard to believe that PL/python is used by untrusted 
users at all, so making it untrusted might not really be a problem in 
real life.

-- Gerhard

PS: Thanks Kevin for submitting the PL/Python patch. I intended to make 
it available at least as an untrusted language, but you beat me :)


Re: PlPython

From
Tom Lane
Date:
Gerhard Häring <gh@ghaering.de> writes:
> ... because most rexec problems are because 
> of the new-style classes in Python 2.2 and later, I asked on 
> comp.lang.python wether it was safe with 2.1 and earlier.

> Martin von L�wis told me it probably wasn't in 
> http://groups.google.com/groups?selm=m3y8ztib79.fsf%40mira.informatik.hu-berlin.de

Hmm.  I don't think the second bug mentioned there applies to us (if our
use of rexec allows the Python code to write any file anywhere, it's
already lost its right to trusted status IMHO).  But the first one looks
like a killer:
http://sourceforge.net/tracker/?group_id=5470&atid=105470&func=detail&aid=577530
According to that discussion, this bug does exist in Python 1.5 :-(
... so Hannu's argument rests on a false premise anyway.
        regards, tom lane


Re: PlPython

From
Hannu Krosing
Date:
Tom Lane kirjutas E, 30.06.2003 kell 06:39:
> Hannu Krosing <hannu@tm.ee> writes:
> > Tom Lane kirjutas E, 30.06.2003 kell 01:21:
> >> Who's still using 1.5, I guess is the question?  And are they likely
> >> to be updating their PG installation when they're not updating Python?
>
> > I guess that anyone who needs safe Restricted Execution will be using
> > 1.5 at least for that purpose until RExec is fixed in 2.x.
>
> I don't find that real compelling ...
>
> The bottom line is that this has to get done.  I have the time to
> convert plpython to untrusted status tomorrow.  I do not have the time,
> the infrastructure, nor the interest to build a conditional setup.
> Unless someone else wants to volunteer to make it happen in a timely
> fashion, untrusted is what it will be.

Fine with me.

Just don't put in any hacks to pretend it is trusted.

---------------
Hannu



Re: PlPython

From
Hannu Krosing
Date:
Gerhard Häring kirjutas E, 30.06.2003 kell 07:39:
> Hannu Krosing wrote:

> > IIRC python 1.5.2 has a perfectly good RExec.
>
> You are likely mistaken. Because I was interested in getting this
> problem solved for plpython and because most rexec problems are because
> of the new-style classes in Python 2.2 and later, I asked on
> comp.lang.python wether it was safe with 2.1 and earlier.
>
> Martin von Löwis told me it probably wasn't in
> http://groups.google.com/groups?selm=m3y8ztib79.fsf%40mira.informatik.hu-berlin.de

Uups! I got that impression from the emails which claimed that new-style
classes make it near impossilble to fix.

> > Or is there a requirement that only latest language versions are used in
> > pg 74 ;)
>
> No, but I find it hard to believe that PL/python is used by untrusted
> users at all, so making it untrusted might not really be a problem in
> real life.

That is most likely true.

And trusted users like to use untrusted languages, so they can send
email, get xml via http and write files ;)

-------------
Hannu



Re: PlPython

From
Tilo Schwarz
Date:
Hannu Krosing writes:
> Tom Lane kirjutas E, 23.06.2003 kell 01:29:
> > Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:
> > > Attached is a patch that removes all of the RExec code from plpython
> > > from the current PostgreSQL CVS.  In addition, plpython needs to be
> > > changed to an untrusted language in createlang.
> >
> > I am inclined to rename plpython to plpythonu, by analogy to pltclu.
>
> ...
>
> > Comments?
>
> could we not just make sure that plpython uses python ver < 2.x and use
> plpythonu for python versions >= 2.x until a secure regex solution comes
> from Guido and folks ?
>
> I guess most plpython users would be much happier with plpython with
> some minor limitations due to older version than with being forced to
> use an untrusted pl altogether.

Actually, there are also people (me, for example :-), who would love to see an
untrusted PlPython to use the full Python power as Pl. So I'd be very happy
with a solution, where I could choose between plpython and plpythonu. But I
don't know if that means having twice the effort compared to only having
plpython _or_ plpythonu.

Regards,
Tilo




Re: [GENERAL] PlPython

From
elein
Date:
For 7.4 (which I expect is the patch's target) it might be
best to make both names point to the same thing with a
clear release note that says that they are the same thing
and that plpython[u] is now untrusted.

That will give people a bit a time to reload their
existing functions.

elein

On Sunday 22 June 2003 15:29, Tom Lane wrote:
> Kevin Jacobs <jacobs@penguin.theopalgroup.com> writes:
> > Attached is a patch that removes all of the RExec code from plpython from
> > the current PostgreSQL CVS.  In addition, plpython needs to be changed to an
> > untrusted language in createlang.
>
> I am inclined to rename plpython to plpythonu, by analogy to pltclu.
> The advantage of doing so is that (a) the name change makes it somewhat
> more obvious that there's a fundamental behavioral change, and (b)
> assuming that the Python folk someday figure out a secure version of
> RExec, we'd want to reinstitute the trusted version of plpython, but
> perhaps not take away the untrusted one.
>
> On the other hand, this would create headaches for people who are trying
> to load dump files that declare plpython or contain plpython-language
> functions.  I can't think of any non-kluge solution to this (kluge
> solutions would include putting special-case code into CREATE FUNCTION
> to change 'plpython' to 'plpythonu' ...)
>
> Comments?
>
>             regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: the planner will ignore your desire to choose an index scan if your
>       joining column's datatypes do not match
>
>

--
=============================================================
elein@varlena.com     Database Consulting     www.varlena.com
PostgreSQL General Bits    http:/www.varlena.com/GeneralBits/
   "Free your mind the rest will follow" -- en vogue


Re: [GENERAL] PlPython

From
elein
Date:
I thought there would be a relatively clear way
to alias them both to the same language library for
a release or two.  But I see your point on transitioning.
Clear notice is really important.

plpython should be phased out if it is not replaced
within a release or two.

If only the change could be transparent to those
people who are not using any untrusted features.

Maybe it is better to break everyone's plpython functions
in 7.4.  That would certainly make sure everyone heard
of the change from trusted to untrusted.  But explanations
of what exactly that means should also be included in the
release notes.

elein

PS: I've built and tested the plpython patch against
7.3.2 and am happy it does not affect the features I count
on.  I'd do it against the development release but
I can't get it to build (for other reasons).

On Monday 23 June 2003 13:53, Tom Lane wrote:
> elein  <elein@varlena.com> writes:
> > For 7.4 (which I expect is the patch's target) it might be
> > best to make both names point to the same thing with a
> > clear release note that says that they are the same thing
> > and that plpython[u] is now untrusted.
>
> I don't know any way to actually do that, though.  If we put two entries
> in pg_language then functions created in plpython will stay associated
> with that entry.  That'd probably be the worst of all possible worlds,
> since a person looking at pg_language would quite reasonably assume that
> plpython was still trusted and the untrusted plpythonu was just an
> addition.  (Especially if he happened to know that such an addition was
> planned long ago.)  You could shoot yourself in the foot pretty badly
> with such a misunderstanding :-(
>
> The behavior that I think would be most useful would be to automatically
> transpose CREATE FUNCTION ... LANGUAGE "plpython" into CREATE FUNCTION
> ... LANGUAGE "plpythonu".  Which we could do with an ugly hack in CREATE
> FUNCTION (ugly, but no worse than things we've done to index opclass
> names, for example).  But it could be too confusing.
>
>             regards, tom lane
>
>

--
=============================================================
elein@varlena.com     Database Consulting     www.varlena.com
PostgreSQL General Bits    http:/www.varlena.com/GeneralBits/
   "Free your mind the rest will follow" -- en vogue


Re: PlPython

From
Kevin Jacobs
Date:
On 30 Jun 2003, Hannu Krosing wrote:
> Tom Lane kirjutas E, 30.06.2003 kell 01:21:
> > Hannu Krosing <hannu@tm.ee> writes:
> > > The version with patch which removes RExec (as Python 2.x is not
> > > supporting it ) is the right thoing to do FOR PYTHON 2.X, but there is
> > > no reason to remove safe execution when using python 1.5.x.
> >
> > Who's still using 1.5, I guess is the question?  And are they likely
> > to be updating their PG installation when they're not updating Python?
>
> Python is designed such that one can install and use different versions
> in parallel - for example the deafult directopries for libraries are
> /usr/lib/python1.5/ and /usr/lib/python2.2/ if you have installed python
> 1.5.x and 2.2.x, also executables are python2, python2.2 and python1.5,
> with plain python being a link to one of these.
>
> I guess that anyone who needs safe Restricted Execution will be using
> 1.5 at least for that purpose until RExec is fixed in 2.x.

Unless a someone steps forward to actively maintain such a configuration,
then trusted PlPython is not going to exist anymore.  The cost of getting
even the most minute detail wrong is just too high tobe left to chance.

-Kevin

--
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com


Re: [GENERAL] PlPython

From
Hannu Krosing
Date:
elein kirjutas T, 24.06.2003 kell 00:42:

There is a realtively clean hack one can use to convert plpython
functions to plpythonu manually - just rename the language for the time
of loading functions - do as superuser

update pg_language set lanname = 'plpython' where lanname = 'plpythonu';

LOAD YOUR Pl/Python FUNCTIONS ;

update pg_language set lanname = 'plpythonu' where lanname = 'plpython';

> PS: I've built and tested the plpython patch against
> 7.3.2 and am happy it does not affect the features I count
> on.

As it should.

The untrusted language gives you *more* power, not less.

The untrusted status means that the user has to be trusted to use that
much power.

----------------
Hannu