Thread: Updates of SE-PostgreSQL 8.4devel patches (r1197)

Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

07 November 2008, 08:22:30

I updated the patch set of SE-PostgreSQL (revision 1197).

[1/6] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1197.patch
[2/6] http://sepgsql.googlecode.com/files/sepostgresql-pg_dump-8.4devel-3-r1197.patch
[3/6] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1197.patch
[4/6] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1197.patch
[5/6] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1197.patch
[6/6] http://sepgsql.googlecode.com/files/sepostgresql-row_acl-8.4devel-3-r1197.patch

List of updates:
- Patches are rebased to the latest CVS HEAD.
- The "NSA SELinux" comments are replaced by the simple "SELinux".
- bugfix: system attributes are preserved prior to invocation of before-row-update-triggers.
- Add a documentation of row-level database acl, within 6th patch. Most of patch size increments come from the new
documentation.

Draft of the SE-PostgreSQL documentation is here: http://wiki.postgresql.org/wiki/SEPostgreSQL

By the way, now a week has gone since beginning of the final CommitFest,
however, the reviewers field on the wiki is empty.
Is there anyone who can help to review the patches?

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

07 November 2008, 14:31:39

On Fri, 2008-11-07 at 18:20 +0900, KaiGai Kohei wrote:
> I updated the patch set of SE-PostgreSQL (revision 1197).
> 
> [1/6] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1197.patch
> [2/6] http://sepgsql.googlecode.com/files/sepostgresql-pg_dump-8.4devel-3-r1197.patch
> [3/6] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1197.patch
> [4/6] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1197.patch
> [5/6] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1197.patch
> [6/6] http://sepgsql.googlecode.com/files/sepostgresql-row_acl-8.4devel-3-r1197.patch
> 
> List of updates:
> - Patches are rebased to the latest CVS HEAD.
> - The "NSA SELinux" comments are replaced by the simple "SELinux".
> - bugfix: system attributes are preserved prior to invocation of
>   before-row-update-triggers.
> - Add a documentation of row-level database acl, within 6th patch.
>   Most of patch size increments come from the new documentation.
> 
> Draft of the SE-PostgreSQL documentation is here:
>   http://wiki.postgresql.org/wiki/SEPostgreSQL
> 
> By the way, now a week has gone since beginning of the final CommitFest,
> however, the reviewers field on the wiki is empty.
> Is there anyone who can help to review the patches?

Some initial thoughts based upon reading the Wiki. I've not been
involved in things up to now, so if this dredges up old discussions,
well, these are my thoughts.

I want SEPostgreSQL, but I'd like it to work without needing to be a
compile time option so people can just use it as/when needed. Plus we
don't want to support what would be/is essentially a fork of Postgres.
Most CPUs will optimise away a simple if-test in various places.

Some users will be able to take advantage of the facilities without
implementing full MLS. Yet we want the full facilities for Government.
Many people currently run multiple customers in different schemas,
though this would let them just run a single set of tables so is much
better for running many small customers.

I'm very unhappy about putting a nonoptional value on tuple headers,
especially because it is updatable. Do we expect MVCC will work with
updatable security contexts? i.e. when you update the security context
of a tuple that won't effect its visibility to existing system users. I
can't imagine you'd want that would you? It's kind of difficult to *not*
get it though.

Looks to me that this feature is useless without things working at row
level. So we can't leave that part out.

The security context on each row could be an optional column present
only if HEAP_HASSECURITYCONTEXT is set (0x0010 see htup.h), just like
OIDs. Use a specific datatype rather than TEXT. That datatype could be
an identifier to pg_security. Security people have big databases too, so
we need to compress the security context more and take out parse time of
string handling. Don't think we should use Oids, they're too big. Might
be easier to use a 2byte field and restrict access to 32,000 contexts,
which is easily enough. TEXT also makes me nervous, just in case there
is some collation/encoding weirdness that allows contexts to be
subverted. Fixed integers are hard to compromise in that respect.

How will unique indexes work? Do you implicitly add security context as
last column on every unique index, or does the uniqueness violation only
occurs within security contexts, or does the uniqueness violation tested
against all contextx that the inserter can currently see? Is there a
change to system catalogs? 

Foreign Key deletions could be handled correctly if you treat them as
updates. If we have the following example

TableA
security_context=y value=2 fk=1

TableB
security_context=x value=1

TableA refers to TableB. Context x cannot see context y.

So if somebody with context x tries to delete value1 from TableB, they
will be refused because of a row they cannot see. In this case the
correct action is to update the tuple in TableB so it now has a
security_context = y. The user with x cannot see it and can be persuaded
he deleted it, while the user with y can still see it.

The section on LOAD doesn't sound very convincing. Loading a module
could immediately subvert security. We could probably tighten up on that
for general use as well. ISTM we need something like a new catalog table
for loadable modules, so we can give them specific security contexts and
potentially store some kind of verification information about them.
Having a single "can load modules" isn't enough with Postgres, since it
would effectively prevent us from loading *any* modules in a secure
database. Which kinda removes much of the benefit of using Postgres.

Is there an issue with relation cache and catalog caches? ISTM that you
should put a security context onto each shared invalidation message, so
that backends know not to maintain caches for data they aren't allowed
to see. Probably overkill, just thinking.

The interaction with SELinux should not be hardcoded. I think we need
some form of plugin/wrapper to allow other systems to work. Not sure
what those are, but this isn't a Linux only project. We want to give
everybody the ability to work with PostgreSQL.

How does Discretionary Access control work with regard to server logs?
You said there was no superuser access, but I don't see any controls for
the logs. Do we need log_max_security_context?

Trusted procedures seem very similar to SECURITY DEFINER functions. Can
you explain what the differences are? I'm sure we don't want to similar
features.

I don't see any reason for the "=" in most of the new DDL syntax. Just
seems superfluous and out of character with normal SQL DDL. 

With DDL we already have table options, so why not include
security_context as an option? If we are adding this to databases as
well, it seems a good time to include a generic database-level options
facility and make "security_context" the first option.

The parameter to enable this facility should be something likeenhanced_security = on

My feeling is that if you want to include these features in core
PostgreSQL you won't be able to maintain the "separate branding" of
SEPostgreSQL, logo etc.. Maybe I'll get used to it. But if we are going
to use it, we should say SEPostgresSQL, not SE-PostgreSQL if we are
saying SELinux.

It would make sense for you to look at the work on replication also. Not
much use having SEPostgreSQL if we can't replicate it securely. And
possibly in-place update so that respects label security.

I've not looked at the code and probably won't have time to do that. But
if I did, I'd say "diff -c". 

Good, thorough work.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Bruce Momjian

Date:

07 November 2008, 17:20:17

Simon Riggs wrote:
> Some initial thoughts based upon reading the Wiki. I've not been
> involved in things up to now, so if this dredges up old discussions,
> well, these are my thoughts.
> 
> I want SEPostgreSQL, but I'd like it to work without needing to be a
> compile time option so people can just use it as/when needed. Plus we
> don't want to support what would be/is essentially a fork of Postgres.
> Most CPUs will optimise away a simple if-test in various places.

Well, I assume SEPostgreSQL will rely on libraries that only exist on
SELinux, so I am afraid there will have to be some type of compile-time
flag, though the goal is to have as much of it enabled as possible at
the SQL level, as we have discussed and the patch reflects.

> Some users will be able to take advantage of the facilities without
> implementing full MLS. Yet we want the full facilities for Government.
> Many people currently run multiple customers in different schemas,
> though this would let them just run a single set of tables so is much
> better for running many small customers.
> 
> I'm very unhappy about putting a nonoptional value on tuple headers,
> especially because it is updatable. Do we expect MVCC will work with
> updatable security contexts? i.e. when you update the security context
> of a tuple that won't effect its visibility to existing system users. I
> can't imagine you'd want that would you? It's kind of difficult to *not*
> get it though.
> 
> Looks to me that this feature is useless without things working at row
> level. So we can't leave that part out.

Having the security column be optional is an interesting idea, perhaps
like we do with oids and with a GUC variable controlling the default. 

I think the patch itself is going to go through many adjustments before
application, I bet.

> The security context on each row could be an optional column present
> only if HEAP_HASSECURITYCONTEXT is set (0x0010 see htup.h), just like
> OIDs. Use a specific datatype rather than TEXT. That datatype could be
> an identifier to pg_security. Security people have big databases too, so
> we need to compress the security context more and take out parse time of
> string handling. Don't think we should use Oids, they're too big. Might
> be easier to use a 2byte field and restrict access to 32,000 contexts,
> which is easily enough. TEXT also makes me nervous, just in case there
> is some collation/encoding weirdness that allows contexts to be
> subverted. Fixed integers are hard to compromise in that respect.

I think the security mechanism is more complex than just assigning a
single security identifier, but perhaps not; I am unsure.

> How will unique indexes work? Do you implicitly add security context as
> last column on every unique index, or does the uniqueness violation only
> occurs within security contexts, or does the uniqueness violation tested
> against all contextx that the inserter can currently see? Is there a
> change to system catalogs? 
> 
> Foreign Key deletions could be handled correctly if you treat them as
> updates. If we have the following example
> 
> TableA
> security_context=y value=2 fk=1
> 
> TableB
> security_context=x value=1
> 
> TableA refers to TableB. Context x cannot see context y.

I think this is the "covert channel" we have been discussing.

> The interaction with SELinux should not be hardcoded. I think we need
> some form of plugin/wrapper to allow other systems to work. Not sure
> what those are, but this isn't a Linux only project. We want to give
> everybody the ability to work with PostgreSQL.

I think that is why the text security column was used, so other systems
could put their own security text in there.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

"Robert Haas"

Date:

07 November 2008, 19:12:54

> Foreign Key deletions could be handled correctly if you treat them as
> updates. If we have the following example
>
> TableA
> security_context=y value=2 fk=1
>
> TableB
> security_context=x value=1
>
> TableA refers to TableB. Context x cannot see context y.
>
> So if somebody with context x tries to delete value1 from TableB, they
> will be refused because of a row they cannot see. In this case the
> correct action is to update the tuple in TableB so it now has a
> security_context = y. The user with x cannot see it and can be persuaded
> he deleted it, while the user with y can still see it.

It seems odd for a low-privilege user to be able to elevate the
privilege of a tuple above their own privilege level.  I also don't
believe that the privilege level is a total order, which might make
this something of a sticky wicket.  But those are just my thoughts as
a non-guru.

...Robert

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

07 November 2008, 20:06:22

On Fri, 2008-11-07 at 13:19 -0500, Bruce Momjian wrote:
> The security context on each row could be an optional column present
> > only if HEAP_HASSECURITYCONTEXT is set (0x0010 see htup.h), just
> like
> > OIDs. Use a specific datatype rather than TEXT. That datatype could
> be
> > an identifier to pg_security. Security people have big databases
> too, so
> > we need to compress the security context more and take out parse
> time of
> > string handling. Don't think we should use Oids, they're too big.
> Might
> > be easier to use a 2byte field and restrict access to 32,000
> contexts,
> > which is easily enough. TEXT also makes me nervous, just in case
> there
> > is some collation/encoding weirdness that allows contexts to be
> > subverted. Fixed integers are hard to compromise in that respect.
> 
> I think the security mechanism is more complex than just assigning a
> single security identifier, but perhaps not; I am unsure.

Maybe. We already handle such complexity for comboids and multixacts, so
I suggest we do the same thing here.

Any system with more than 32,000 security contexts is going to be
unmanageable and probably therefore insecure...

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

07 November 2008, 20:10:43

On Fri, 2008-11-07 at 15:12 -0500, Robert Haas wrote:
> > Foreign Key deletions could be handled correctly if you treat them as
> > updates. If we have the following example
> >
> > TableA
> > security_context=y value=2 fk=1
> >
> > TableB
> > security_context=x value=1
> >
> > TableA refers to TableB. Context x cannot see context y.
> >
> > So if somebody with context x tries to delete value1 from TableB, they
> > will be refused because of a row they cannot see. In this case the
> > correct action is to update the tuple in TableB so it now has a
> > security_context = y. The user with x cannot see it and can be persuaded
> > he deleted it, while the user with y can still see it.
> 
> It seems odd for a low-privilege user to be able to elevate the
> privilege of a tuple above their own privilege level.  I also don't
> believe that the privilege level is a total order, which might make
> this something of a sticky wicket.  But those are just my thoughts as
> a non-guru.

The low-privilege user isn't elevating the label. If the tuple was
visible by multiple labels it was already elevated. All I am suggesting
is the system remove the one it can see, leaving the other ones intact.
This makes the row appear to be deleted by the lower privileged user,
whereas in fact it was merely updated. There need not be any ordering to
the labels for this scheme to work.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

"Robert Haas"

Date:

07 November 2008, 20:38:48

> The low-privilege user isn't elevating the label. If the tuple was
> visible by multiple labels it was already elevated. All I am suggesting
> is the system remove the one it can see, leaving the other ones intact.
> This makes the row appear to be deleted by the lower privileged user,
> whereas in fact it was merely updated. There need not be any ordering to
> the labels for this scheme to work.

I see.  That seems like it makes sense, but what about the update case?

...Robert

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Martijn van Oosterhout

Date:

07 November 2008, 20:51:59

On Fri, Nov 07, 2008 at 01:50:18PM +0000, Simon Riggs wrote:
> How will unique indexes work? Do you implicitly add security context as
> last column on every unique index, or does the uniqueness violation only
> occurs within security contexts, or does the uniqueness violation tested
> against all contextx that the inserter can currently see? Is there a
> change to system catalogs?

The wiki clearly states that the unique test is prior to any filtering.
Anything else seems crazy to me.

http://wiki.postgresql.org/wiki/SEPostgreSQL#Unique_constraint

> Foreign Key deletions could be handled correctly if you treat them as
> updates. If we have the following example

Why? If a client does a delete and the database says OK, the tuple
should be gone, *for everyone*.

http://wiki.postgresql.org/wiki/SEPostgreSQL#Foreign_Key_constraint

It is the responsibility of the DB administrator to worry about covert
channels.

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> Please line up in a tree and maintain the heap invariant while
> boarding. Thank you for flying nlogn airlines.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Tom Lane

Date:

07 November 2008, 20:52:25

Simon Riggs <simon@2ndQuadrant.com> writes:
> Any system with more than 32,000 security contexts is going to be
> unmanageable and probably therefore insecure...

Perhaps, but it's doubtful that such a restriction will actually save
any space after you consider the alignment behavior.  I'd go with an
Oid.
        regards, tom lane

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Bruce Momjian

Date:

07 November 2008, 20:53:15

Simon Riggs wrote:
> > > So if somebody with context x tries to delete value1 from TableB, they
> > > will be refused because of a row they cannot see. In this case the
> > > correct action is to update the tuple in TableB so it now has a
> > > security_context = y. The user with x cannot see it and can be persuaded
> > > he deleted it, while the user with y can still see it.
> > 
> > It seems odd for a low-privilege user to be able to elevate the
> > privilege of a tuple above their own privilege level.  I also don't
> > believe that the privilege level is a total order, which might make
> > this something of a sticky wicket.  But those are just my thoughts as
> > a non-guru.
> 
> The low-privilege user isn't elevating the label. If the tuple was
> visible by multiple labels it was already elevated. All I am suggesting
> is the system remove the one it can see, leaving the other ones intact.
> This makes the row appear to be deleted by the lower privileged user,
> whereas in fact it was merely updated. There need not be any ordering to
> the labels for this scheme to work.

Simon, would you read the chapter on "covert channels"?  You might
understand it better than I do and it might give you some ideas:
   http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.33.5950

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

08 November 2008, 07:21:17

Simon, Thanks for your comments.

> Some initial thoughts based upon reading the Wiki. I've not been
> involved in things up to now, so if this dredges up old discussions,
> well, these are my thoughts.
> 
> I want SEPostgreSQL, but I'd like it to work without needing to be a
> compile time option so people can just use it as/when needed. Plus we
> don't want to support what would be/is essentially a fork of Postgres.
> Most CPUs will optimise away a simple if-test in various places.

As Bruce also mentioned, it has to be linked with libselinux to communicate
in-kernel SELinux, but it is not onw of the universal libraries, so, it
has to be enabled/disabled on build-time option.

In addition, we can also disable the feature by the following configuration. sepostgresql = disabled (at
$PGDATA/postgresql.conf)
 

TODO: add a description about the guc variable. It can have four state:     "default", "enforcing", "permissive" and
"disabled".

> Some users will be able to take advantage of the facilities without
> implementing full MLS. Yet we want the full facilities for Government.
> Many people currently run multiple customers in different schemas,
> though this would let them just run a single set of tables so is much
> better for running many small customers.

SELinux community also provides a MLS enabled policy, but it is not
a default one now. SE-PostgreSQL has all the its access controls
decision externally, so it is out of our coltrols.

> I'm very unhappy about putting a nonoptional value on tuple headers,
> especially because it is updatable. Do we expect MVCC will work with
> updatable security contexts? i.e. when you update the security context
> of a tuple that won't effect its visibility to existing system users. I
> can't imagine you'd want that would you? It's kind of difficult to *not*
> get it though.

When a user updates the security context of some tuples, it is invisible
for other clients until its commit, like as other normal data.
Sorry, it is unclear for me what is the concern you mention.

> Looks to me that this feature is useless without things working at row
> level. So we can't leave that part out.

I guess you are saying the core PostgreSQL also has table and column
level granularities, but its criteria to make a decision is different.
SE-PostgreSQL makes its decision based on the privileges of peer process,
not a database role.

> The security context on each row could be an optional column present
> only if HEAP_HASSECURITYCONTEXT is set (0x0010 see htup.h), just like
> OIDs. Use a specific datatype rather than TEXT. That datatype could be
> an identifier to pg_security. Security people have big databases too, so
> we need to compress the security context more and take out parse time of
> string handling. Don't think we should use Oids, they're too big. Might
> be easier to use a 2byte field and restrict access to 32,000 contexts,
> which is easily enough. TEXT also makes me nervous, just in case there
> is some collation/encoding weirdness that allows contexts to be
> subverted. Fixed integers are hard to compromise in that respect.

An issue is who can decide the existance or needs of security system
attribute. If the table owner can disable it, we cannot say it as
a mandatory access control feature, so the security attribute has to
be always appeared when the security feature is enabled.

Here is an answer for the expected question.
When we refer the "security_context" system column of tuples without
HEAP_HAS_SECURITY, it returns an alternative label called as "unlabeled_t",
because it has no labels.

The reason why we adopt TEXT type is SELinux requires userspace
object manager makes queries via text represented security context,
and it also can be used for other security feature to show client
its security attribute in human readable format.

For your information, in-kernel SELinux can handle 2^32 - 1 of security
context internally in theoretical maximum, so using Oid as a security
identifier is a fair decision to avoid an implementation to handle overflow
cases.

> How will unique indexes work? Do you implicitly add security context as
> last column on every unique index, or does the uniqueness violation only
> occurs within security contexts, or does the uniqueness violation tested
> against all contextx that the inserter can currently see? Is there a
> change to system catalogs?

The unique/primary key constraint works at the lowest level.
So, it has a possibility that invisible tuple prevent to insert a tuple.

> Foreign Key deletions could be handled correctly if you treat them as
> updates. If we have the following example
> 
> TableA
> security_context=y value=2 fk=1
> 
> TableB
> security_context=x value=1
> 
> TableA refers to TableB. Context x cannot see context y.
> 
> So if somebody with context x tries to delete value1 from TableB, they
> will be refused because of a row they cannot see. In this case the
> correct action is to update the tuple in TableB so it now has a
> security_context = y. The user with x cannot see it and can be persuaded
> he deleted it, while the user with y can still see it.

It is also discussed before.
In this case, SE-PostgreSQL prevent to delete a tuple on the TableB
to keep referential integrity. So, it enables unprivileged users to
infer the existance of refering tuples, but it is a limitation called
as "covert channel".

Your example is the simplest case, so it seems to work well, but
most of cases are not obvious. If the TE policy prevent accesses
on tuples with security_context=x, we have no obvious way to decide
what is a proper security context to be updated.

> The section on LOAD doesn't sound very convincing. Loading a module
> could immediately subvert security. We could probably tighten up on that
> for general use as well. ISTM we need something like a new catalog table
> for loadable modules, so we can give them specific security contexts and
> potentially store some kind of verification information about them.
> Having a single "can load modules" isn't enough with Postgres, since it
> would effectively prevent us from loading *any* modules in a secure
> database. Which kinda removes much of the benefit of using Postgres.

SE-PostgreSQL applies access controls for individual loadable modules.
When a function implemented within an external modules tries to be loaded,
it checks security context between the database and the file of loadable
modules. (No need to say, in-kernel SELinux assigns its security context
for files in common format, so we can compare them each other.)

Perhaps, the description I wrote can easily make misunderstanding.
If you can read it says widespread load modules permission, I'll revise
its representation.
 http://wiki.postgresql.org/wiki/SEPostgreSQL#Loading_shared_library_module

> Is there an issue with relation cache and catalog caches? ISTM that you
> should put a security context onto each shared invalidation message, so
> that backends know not to maintain caches for data they aren't allowed
> to see. Probably overkill, just thinking.

Sorry, I cannot understand what you concerned.
The pg_security system catalog is only modified when a newly appeared
text representation of security attribute is given. So, it is read only
for most of cases.

> The interaction with SELinux should not be hardcoded. I think we need
> some form of plugin/wrapper to allow other systems to work. Not sure
> what those are, but this isn't a Linux only project. We want to give
> everybody the ability to work with PostgreSQL.

The PGACE security framework already enables what you want.
The "rowacl" module is a proof of concept, and we can add a new mechanism
on the framework, if necessary.
Do you know the LSM (Linux Security Module) system?

> How does Discretionary Access control work with regard to server logs?
> You said there was no superuser access, but I don't see any controls for
> the logs. Do we need log_max_security_context?

It writes out server logs to filesystem, so its access controls should
be applied in-kernel SELinux. SE-PostgreSQL does not care system call
invocation, because it should be a task in kernel features.
It works as a reference monitor for SQL queries.

> Trusted procedures seem very similar to SECURITY DEFINER functions. Can
> you explain what the differences are? I'm sure we don't want to similar
> features.

It does not change the working security context of the client.

Please consider whether a set-uid program on operating system makes
unnecessary domain transition of SELinux, or not. They have individual
purposes and roles, so we need both of them.

> I don't see any reason for the "=" in most of the new DDL syntax. Just
> seems superfluous and out of character with normal SQL DDL. 

Because I thought the syntax is most friendliness for end-users
two years ago. It looks obvious the option specifies the security
context of resources.
Sorry, it is not a clear reason, but it easily makes us to understand
the option.

> With DDL we already have table options, so why not include
> security_context as an option? If we are adding this to databases as
> well, it seems a good time to include a generic database-level options
> facility and make "security_context" the first option.

How do we implement the "security_context" option for columns?
When we want to define a table with explicitly labeled column,
I don't think the table option is a user friendly interface.

> The parameter to enable this facility should be something like
>     enhanced_security = on

As I noted above, it already provides: sepostgresql = [ default | enforcing | permissive | disabled ]

> My feeling is that if you want to include these features in core
> PostgreSQL you won't be able to maintain the "separate branding" of
> SEPostgreSQL, logo etc.. Maybe I'll get used to it. But if we are going
> to use it, we should say SEPostgresSQL, not SE-PostgreSQL if we are
> saying SELinux.

Please fee free to call it as SE-PostgreSQL, SEPostgreSQL, sepgsql
and others. I don't think it is a significant matter.
However, in my hope, I don't want to discard the mascot and logo,
because it was a present from my friend who works as an illustrator.

In Japan, a turtle is used as a mascot of PostgreSQL. :-) http://www.postgresql.jp/npo/logo/

> It would make sense for you to look at the work on replication also. Not
> much use having SEPostgreSQL if we can't replicate it securely. And
> possibly in-place update so that respects label security.

The log-shipping replication mechanism will works correctly,
but I have not evaluated it yet. In this case, the master
and slaves should have same security policy for databases.

> I've not looked at the code and probably won't have time to do that. But
> if I did, I'd say "diff -c". 

OK, I'll fix the scripts to generate patch set.
The next series will provided with "diff -c" style.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

08 November 2008, 07:21:28

Simon Riggs wrote:
> On Fri, 2008-11-07 at 15:12 -0500, Robert Haas wrote:
>>> Foreign Key deletions could be handled correctly if you treat them as
>>> updates. If we have the following example
>>>
>>> TableA
>>> security_context=y value=2 fk=1
>>>
>>> TableB
>>> security_context=x value=1
>>>
>>> TableA refers to TableB. Context x cannot see context y.
>>>
>>> So if somebody with context x tries to delete value1 from TableB, they
>>> will be refused because of a row they cannot see. In this case the
>>> correct action is to update the tuple in TableB so it now has a
>>> security_context = y. The user with x cannot see it and can be persuaded
>>> he deleted it, while the user with y can still see it.
>> It seems odd for a low-privilege user to be able to elevate the
>> privilege of a tuple above their own privilege level.  I also don't
>> believe that the privilege level is a total order, which might make
>> this something of a sticky wicket.  But those are just my thoughts as
>> a non-guru.
> 
> The low-privilege user isn't elevating the label. If the tuple was
> visible by multiple labels it was already elevated. All I am suggesting
> is the system remove the one it can see, leaving the other ones intact.
> This makes the row appear to be deleted by the lower privileged user,
> whereas in fact it was merely updated. There need not be any ordering to
> the labels for this scheme to work.

SELinux does not allow a object has two or more labels.

You implicitly assume the security mechanism allows users to access
when one of the labels allows it, but there is no consensus.

At the past, an idea of multiple labels is discussed in SELinux community,
but it was finally dropped because we cannot define the behavior when the
security policy makes different decision for two different labels.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

08 November 2008, 08:23:03

On Fri, 2008-11-07 at 16:52 -0500, Bruce Momjian wrote:
> Simon Riggs wrote:
> > > > So if somebody with context x tries to delete value1 from TableB, they
> > > > will be refused because of a row they cannot see. In this case the
> > > > correct action is to update the tuple in TableB so it now has a
> > > > security_context = y. The user with x cannot see it and can be persuaded
> > > > he deleted it, while the user with y can still see it.
> > > 
> > > It seems odd for a low-privilege user to be able to elevate the
> > > privilege of a tuple above their own privilege level.  I also don't
> > > believe that the privilege level is a total order, which might make
> > > this something of a sticky wicket.  But those are just my thoughts as
> > > a non-guru.
> > 
> > The low-privilege user isn't elevating the label. If the tuple was
> > visible by multiple labels it was already elevated. All I am suggesting
> > is the system remove the one it can see, leaving the other ones intact.
> > This makes the row appear to be deleted by the lower privileged user,
> > whereas in fact it was merely updated. There need not be any ordering to
> > the labels for this scheme to work.
> 
> Simon, would you read the chapter on "covert channels"?  You might
> understand it better than I do and it might give you some ideas:
> 
>        http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.33.5950

It's probably easier just to say "here is the specification we;re
working to implement".

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

08 November 2008, 08:47:32

On Sat, 2008-11-08 at 14:21 +0900, KaiGai Kohei wrote:

> > Some users will be able to take advantage of the facilities without
> > implementing full MLS. Yet we want the full facilities for Government.
> > Many people currently run multiple customers in different schemas,
> > though this would let them just run a single set of tables so is much
> > better for running many small customers.
> 
> SELinux community also provides a MLS enabled policy, but it is not
> a default one now. SE-PostgreSQL has all the its access controls
> decision externally, so it is out of our coltrols.

I have a couple of concerns: 

1. if we make a direct link with SELinux then the code will be *much*
less used and tested. It will be a constant battle to maintain
SEPostgres in a bug-free state. I want to decouple the link so this code
goes into normal Postgres. Other comments below are made with that
thought in mind.

2. I see another use case here. For example, a customer runs a
Software-as-a-service company where the same service is offered to 500
customers. The same database schema is there for each customer, yet they
must never see each other's data. Today, that is implemented as 500
schemas, plus 1 schema with common data in it. ISTM we would be able to
implement this better using SEPostgreSQL, with/without using SELinux.
The need for common data is why in some cases we would want access
controls placed only on certain tables.

Fulfilling use case (2) will also meet my concerns in (1).

So I would prefer it if the solution was designed closely with SELinux,
but usable and useful in other cases also. The link to SELinux could be
done via a contrib plugin. A plugin is then optional. But the main
plugin we provide can be built directly with SELinux.

> > The security context on each row could be an optional column present
> > only if HEAP_HASSECURITYCONTEXT is set (0x0010 see htup.h), just like
> > OIDs. Use a specific datatype rather than TEXT. That datatype could be
> > an identifier to pg_security. Security people have big databases too, so
> > we need to compress the security context more and take out parse time of
> > string handling. Don't think we should use Oids, they're too big. Might
> > be easier to use a 2byte field and restrict access to 32,000 contexts,
> > which is easily enough. TEXT also makes me nervous, just in case there
> > is some collation/encoding weirdness that allows contexts to be
> > subverted. Fixed integers are hard to compromise in that respect.
> 
> An issue is who can decide the existance or needs of security system
> attribute. If the table owner can disable it, we cannot say it as
> a mandatory access control feature, so the security attribute has to
> be always appeared when the security feature is enabled.
> 
> Here is an answer for the expected question.
> When we refer the "security_context" system column of tuples without
> HEAP_HAS_SECURITY, it returns an alternative label called as "unlabeled_t",
> because it has no labels.

Not really an issue. Just use a parameter. security_controls = mandatory
or change the meaning of the existing one. Similar to default_with_oids,
just that we have a setting where it isn't optional.

> The reason why we adopt TEXT type is SELinux requires userspace
> object manager makes queries via text represented security context,
> and it also can be used for other security feature to show client
> its security attribute in human readable format.

AFAICS, neither of those things means the datatype has to be TEXT.

> For your information, in-kernel SELinux can handle 2^32 - 1 of security
> context internally in theoretical maximum, so using Oid as a security
> identifier is a fair decision to avoid an implementation to handle overflow
> cases.

OK, so we need 4 bytes. I can live with that - at least its not ~10
bytes/row. Does it need to be an Oid, or just the size of an Oid?

> > The section on LOAD doesn't sound very convincing. Loading a module
> > could immediately subvert security. We could probably tighten up on that
> > for general use as well. ISTM we need something like a new catalog table
> > for loadable modules, so we can give them specific security contexts and
> > potentially store some kind of verification information about them.
> > Having a single "can load modules" isn't enough with Postgres, since it
> > would effectively prevent us from loading *any* modules in a secure
> > database. Which kinda removes much of the benefit of using Postgres.
> 
> SE-PostgreSQL applies access controls for individual loadable modules.
> When a function implemented within an external modules tries to be loaded,
> it checks security context between the database and the file of loadable
> modules. (No need to say, in-kernel SELinux assigns its security context
> for files in common format, so we can compare them each other.)
> 
> Perhaps, the description I wrote can easily make misunderstanding.
> If you can read it says widespread load modules permission, I'll revise
> its representation.
> 
>   http://wiki.postgresql.org/wiki/SEPostgreSQL#Loading_shared_library_module
> 

> > How does Discretionary Access control work with regard to server logs?
> > You said there was no superuser access, but I don't see any controls for
> > the logs. Do we need log_max_security_context?
> 
> It writes out server logs to filesystem, so its access controls should
> be applied in-kernel SELinux. SE-PostgreSQL does not care system call
> invocation, because it should be a task in kernel features.
> It works as a reference monitor for SQL queries.

Yes, the file will be protected. My thought was that we might wish to
control the levels of message placed in the file. Otherwise you would
require top level access to view the log. It doesn't seem likely that
they would want to give Top Secret level access to DBAs. Nor should any
level of DBAs be able to inspect who is running which SQL by simple
manipulation of parameters such as log_min_duration_statement.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

08 November 2008, 08:58:59

Simon Riggs wrote:
> On Fri, 2008-11-07 at 16:52 -0500, Bruce Momjian wrote:
>> Simon Riggs wrote:
>>>>> So if somebody with context x tries to delete value1 from TableB, they
>>>>> will be refused because of a row they cannot see. In this case the
>>>>> correct action is to update the tuple in TableB so it now has a
>>>>> security_context = y. The user with x cannot see it and can be persuaded
>>>>> he deleted it, while the user with y can still see it.
>>>> It seems odd for a low-privilege user to be able to elevate the
>>>> privilege of a tuple above their own privilege level.  I also don't
>>>> believe that the privilege level is a total order, which might make
>>>> this something of a sticky wicket.  But those are just my thoughts as
>>>> a non-guru.
>>> The low-privilege user isn't elevating the label. If the tuple was
>>> visible by multiple labels it was already elevated. All I am suggesting
>>> is the system remove the one it can see, leaving the other ones intact.
>>> This makes the row appear to be deleted by the lower privileged user,
>>> whereas in fact it was merely updated. There need not be any ordering to
>>> the labels for this scheme to work.
>> Simon, would you read the chapter on "covert channels"?  You might
>> understand it better than I do and it might give you some ideas:
>>
>>        http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.33.5950
> 
> It's probably easier just to say "here is the specification we;re
> working to implement".

This document gives us some of hints to be considered when we
apply mandatory access control facilities on database systems.

However, it is not a specification of SE-PostgreSQL.
The series of documents assumes traditional multi-level-security
system, so it does not care about flexible policy, type-enforcement
rules and collaborating with operating system.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

08 November 2008, 10:20:25

On Sat, 2008-11-08 at 18:58 +0900, KaiGai Kohei wrote:

> This document gives us some of hints to be considered when we
> apply mandatory access control facilities on database systems.
> 
> However, it is not a specification of SE-PostgreSQL.
> The series of documents assumes traditional multi-level-security
> system, so it does not care about flexible policy, type-enforcement
> rules and collaborating with operating system.

I'm sorry, but I don't understand your answer. 

The wiki seemed to indicate, to me, that the FK situation was a problem,
so I was trying to provide a solution. Personally, I could live with it
either way. But the important thing is: will this aspect prevent
SEPostgreSQL from achieving Common Criteria certification, or not? 

If it will pass, then I'm happy, even if a different, better solution
exists. If it will fail, then we must act. I'm not qualified to say what
will happen, but it would be good to see a very clear answer on this. If
it was already resolved, then please accept my apologies for raising the
issue again. Please could you update the Wiki docs to explain the agreed
resolution, its reasons and references? The design choices we make will
be questioned again in the future, so it will be good to have them
clear. Thanks.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

08 November 2008, 10:37:36

Simon Riggs wrote:
> On Sat, 2008-11-08 at 14:21 +0900, KaiGai Kohei wrote:
> 
>>> Some users will be able to take advantage of the facilities without
>>> implementing full MLS. Yet we want the full facilities for Government.
>>> Many people currently run multiple customers in different schemas,
>>> though this would let them just run a single set of tables so is much
>>> better for running many small customers.
>> SELinux community also provides a MLS enabled policy, but it is not
>> a default one now. SE-PostgreSQL has all the its access controls
>> decision externally, so it is out of our coltrols.
> 
> I have a couple of concerns: 
> 
> 1. if we make a direct link with SELinux then the code will be *much*
> less used and tested. It will be a constant battle to maintain
> SEPostgres in a bug-free state. I want to decouple the link so this code
> goes into normal Postgres. Other comments below are made with that
> thought in mind.

I don't deny that SELinux has not been our majority yet.
However, I also think other configuratin options have similar issues
less or more. We cannot test all the combinations of the options, so
it should not be a reason.

> 2. I see another use case here. For example, a customer runs a
> Software-as-a-service company where the same service is offered to 500
> customers. The same database schema is there for each customer, yet they
> must never see each other's data. Today, that is implemented as 500
> schemas, plus 1 schema with common data in it. ISTM we would be able to
> implement this better using SEPostgreSQL, with/without using SELinux.
> The need for common data is why in some cases we would want access
> controls placed only on certain tables.
> 
> Fulfilling use case (2) will also meet my concerns in (1).

We can assign a proper label for common tables, which allows everyone
to access it. I guess it can be implement wih MCS policy so well.
Any customer has its individual category (c0 to c499) and their data
is also labeled as same category, but the common table is kept
uncategorized.

> So I would prefer it if the solution was designed closely with SELinux,
> but usable and useful in other cases also. The link to SELinux could be
> done via a contrib plugin. A plugin is then optional. But the main
> plugin we provide can be built directly with SELinux.

The reason why SE-PostgreSQL feature is implemented on the PGACE security
framework is to provide end-users options.
We can switch preferable security feature or turn it off at this level.

If necessary, we can implement another option on the PGACE. The rowacl is
proof of the concept. It seems to me you consider a pluggable interface
to operating system enables to apply other security server, but, they can
have different security model, different security attribute and so on.

I think it is an appropriate decision to switch security features at the
hook level which does not provide any security model, more than integration
of different security design.

>>> The security context on each row could be an optional column present
>>> only if HEAP_HASSECURITYCONTEXT is set (0x0010 see htup.h), just like
>>> OIDs. Use a specific datatype rather than TEXT. That datatype could be
>>> an identifier to pg_security. Security people have big databases too, so
>>> we need to compress the security context more and take out parse time of
>>> string handling. Don't think we should use Oids, they're too big. Might
>>> be easier to use a 2byte field and restrict access to 32,000 contexts,
>>> which is easily enough. TEXT also makes me nervous, just in case there
>>> is some collation/encoding weirdness that allows contexts to be
>>> subverted. Fixed integers are hard to compromise in that respect.
>> An issue is who can decide the existance or needs of security system
>> attribute. If the table owner can disable it, we cannot say it as
>> a mandatory access control feature, so the security attribute has to
>> be always appeared when the security feature is enabled.
>>
>> Here is an answer for the expected question.
>> When we refer the "security_context" system column of tuples without
>> HEAP_HAS_SECURITY, it returns an alternative label called as "unlabeled_t",
>> because it has no labels.
> 
> Not really an issue. Just use a parameter. security_controls = mandatory
> or change the meaning of the existing one. Similar to default_with_oids,
> just that we have a setting where it isn't optional.

This choice should be contained in the security model which can be choosen
at the PGACE hook level. At least, I cannot imagine SELinux without mandatory
feature. However, I don't deny other security model which allows mandatory or
discretionary.

>> The reason why we adopt TEXT type is SELinux requires userspace
>> object manager makes queries via text represented security context,
>> and it also can be used for other security feature to show client
>> its security attribute in human readable format.
> 
> AFAICS, neither of those things means the datatype has to be TEXT.

We cannot represent a security context in TIMESTAMP or FLOAT datatype. :-)

From the viewpoint of user interfaces, I belive TEXT is an appropriate
datatype. It enables to provide an interface to manage security context
as they handle it on operating system.

>> For your information, in-kernel SELinux can handle 2^32 - 1 of security
>> context internally in theoretical maximum, so using Oid as a security
>> identifier is a fair decision to avoid an implementation to handle overflow
>> cases.
> 
> OK, so we need 4 bytes. I can live with that - at least its not ~10
> bytes/row. Does it need to be an Oid, or just the size of an Oid?

It utilize the fortune that Oid has 32bit length.
It enables us to indicate a tuple within pg_security without any
additional enhancement.

>>> How does Discretionary Access control work with regard to server logs?
>>> You said there was no superuser access, but I don't see any controls for
>>> the logs. Do we need log_max_security_context?
>> It writes out server logs to filesystem, so its access controls should
>> be applied in-kernel SELinux. SE-PostgreSQL does not care system call
>> invocation, because it should be a task in kernel features.
>> It works as a reference monitor for SQL queries.
> 
> Yes, the file will be protected. My thought was that we might wish to
> control the levels of message placed in the file. Otherwise you would
> require top level access to view the log. It doesn't seem likely that
> they would want to give Top Secret level access to DBAs. Nor should any
> level of DBAs be able to inspect who is running which SQL by simple
> manipulation of parameters such as log_min_duration_statement.

In this case, I recommend you to utilize TE rules, not MLS rules.
The default security policy provides postgresql_log_t type which prevent
to manupulate it from others.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

08 November 2008, 11:24:47

Simon Riggs wrote:
> On Sat, 2008-11-08 at 18:58 +0900, KaiGai Kohei wrote:
> 
>> This document gives us some of hints to be considered when we
>> apply mandatory access control facilities on database systems.
>>
>> However, it is not a specification of SE-PostgreSQL.
>> The series of documents assumes traditional multi-level-security
>> system, so it does not care about flexible policy, type-enforcement
>> rules and collaborating with operating system.
> 
> I'm sorry, but I don't understand your answer. 

What I wanted to say is that the security design of SELinux is combination
of TE(type enforcement), RBAC(role based access controls) and MLS(multi
level security) so we cannot apply specification of the document as-is.
In addition, its security policy is not hard-wired. These differences
gives us some more technical hurdles.

> The wiki seemed to indicate, to me, that the FK situation was a problem,
> so I was trying to provide a solution. Personally, I could live with it
> either way. But the important thing is: will this aspect prevent
> SEPostgreSQL from achieving Common Criteria certification, or not? 

Please note that I've learned the common criteria for a few years but
not a authority, and the answer may depends on certification agency.

In my understanding, it depends on assurance level of the certification
and what functional components are required by the its environment to
be used and threats to be considered here.
If we don't consider who can be a sponsor of the certification, it has
enough functionalities to pass the certification expect for extreme
requirements which well over enterprise class systems.

The covert channel analysis is contained at the FDP_IFF section in the
Common Criteria part 2, and it defines several classes of requirements
in information flow controls.
It defines six components and FDP_IFF.3, 4, 5 mentions the handling of
covert channels, but the 3 and 4 does not require there is no covert
channels.

FYI, some of certified database products also don't mention them.
For example, the certified Oracle Label Security 10g is evaluated as
EAL4+ class, but it mentions only FDP_IFF.2, not 3, 4 and 5.
The FDP_IFF.2 is label based mandatory access controls as SE-PostgreSQL
provides.

> Please could you update the Wiki docs to explain the agreed
> resolution, its reasons and references? The design choices we make will
> be questioned again in the future, so it will be good to have them
> clear. Thanks.

OK, I'll add it to the wiki document.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Updates of SE-PostgreSQL 8.4devel patches (r1206)

From

KaiGai Kohei

Date:

10 November 2008, 09:21:46

I updated the patch set of SE-PostgreSQL (revision 1206)

[1/6] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1206.patch
[2/6] http://sepgsql.googlecode.com/files/sepostgresql-pg_dump-8.4devel-3-r1206.patch
[3/6] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1206.patch
[4/6] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1206.patch
[5/6] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1206.patch
[6/6] http://sepgsql.googlecode.com/files/sepostgresql-row_acl-8.4devel-3-r1206.patch

Draft of the SE-PostgreSQL documentation is here: http://wiki.postgresql.org/wiki/SEPostgreSQL

List of updates:
- Patches are rebased to the latest CVS HEAD.
- Style of patches are changes by "diff -c".
- bugfix: {use} permission on user defined function made possibility to leak information without {select} permission.
- The wikipage is updated as I promised.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

12 November 2008, 18:33:17

On Fri, 2008-11-07 at 16:52 -0500, Bruce Momjian wrote:

> Simon, would you read the chapter on "covert channels"?  You might
> understand it better than I do and it might give you some ideas:
> 
>        http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.33.5950

OK, read that now.

Looks to me the covert channel debate will remain open whichever we do.

I agree with you that careful design avoids the problem, for the most
part. Even without that, it appears we have enough to achieve
certification.

The only remaining problem for me now is the size of the security
context column added to each row. I can accept a fixed length 4 byte
value, but anything longer just seems that it will render this unusable.
Normal apps should be able to benefit from row level security, as well
as high-security apps. The additional row overhead is enough to prevent
that, as well as put off many very large high security apps - which is
catastrophic because many of them are very large these days.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

13 November 2008, 00:44:55

Simon Riggs wrote:
> The only remaining problem for me now is the size of the security
> context column added to each row. I can accept a fixed length 4 byte
> value, but anything longer just seems that it will render this unusable.
> Normal apps should be able to benefit from row level security, as well
> as high-security apps. The additional row overhead is enough to prevent
> that, as well as put off many very large high security apps - which is
> catastrophic because many of them are very large these days.

It's unclear for me what is the point you said.
I guess you concern the fixed length field is always allocated in
the case when any security feature is not enabled also, or performance
degradation on the large scale databases.
If incorrect, please tell me in another expression.

At first, the fixed length 4 byte field is allocated only when
the SE-PostgreSQL (or other security feature) is enabled. It can be
controlled via PGACE hook. The pgaceSecurityAttributeNecessary() can
return bool value, and it indicates the necessity of the security
field. If SE-PostgreSQL is disabled on compile-time or run-time,
the fixed length 4 byte value is not allocated.

In the next, we assumes the users of SE-PostgreSQL don't give its
performance the first priority. However, the past measurement shows
its cost is far from the representation of "catastrophic".
The following article shows the result of pgbench at the 8.4devel
with rivision 1063.
 http://kaigai.sblo.jp/article/20303941.html

It shows 8% of degradation in the worst cases, and it has a trend
that larger scale database has smaller degradation.
(The measurement is same as I introduced a month ago.)

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Updates of SE-PostgreSQL 8.4devel patches (r1216)

From

KaiGai Kohei

Date:

14 November 2008, 10:35:05

I updated the patch set of SE-PostgreSQL (revision 1216).

There are no new features and bugfixes, but it was rebased to the latest
CVS HEAD, because the previous r1206 got a few conflictions.

These patches are ready for reviewing.
We need your help to make progress them so much!

[1/6] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1216.patch
[2/6] http://sepgsql.googlecode.com/files/sepostgresql-pg_dump-8.4devel-3-r1216.patch
[3/6] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1216.patch
[4/6] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1216.patch
[5/6] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1216.patch
[6/6] http://sepgsql.googlecode.com/files/sepostgresql-row_acl-8.4devel-3-r1216.patch

Draft of the SE-PostgreSQL documentation is here: http://wiki.postgresql.org/wiki/SEPostgreSQL

I could get a help from a native English speaker who works in SELinux community
as a documentation maintainer, so I'm also updating the description now.
If you notice anything, please feel free to comment also.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

14 November 2008, 13:48:33

On Thu, 2008-11-13 at 10:44 +0900, KaiGai Kohei wrote:

> It's unclear for me what is the point you said.
> I guess you concern the fixed length field is always allocated in
> the case when any security feature is not enabled also, or performance
> degradation on the large scale databases.
> If incorrect, please tell me in another expression.
> 
> At first, the fixed length 4 byte field is allocated only when
> the SE-PostgreSQL (or other security feature) is enabled. It can be
> controlled via PGACE hook. The pgaceSecurityAttributeNecessary() can
> return bool value, and it indicates the necessity of the security
> field. If SE-PostgreSQL is disabled on compile-time or run-time,
> the fixed length 4 byte value is not allocated.

I'm sorry for not making my thoughts clearer. Let me try again:

As I understand it, when enabled, the overhead for each row is more than
4 bytes because you include a text field also, which you say has a
restricted number of values. IMHO the overhead is unacceptable, given
that our row overhead is already high. I would prefer to make the
maximum overhead per row 4 bytes only, which matches the maximum number
of required labels. This will allow very large databases to use this
feature.

I would also like to see the feature part of normal Postgres, rather
than as a compile time option. The per-row overhead would then be
optional, just as WITH OIDS is optional. This would allow many
applications to take advantage of row level security, without the need
for switching to a different executable and without the need to enable
it for every table. For high security applications, default_row_security
= on would obviously be a requirement. With a single executable on all
distros we will have more robust software and it will be easier to
configure and use.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

14 November 2008, 14:58:23

Simon Riggs wrote:
> On Thu, 2008-11-13 at 10:44 +0900, KaiGai Kohei wrote:
> 
>> It's unclear for me what is the point you said.
>> I guess you concern the fixed length field is always allocated in
>> the case when any security feature is not enabled also, or performance
>> degradation on the large scale databases.
>> If incorrect, please tell me in another expression.
>>
>> At first, the fixed length 4 byte field is allocated only when
>> the SE-PostgreSQL (or other security feature) is enabled. It can be
>> controlled via PGACE hook. The pgaceSecurityAttributeNecessary() can
>> return bool value, and it indicates the necessity of the security
>> field. If SE-PostgreSQL is disabled on compile-time or run-time,
>> the fixed length 4 byte value is not allocated.
> 
> I'm sorry for not making my thoughts clearer. Let me try again:
> 
> As I understand it, when enabled, the overhead for each row is more than
> 4 bytes because you include a text field also, which you say has a
> restricted number of values. IMHO the overhead is unacceptable, given
> that our row overhead is already high. I would prefer to make the
> maximum overhead per row 4 bytes only, which matches the maximum number
> of required labels. This will allow very large databases to use this
> feature.

Indeed, the additional storage comsumption is not just only fixed 4 bytes
per tuple, because these security identifiers require its text representation.
However, the rate of them are nearly zero in very large databases, especially.

Please consider the following example:

postgres=# SELECT security_context, * FROM t1;             security_context               | a |  b
---------------------------------------------+---+-----unconfined_u:object_r:sepgsql_table_t:s0    | 1 |
aaaunconfined_u:object_r:sepgsql_table_t:s0   | 2 | bbbunconfined_u:object_r:sepgsql_table_t:s0    | 3 |
cccunconfined_u:object_r:sepgsql_table_t:s0:c0| 4 | dddunconfined_u:object_r:sepgsql_table_t:s0:c0 | 5 |
eeeunconfined_u:object_r:sepgsql_table_t:s0:c1| 6 | fffunconfined_u:object_r:sepgsql_table_t:s0:c1 | 7 |
gggunconfined_u:object_r:sepgsql_table_t:s0:c2| 8 | hhh

(8 rows)

postgres=# SELECT oid, * FROM pg_security; oid  |                  seclabel
-------+--------------------------------------------- 3397 | unconfined_u:object_r:sepgsql_table_t:s0 3398 |
unconfined_u:object_r:sepgsql_proc_t:s03399 | unconfined_u:object_r:sepgsql_db_t:s016390 |
unconfined_u:object_r:sepgsql_table_t:s0:c016391| unconfined_u:object_r:sepgsql_table_t:s0:c116392 |
unconfined_u:object_r:sepgsql_table_t:s0:c2
(6 rows)

The table "t1" contains eight tuples, and there are four kind of security context.
Every tuples has fixed 4-bytes identifier to indicate oid of pg_security which
contains text representation.
(Its output handler translate it into human readable text form.)
The first three tuples have "3397" as its security identifier which indicates
the tuple of "unconfined_u:object_r:sepgsql_table_t:s0" within pg_security.
This entry is shared by them.

If a table has 1,000,000 of tuples labeled by 100 kind of security contexts,
we only need 100 tuples on the pg_security system catalog. In this case, the sum
of text representation length is 5,000 byte at maximum, it is small enough.

In other word, the relationship between a security identifier and an entry of
pg_security is n:1, not 1:1.
Sorry, it seems to me you misunderstand something.

> I would also like to see the feature part of normal Postgres, rather
> than as a compile time option. The per-row overhead would then be
> optional, just as WITH OIDS is optional. This would allow many
> applications to take advantage of row level security, without the need
> for switching to a different executable and without the need to enable
> it for every table. For high security applications, default_row_security
> = on would obviously be a requirement. With a single executable on all
> distros we will have more robust software and it will be easier to
> configure and use.

An issue is who can enable or disable the row-level security option.
If the owner of table can do it discretionary, we don't call it a
"mandatory" access control feature.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

14 November 2008, 15:36:00

On Sat, 2008-11-15 at 00:58 +0900, KaiGai Kohei wrote:

> Sorry, it seems to me you misunderstand something.

Yep, seems so. Thank goodness for that. Thanks for putting me straight.

> > I would also like to see the feature part of normal Postgres, rather
> > than as a compile time option. The per-row overhead would then be
> > optional, just as WITH OIDS is optional. This would allow many
> > applications to take advantage of row level security, without the need
> > for switching to a different executable and without the need to enable
> > it for every table. For high security applications, default_row_security
> > = on would obviously be a requirement. With a single executable on all
> > distros we will have more robust software and it will be easier to
> > configure and use.
> 
> An issue is who can enable or disable the row-level security option.
> If the owner of table can do it discretionary, we don't call it a
> "mandatory" access control feature.

It seems fairly easy to do that with a GUC, or at least an option on
CREATE DATABASE, with no equivalent ALTER DATABASE option. Once created
with security, a table would not be able to turn off security. So nobody
would be able to turn off security for existing data.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

15 November 2008, 15:40:46

Simon Riggs wrote:
>>> I would also like to see the feature part of normal Postgres, rather
>>> than as a compile time option. The per-row overhead would then be
>>> optional, just as WITH OIDS is optional. This would allow many
>>> applications to take advantage of row level security, without the need
>>> for switching to a different executable and without the need to enable
>>> it for every table. For high security applications, default_row_security
>>> = on would obviously be a requirement. With a single executable on all
>>> distros we will have more robust software and it will be easier to
>>> configure and use.
>> An issue is who can enable or disable the row-level security option.
>> If the owner of table can do it discretionary, we don't call it a
>> "mandatory" access control feature.
> 
> It seems fairly easy to do that with a GUC, or at least an option on
> CREATE DATABASE, with no equivalent ALTER DATABASE option. Once created
> with security, a table would not be able to turn off security. So nobody
> would be able to turn off security for existing data.

SE-PostgreSQL already has a GUC option to enable/disable the feature,
so I don't think a GUC option to control features is not a strange idea.
In fact, I also had considered an option to disable the row-level access
control feature in the similar way, when I got a comment that platform
independent row-level security is a prerequisite to merge SE-PostgreSQL.
However, I made a decision to implement the "rowacl" feature, not a new
GUC option to disable row-level security.

I had two reasons why I didn't implement the option.

The first is the relationship between table/column-level access controls
and row-level controls on system catalogs is unclear.
For example, SE-PostgreSQL handles DELETE on pg_class as an operation to
drop table, because it has same meanings. The table-level access controls
break down, if only row-level checks are disabled.

The other is an issue on implementation. Even if row-level security is
disabled, tuples within pg_class, pg_attribute, pg_proc and pg_largeobject
has to hold its security context, because it means the security context of
tables, columns, procedure and largeobjects.
However, the length of a tuple is computed at heap_form_tuple(), and its
arguments don't contains the object id of relation. Thus, we cannot make
a decision whether the newly created tuple should have a security field, or
not. The heap_form_tuple() is invoked from 60 of functions. It might be
possible to change all the argument list to deliver relation id. But it
seems to me excess updates for the purpose.

Thus, I recommend you to assign uniformed security context whole of the table.
The default security policy allows all the clients to operate anything on
tuples labeled as "sepgsql_table_t" except for relabeling them.
(BTW, it is a default type when here is no special configuration.)

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

17 November 2008, 17:12:29

On Sun, 2008-11-16 at 01:40 +0900, KaiGai Kohei wrote:

> I had two reasons why I didn't implement the option.
> 
> The first is the relationship between table/column-level access controls
> and row-level controls on system catalogs is unclear.
> For example, SE-PostgreSQL handles DELETE on pg_class as an operation to
> drop table, because it has same meanings. The table-level access controls
> break down, if only row-level checks are disabled.

If pg_class needs a special case, no problem.

> The other is an issue on implementation. Even if row-level security is
> disabled, tuples within pg_class, pg_attribute, pg_proc and pg_largeobject
> has to hold its security context, because it means the security context of
> tables, columns, procedure and largeobjects.
> However, the length of a tuple is computed at heap_form_tuple(), and its
> arguments don't contains the object id of relation. Thus, we cannot make
> a decision whether the newly created tuple should have a security field, or
> not. The heap_form_tuple() is invoked from 60 of functions. It might be
> possible to change all the argument list to deliver relation id. But it
> seems to me excess updates for the purpose.

WITH OIDS seems to work well without problem. Why is this different?

I'm sure the problem is solvable. If you just formed the tuple you
probably know enough about a table to get security context. If not,
perhaps security context would be assigned/checked at a lower level.

It was designed with the security community in mind, but it would be
great if it can work for more people. Please could you look to see if
this is possible?

I think it will benefit everybody if this feature can work as an option
of the main server. Currently, this feature seems to support only one
configuration: a special build for SELinux on Red Hat. That's nice, but
I want to see the patch open things up more for other distros/OS, plus
options for people who don't want MAC, but do want row level security. 

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

18 November 2008, 01:51:52

Simon Riggs wrote:
>> The other is an issue on implementation. Even if row-level security is
>> disabled, tuples within pg_class, pg_attribute, pg_proc and pg_largeobject
>> has to hold its security context, because it means the security context of
>> tables, columns, procedure and largeobjects.
>> However, the length of a tuple is computed at heap_form_tuple(), and its
>> arguments don't contains the object id of relation. Thus, we cannot make
>> a decision whether the newly created tuple should have a security field, or
>> not. The heap_form_tuple() is invoked from 60 of functions. It might be
>> possible to change all the argument list to deliver relation id. But it
>> seems to me excess updates for the purpose.
> 
> WITH OIDS seems to work well without problem. Why is this different?

It is not a problem, but things which take a decision.

The heap_form_tuple() makes a new tuple according to the given
TupleDesc which has a bool variable of "tdhasoid".
This structure can be initialized at various functions. For example,
47 of functions invoke CreateTemplateTupleDesc() which requires an
argument of "bool hasoid".
If we follow the way of WITH OIDS, it will be necessary to modify
the declaration and 47 of invocations, at least.

I don't think your suggestion is a bad idea, except for the number
of modification on the core implementation.

> I think it will benefit everybody if this feature can work as an option
> of the main server. Currently, this feature seems to support only one
> configuration: a special build for SELinux on Red Hat. That's nice, but
> I want to see the patch open things up more for other distros/OS, plus
> options for people who don't want MAC, but do want row level security. 

This feature is not limited to Red Hat and related.

Some of distributions now provides SELinux option, but not a default.
I know Debian, Ubuntu, Gentoo and SuSE are doing.
Their default security policies are designed based on the upstreamed
SELinux policy which already has rules for SE-PostgreSQL. So, it will
be delivered to other distributions and enlarge its supporting platforms.

For other operating systems, I know Trusted Solaris folks have a plan
to implement their facilities on the PGACE security framework, at least.

My 6th patch (rowacl) provides a DAC based row-level security implementation.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

18 November 2008, 03:10:57

KaiGai Kohei wrote:
> Simon Riggs wrote:
>>> The other is an issue on implementation. Even if row-level security is
>>> disabled, tuples within pg_class, pg_attribute, pg_proc and pg_largeobject
>>> has to hold its security context, because it means the security context of
>>> tables, columns, procedure and largeobjects.
>>> However, the length of a tuple is computed at heap_form_tuple(), and its
>>> arguments don't contains the object id of relation. Thus, we cannot make
>>> a decision whether the newly created tuple should have a security field, or
>>> not. The heap_form_tuple() is invoked from 60 of functions. It might be
>>> possible to change all the argument list to deliver relation id. But it
>>> seems to me excess updates for the purpose.
>> WITH OIDS seems to work well without problem. Why is this different?
> 
> It is not a problem, but things which take a decision.
> 
> The heap_form_tuple() makes a new tuple according to the given
> TupleDesc which has a bool variable of "tdhasoid".
> This structure can be initialized at various functions. For example,
> 47 of functions invoke CreateTemplateTupleDesc() which requires an
> argument of "bool hasoid".
> If we follow the way of WITH OIDS, it will be necessary to modify
> the declaration and 47 of invocations, at least.
> 
> I don't think your suggestion is a bad idea, except for the number
> of modification on the core implementation.

In addition, I want you to make clear the priority of the facility.

We can disable row-level access control factually with uniformed
security context which allows client any kind of accesses.
In this case, all tuples shares a single text representation,
so the size of external text representation is enough small to
ignore.

I guess you concerned the per-tuple security attribute because
it has 1:1 mapped text representation on pg_security.
However, it is incorrect.
I don't think we should give high priority for the feature to
kill per-tuple security attribute now.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

18 November 2008, 04:26:56

On Tue, 2008-11-18 at 11:51 +0900, KaiGai Kohei wrote:
> Simon Riggs wrote:
> >> The other is an issue on implementation. Even if row-level security is
> >> disabled, tuples within pg_class, pg_attribute, pg_proc and pg_largeobject
> >> has to hold its security context, because it means the security context of
> >> tables, columns, procedure and largeobjects.
> >> However, the length of a tuple is computed at heap_form_tuple(), and its
> >> arguments don't contains the object id of relation. Thus, we cannot make
> >> a decision whether the newly created tuple should have a security field, or
> >> not. The heap_form_tuple() is invoked from 60 of functions. It might be
> >> possible to change all the argument list to deliver relation id. But it
> >> seems to me excess updates for the purpose.
> > 
> > WITH OIDS seems to work well without problem. Why is this different?
> 
> It is not a problem, but things which take a decision.
> 
> The heap_form_tuple() makes a new tuple according to the given
> TupleDesc which has a bool variable of "tdhasoid".
> This structure can be initialized at various functions. For example,
> 47 of functions invoke CreateTemplateTupleDesc() which requires an
> argument of "bool hasoid".
> If we follow the way of WITH OIDS, it will be necessary to modify
> the declaration and 47 of invocations, at least.
> 
> I don't think your suggestion is a bad idea, except for the number
> of modification on the core implementation.

We manage to include a NULL bitmap without doing this.

The hasoids option seems to be unused on most of those call points. How
many of those call points would need security context?

The change you suggest looks possible, so maybe it is the way, but it
also looks fairly ugly already. If we did this it would not be by adding
another bool, but by adding an options object.

Another way would be to include a security context in all newly created
tuples, but remove it during heap_update, heap_insert etc if it is
unused by the relation. That seems more straightforward.

> > I think it will benefit everybody if this feature can work as an option
> > of the main server. Currently, this feature seems to support only one
> > configuration: a special build for SELinux on Red Hat. That's nice, but
> > I want to see the patch open things up more for other distros/OS, plus
> > options for people who don't want MAC, but do want row level security. 
> 
> This feature is not limited to Red Hat and related.
> 
> Some of distributions now provides SELinux option, but not a default.
> I know Debian, Ubuntu, Gentoo and SuSE are doing.

SUSE?

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

18 November 2008, 04:31:17

On Tue, 2008-11-18 at 13:10 +0900, KaiGai Kohei wrote:
> KaiGai Kohei wrote:
> > Simon Riggs wrote:
> >>> The other is an issue on implementation. Even if row-level security is
> >>> disabled, tuples within pg_class, pg_attribute, pg_proc and pg_largeobject
> >>> has to hold its security context, because it means the security context of
> >>> tables, columns, procedure and largeobjects.
> >>> However, the length of a tuple is computed at heap_form_tuple(), and its
> >>> arguments don't contains the object id of relation. Thus, we cannot make
> >>> a decision whether the newly created tuple should have a security field, or
> >>> not. The heap_form_tuple() is invoked from 60 of functions. It might be
> >>> possible to change all the argument list to deliver relation id. But it
> >>> seems to me excess updates for the purpose.
> >> WITH OIDS seems to work well without problem. Why is this different?
> > 
> > It is not a problem, but things which take a decision.
> > 
> > The heap_form_tuple() makes a new tuple according to the given
> > TupleDesc which has a bool variable of "tdhasoid".
> > This structure can be initialized at various functions. For example,
> > 47 of functions invoke CreateTemplateTupleDesc() which requires an
> > argument of "bool hasoid".
> > If we follow the way of WITH OIDS, it will be necessary to modify
> > the declaration and 47 of invocations, at least.
> > 
> > I don't think your suggestion is a bad idea, except for the number
> > of modification on the core implementation.
> 
> In addition, I want you to make clear the priority of the facility.
> 
> We can disable row-level access control factually with uniformed
> security context which allows client any kind of accesses.
> In this case, all tuples shares a single text representation,
> so the size of external text representation is enough small to
> ignore.

I don't think it is small enough to ignore. 

> I guess you concerned the per-tuple security attribute because
> it has 1:1 mapped text representation on pg_security.
> However, it is incorrect.
> I don't think we should give high priority for the feature to
> kill per-tuple security attribute now.

*Maybe* if you view my request this way:

If a table is defined such that all of its tuples have an identical
security-context then we can optimise away the additional field when
storing tuples.

That might help, it might not. Just trying to help us think of different
ways to allow this within the main server variant.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

18 November 2008, 05:03:10

Simon Riggs wrote:
> On Tue, 2008-11-18 at 11:51 +0900, KaiGai Kohei wrote:
>> Simon Riggs wrote:
>>>> The other is an issue on implementation. Even if row-level security is
>>>> disabled, tuples within pg_class, pg_attribute, pg_proc and pg_largeobject
>>>> has to hold its security context, because it means the security context of
>>>> tables, columns, procedure and largeobjects.
>>>> However, the length of a tuple is computed at heap_form_tuple(), and its
>>>> arguments don't contains the object id of relation. Thus, we cannot make
>>>> a decision whether the newly created tuple should have a security field, or
>>>> not. The heap_form_tuple() is invoked from 60 of functions. It might be
>>>> possible to change all the argument list to deliver relation id. But it
>>>> seems to me excess updates for the purpose.
>>> WITH OIDS seems to work well without problem. Why is this different?
>> It is not a problem, but things which take a decision.
>>
>> The heap_form_tuple() makes a new tuple according to the given
>> TupleDesc which has a bool variable of "tdhasoid".
>> This structure can be initialized at various functions. For example,
>> 47 of functions invoke CreateTemplateTupleDesc() which requires an
>> argument of "bool hasoid".
>> If we follow the way of WITH OIDS, it will be necessary to modify
>> the declaration and 47 of invocations, at least.
>>
>> I don't think your suggestion is a bad idea, except for the number
>> of modification on the core implementation.
> 
> We manage to include a NULL bitmap without doing this.

The heap_form_tuple() has an argument of "bool *isnull".
It is a hint of NULL bitmap.

> The hasoids option seems to be unused on most of those call points. How
> many of those call points would need security context?

If we focus on the CreateTemplateTupleDesc(), 5 of call points give
possibile "hasoid" argument, and rest of them always give "false".
I guess it will be same in the security context cases.
However, we have to change all the call points when the declaration
is changed.

> Another way would be to include a security context in all newly created
> tuples, but remove it during heap_update, heap_insert etc if it is
> unused by the relation. That seems more straightforward.

It is not a reasonable option.

The length of HeapTupleData is determined during heap_form_tuple(),
and it is unchanged later. Thus, we have to interpose here, as object
identifier doing.

>> Some of distributions now provides SELinux option, but not a default.
>> I know Debian, Ubuntu, Gentoo and SuSE are doing.
> 
> SUSE?

The "u" might be a large-letter.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

18 November 2008, 06:04:54

On Tue, 2008-11-18 at 15:02 +0900, KaiGai Kohei wrote:

> If we focus on the CreateTemplateTupleDesc(), 5 of call points give
> possibile "hasoid" argument, and rest of them always give "false".
> I guess it will be same in the security context cases.
> However, we have to change all the call points when the declaration
> is changed.

Looks promising.

> > Another way would be to include a security context in all newly
> created
> > tuples, but remove it during heap_update, heap_insert etc if it is
> > unused by the relation. That seems more straightforward.
> 
> It is not a reasonable option.
> 
> The length of HeapTupleData is determined during heap_form_tuple(),
> and it is unchanged later. Thus, we have to interpose here, as object
> identifier doing.

Currently yes. Is there a reason not to? Do we rely on the tuple length
staying same after those operations?

Just considering multiple ways of making the context optional.

> >> Some of distributions now provides SELinux option, but not a
> default.
> >> I know Debian, Ubuntu, Gentoo and SuSE are doing.
> > 
> > SUSE?
> 
> The "u" might be a large-letter.

Sorry, I wasn't correcting your spelling! :-) 
I was asking whether Su/USE are definitely supporting SELinux now? I
have not heard that.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

18 November 2008, 06:52:03

Simon Riggs wrote:
>>> Another way would be to include a security context in all newly
>> created
>>> tuples, but remove it during heap_update, heap_insert etc if it is
>>> unused by the relation. That seems more straightforward.
>> It is not a reasonable option.
>>
>> The length of HeapTupleData is determined during heap_form_tuple(),
>> and it is unchanged later. Thus, we have to interpose here, as object
>> identifier doing.
> 
> Currently yes. Is there a reason not to? Do we rely on the tuple length
> staying same after those operations?
> 
> Just considering multiple ways of making the context optional.

Indeed, we can consider several options.

However, I don't want to change existing semantics in the core implementation
as far as possible. If we have to choose one of them, I prefer to add TupleDesc
a bool variable to show necessity of security field, because it requires many
points to be updated, but most of them are obvious.

Anyway, I've started to work with the prior approach.
Now we have less than two weeks remained in the CommitFest:Nov, so we have
no time to be spent uselessly.

>>> SUSE?
>> The "u" might be a large-letter.
> 
> Sorry, I wasn't correcting your spelling! :-) 
> I was asking whether Su/USE are definitely supporting SELinux now? I
> have not heard that.

It is a recent news.
http://news.opensuse.org/2008/08/20/opensuse-to-add-selinux-basic-enablement-in-111/

The openSUSE pressed they start to support SELinux, not only AppArmor.
However, I don't have enough information for the roadmap of SUSE Enterprise Server
which is a production version of Novell.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

18 November 2008, 08:02:32

>>> The length of HeapTupleData is determined during heap_form_tuple(),
>>> and it is unchanged later. Thus, we have to interpose here, as object
>>> identifier doing.
>> Currently yes. Is there a reason not to? Do we rely on the tuple length
>> staying same after those operations?
>>
>> Just considering multiple ways of making the context optional.
> 
> Indeed, we can consider several options.
> 
> However, I don't want to change existing semantics in the core implementation
> as far as possible. If we have to choose one of them, I prefer to add TupleDesc
> a bool variable to show necessity of security field, because it requires many
> points to be updated, but most of them are obvious.

Ah, however, it made an explosion of the number of points to be patched soon.

I'll try your approash in first, as follows:

1. When the SECURITY_SYSATTR_NAME is defined, heap_form_tuple() always allocate  a field to store security attribute.
2. The security mechanism can store its security attribute on the hooks for  insert or update tuples. It also can
remainthe field as InvalidOid.
 
3. The modified heap_insert() and heap_update() cut off the security field  and moves backward region by sizeof(Oid)
bytes,when it is remained as  InvalidOid.
 

> Anyway, I've started to work with the prior approach.
> Now we have less than two weeks remained in the CommitFest:Nov, so we have
> no time to be spent uselessly.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

18 November 2008, 09:09:22

On Tue, 2008-11-18 at 16:51 +0900, KaiGai Kohei wrote:

> Anyway, I've started to work with the prior approach.

Sounds good.

> Now we have less than two weeks remained in the CommitFest:Nov, so we have
> no time to be spent uselessly.
> 
> >>> SUSE?
> >> The "u" might be a large-letter.
> > 
> > Sorry, I wasn't correcting your spelling! :-) 
> > I was asking whether Su/USE are definitely supporting SELinux now? I
> > have not heard that.
> 
> It is a recent news.
> http://news.opensuse.org/2008/08/20/opensuse-to-add-selinux-basic-enablement-in-111/
> 
> The openSUSE pressed they start to support SELinux, not only AppArmor.
> However, I don't have enough information for the roadmap of SUSE Enterprise Server
> which is a production version of Novell.

OK, I heard that slightly differently. Thanks for the link.
The report I read emphasised the lack of support.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

18 November 2008, 11:41:27

Simon Riggs wrote:
> On Tue, 2008-11-18 at 16:51 +0900, KaiGai Kohei wrote:
> 
>> Anyway, I've started to work with the prior approach.
> 
> Sounds good.

The matters currently I faces:

* In the approach.1 (add "tdhassecurity" to TupleDesc)
An explosion of the number of points to be patched. :(

* In the approach.2 (strip security field in heap_insert/update, if unused)

Some implementations assumes an older and newer tuple have
same length of its header, However, a security field can be
striped in the heap_insert/update(), if unused.
Thus, this approach has to allow them to have different length
between older tuple and a result of heap_form_tuple().

In this approach, we have to list up all the implementations
which assume fixed length tuple-header, and fix them.

----
My preference is second one.
It will also enable to clean up many of "hasoid" bool abound the implementation.
However, I want to challenge the enhancement at the v8.5 development cycle,
as a part of writable "oid" system column feature.
(I have a plan to propose the feature next to the SE-PostgreSQL.)

A concern is why you suggested this feature at the last half of the November. :(
*Now*, I don't want to merge the feature to disable row-level security into the
current patch set, because it damages qualities of the codes.

Is there any other opinions?

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

18 November 2008, 12:21:59

On Tue, 2008-11-18 at 21:40 +0900, KaiGai Kohei wrote:

> A concern is why you suggested this feature at the last half of the
> November. :(

I gave you my feedback as soon as I read the Wiki article. Even then I
didn't understand some aspects of the patch design, which was
unfortunate. But these things take time.

I would encourage all developers to post a design description of what
their patch does long before they write it. That way this kind of
feedback comes as early as possible.

Please don't worry. You've done a great job and many people would like
to see your work succeed. Getting things right takes time and it really
is worth it in the end.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Tom Lane

Date:

18 November 2008, 15:08:23

KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
> I'll try your approash in first, as follows:

This seems a vast amount of uglification to avoid adding an argument to
CreateTemplateTupleDesc.  We do that kind of thing all the time --- it
is a simple and reliable change to make.

When designing a patch, you should generally try to make the code look
like the patch has been there all along.  Contorting logic to avoid
a simple API change is not good.
        regards, tom lane

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

18 November 2008, 15:42:47

Tom Lane wrote:
> KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
>> I'll try your approash in first, as follows:
> 
> This seems a vast amount of uglification to avoid adding an argument to
> CreateTemplateTupleDesc.  We do that kind of thing all the time --- it
> is a simple and reliable change to make.
> 
> When designing a patch, you should generally try to make the code look
> like the patch has been there all along.  Contorting logic to avoid
> a simple API change is not good.

Hmm..., I think your opinion is right, indeed.

In my conclusion, I want to postpone the feature to toggle row-level
access controls due to its scale of changes.

I can understand the requirements for reduction of storage consumption
with security attribute. I'll implement it next to the main feature of
SE-PostgreSQL.

Simon,
I'm sorry, if my expression felt you uncomfortable.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

18 November 2008, 16:14:19

On Wed, 2008-11-19 at 01:42 +0900, KaiGai Kohei wrote:
> Simon,

> I'm sorry, if my expression felt you uncomfortable.

No worries at all. I know exactly how you feel. Good Luck.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Bruce Momjian

Date:

20 November 2008, 00:07:03

Tom Lane wrote:
> KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
> > I'll try your approash in first, as follows:
> 
> This seems a vast amount of uglification to avoid adding an argument to
> CreateTemplateTupleDesc.  We do that kind of thing all the time --- it
> is a simple and reliable change to make.
> 
> When designing a patch, you should generally try to make the code look
> like the patch has been there all along.  Contorting logic to avoid
> a simple API change is not good.

Just to chime in, I agree with Simon's direction of making the security
specification for the table match WITH OIDS, and agree with Tom that the
implementation should follow the WITH OIDS API for clarity, not trying
to reduce the change footprint.  Basically, if WITH OIDS and security
definer behave the same in the API, there is little additional code
_complexity_, even if the patch is now larger.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

20 November 2008, 03:06:45

Bruce Momjian wrote:
> Tom Lane wrote:
>> KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
>>> I'll try your approash in first, as follows:
>> This seems a vast amount of uglification to avoid adding an argument to
>> CreateTemplateTupleDesc.  We do that kind of thing all the time --- it
>> is a simple and reliable change to make.
>>
>> When designing a patch, you should generally try to make the code look
>> like the patch has been there all along.  Contorting logic to avoid
>> a simple API change is not good.
> 
> Just to chime in, I agree with Simon's direction of making the security
> specification for the table match WITH OIDS, and agree with Tom that the
> implementation should follow the WITH OIDS API for clarity, not trying
> to reduce the change footprint.  Basically, if WITH OIDS and security
> definer behave the same in the API, there is little additional code
> _complexity_, even if the patch is now larger.

OK, I'll try to start implementing the feature again.

However, the toggle of row-level security feature should be controled
via a GUC option, not a discretionary option.
I'll add a "sepostgresql_row_level" option defined as bool to control
it on start up time.

In addition, please do not stop reviewing the current patch set
due to lack of the feature to disable row-level security.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Bruce Momjian

Date:

20 November 2008, 03:35:59

KaiGai Kohei wrote:
> Bruce Momjian wrote:
> > Tom Lane wrote:
> >> KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
> >>> I'll try your approash in first, as follows:
> >> This seems a vast amount of uglification to avoid adding an argument to
> >> CreateTemplateTupleDesc.  We do that kind of thing all the time --- it
> >> is a simple and reliable change to make.
> >>
> >> When designing a patch, you should generally try to make the code look
> >> like the patch has been there all along.  Contorting logic to avoid
> >> a simple API change is not good.
> > 
> > Just to chime in, I agree with Simon's direction of making the security
> > specification for the table match WITH OIDS, and agree with Tom that the
> > implementation should follow the WITH OIDS API for clarity, not trying
> > to reduce the change footprint.  Basically, if WITH OIDS and security
> > definer behave the same in the API, there is little additional code
> > _complexity_, even if the patch is now larger.
> 
> OK, I'll try to start implementing the feature again.
> 
> However, the toggle of row-level security feature should be controled
> via a GUC option, not a discretionary option.
> I'll add a "sepostgresql_row_level" option defined as bool to control
> it on start up time.

This sounds similar to BSD capability were certain security settings can
only be changed in single-user mode.

> In addition, please do not stop reviewing the current patch set
> due to lack of the feature to disable row-level security.

Of course.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Bruce Momjian

Date:

20 November 2008, 14:31:45

Bruce Momjian wrote:
> > However, the toggle of row-level security feature should be controled
> > via a GUC option, not a discretionary option.
> > I'll add a "sepostgresql_row_level" option defined as bool to control
> > it on start up time.
> 
> This sounds similar to BSD capability were certain security settings can
> only be changed in single-user mode.

Actually, an interesting idea would be to allow "sepostgresql_row_level"
to be turned on, but not off.  That means if it was turned on in
postgresql.conf, it could not be turned off, but if it is off in
postgresql.conf, it could be turned on via SET or via ALTER
USER/DATABASE;  I think that would be a nice capability.

On a related note, KaiGai, you are now starting the long road of getting
feedback with the ultimate goal of getting your patch into CVS.  I will
warn you that there is often much work during this stage, and it might
stretch into January as we request adjustments, but ultimately your
feature and Postgres will be better for it.  Thanks for sticking with
it.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

21 November 2008, 00:54:00

Bruce Momjian wrote:
> Bruce Momjian wrote:
>>> However, the toggle of row-level security feature should be controled
>>> via a GUC option, not a discretionary option.
>>> I'll add a "sepostgresql_row_level" option defined as bool to control
>>> it on start up time.
>> This sounds similar to BSD capability were certain security settings can
>> only be changed in single-user mode.
> 
> Actually, an interesting idea would be to allow "sepostgresql_row_level"
> to be turned on, but not off.  That means if it was turned on in
> postgresql.conf, it could not be turned off, but if it is off in
> postgresql.conf, it could be turned on via SET or via ALTER
> USER/DATABASE;  I think that would be a nice capability.

I think the "sepostgresql_mode" and "sepostgresql_row_level" should not
be toggled frequently.

Please consider SELinux/SE-PostgreSQL requires various kind of objects
(including database objects) to be labeled properly at the initial state.
If it allows clients to turn on row-level security feature, it means many
"unlabeled" tuples appear suddenly. In generally, these have to be labeled
before the system get being available.

> On a related note, KaiGai, you are now starting the long road of getting
> feedback with the ultimate goal of getting your patch into CVS.  I will
> warn you that there is often much work during this stage, and it might
> stretch into January as we request adjustments, but ultimately your
> feature and Postgres will be better for it.  Thanks for sticking with
> it.

Don't worry, I'm be available for the works, and give a lot for inclusion
of the feature at v8.4.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Bruce Momjian

Date:

21 November 2008, 01:59:13

KaiGai Kohei wrote:
> Bruce Momjian wrote:
> > Bruce Momjian wrote:
> >>> However, the toggle of row-level security feature should be controled
> >>> via a GUC option, not a discretionary option.
> >>> I'll add a "sepostgresql_row_level" option defined as bool to control
> >>> it on start up time.
> >> This sounds similar to BSD capability were certain security settings can
> >> only be changed in single-user mode.
> > 
> > Actually, an interesting idea would be to allow "sepostgresql_row_level"
> > to be turned on, but not off.  That means if it was turned on in
> > postgresql.conf, it could not be turned off, but if it is off in
> > postgresql.conf, it could be turned on via SET or via ALTER
> > USER/DATABASE;  I think that would be a nice capability.
> 
> I think the "sepostgresql_mode" and "sepostgresql_row_level" should not
> be toggled frequently.
> 
> Please consider SELinux/SE-PostgreSQL requires various kind of objects
> (including database objects) to be labeled properly at the initial state.
> If it allows clients to turn on row-level security feature, it means many
> "unlabeled" tuples appear suddenly. In generally, these have to be labeled
> before the system get being available.

I was thinking more about people are using the SQL-level row
permissions.  Are they going to want it for all tables for all
databases, or not at all?  I assumed they would want to be more
selective.  I agree the SE-Linux users would probably not toggle it for
different objects and databases frequently.

Actually, you called it "sepostgresql_mode", SE*, but how are we going
to control the SQL-level row permissions?  Are we using this name?  I
didn't think so because it seems so associated withe SE-Linux, and if
not, how would one say whether a table has SQL-level row permissions?

> > On a related note, KaiGai, you are now starting the long road of getting
> > feedback with the ultimate goal of getting your patch into CVS.  I will
> > warn you that there is often much work during this stage, and it might
> > stretch into January as we request adjustments, but ultimately your
> > feature and Postgres will be better for it.  Thanks for sticking with
> > it.
> 
> Don't worry, I'm be available for the works, and give a lot for inclusion
> of the feature at v8.4.

Great.  Sometimes these bold additions can require perhaps thirty
changes before they are added to CVS, I am afraid to say.  It can be a
painful part of open source development, even though in the end it is
worth it. (While it is happening, it doesn't seem very useful.)

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

21 November 2008, 02:50:59

>> Please consider SELinux/SE-PostgreSQL requires various kind of objects
>> (including database objects) to be labeled properly at the initial state.
>> If it allows clients to turn on row-level security feature, it means many
>> "unlabeled" tuples appear suddenly. In generally, these have to be labeled
>> before the system get being available.
> 
> I was thinking more about people are using the SQL-level row
> permissions.  Are they going to want it for all tables for all
> databases, or not at all?

We don't have a reason why the SQL-level row permissions should be toggled
in global only. It it designed based on DAC policy, so it is quite natural
to be controled by owner of resources.
(However, it is not implemented yet.)

> Actually, you called it "sepostgresql_mode", SE*, but how are we going
> to control the SQL-level row permissions?  Are we using this name?  I
> didn't think so because it seems so associated withe SE-Linux, and if
> not, how would one say whether a table has SQL-level row permissions?

A new GUC variable of "row_acl_is_enabled" allows us to toggle the SQL-level
row permission feature, when the binary is built with "--enable-row-acl".
In this case, the "sepostgresql_*" are enclosed by #ifdef HAVE_SELINUX, so
these are not available.

>>> On a related note, KaiGai, you are now starting the long road of getting
>>> feedback with the ultimate goal of getting your patch into CVS.  I will
>>> warn you that there is often much work during this stage, and it might
>>> stretch into January as we request adjustments, but ultimately your
>>> feature and Postgres will be better for it.  Thanks for sticking with
>>> it.
>> Don't worry, I'm be available for the works, and give a lot for inclusion
>> of the feature at v8.4.
> 
> Great.  Sometimes these bold additions can require perhaps thirty
> changes before they are added to CVS, I am afraid to say.  It can be a
> painful part of open source development, even though in the end it is
> worth it. (While it is happening, it doesn't seem very useful.)

s/painful/dynamic/g :-)

If you can ready to post some of comments, earlier is happier for me.
It is unclear for me when the CommtiFest:Nov is finished, but it is sure
we don't have enough days.

In my guess, I'll be able to complete to put a flag within TupleDesc to
control security field of HeapTupleHeader by tomorrow. And I have a plan
to check its behavior in this weekend before merge them into my tree.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Tom Lane

Date:

21 November 2008, 03:04:41

KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
> It is unclear for me when the CommtiFest:Nov is finished, but it is sure
> we don't have enough days.

This commitfest goes until it's done.  I knew at the beginning that we'd
not be done at the end of November.  The original schedule allowed two
months for this fest; we'll see whether that was optimistic or not.
        regards, tom lane

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Bruce Momjian

Date:

21 November 2008, 03:53:48

KaiGai Kohei wrote:
> >> Please consider SELinux/SE-PostgreSQL requires various kind of objects
> >> (including database objects) to be labeled properly at the initial state.
> >> If it allows clients to turn on row-level security feature, it means many
> >> "unlabeled" tuples appear suddenly. In generally, these have to be labeled
> >> before the system get being available.
> > 
> > I was thinking more about people are using the SQL-level row
> > permissions.  Are they going to want it for all tables for all
> > databases, or not at all?
> 
> We don't have a reason why the SQL-level row permissions should be toggled
> in global only. It it designed based on DAC policy, so it is quite natural
> to be controled by owner of resources.
> (However, it is not implemented yet.)

Yes, that was my question --- how will SQL-level row permissions be
controlled by the user.

> > Actually, you called it "sepostgresql_mode", SE*, but how are we going
> > to control the SQL-level row permissions?  Are we using this name?  I
> > didn't think so because it seems so associated withe SE-Linux, and if
> > not, how would one say whether a table has SQL-level row permissions?
> 
> A new GUC variable of "row_acl_is_enabled" allows us to toggle the SQL-level
> row permission feature, when the binary is built with "--enable-row-acl".

I assumed we would always enable SQL-level row permissions.  Why would
it be a compile-time option?

> In this case, the "sepostgresql_*" are enclosed by #ifdef HAVE_SELINUX, so
> these are not available.

Right.

> >>> On a related note, KaiGai, you are now starting the long road of getting
> >>> feedback with the ultimate goal of getting your patch into CVS.  I will
> >>> warn you that there is often much work during this stage, and it might
> >>> stretch into January as we request adjustments, but ultimately your
> >>> feature and Postgres will be better for it.  Thanks for sticking with
> >>> it.
> >> Don't worry, I'm be available for the works, and give a lot for inclusion
> >> of the feature at v8.4.
> > 
> > Great.  Sometimes these bold additions can require perhaps thirty
> > changes before they are added to CVS, I am afraid to say.  It can be a
> > painful part of open source development, even though in the end it is
> > worth it. (While it is happening, it doesn't seem very useful.)
> 
> s/painful/dynamic/g :-)

That's looking on the bright side of things.  :-)

> If you can ready to post some of comments, earlier is happier for me.

My guess is we will continue to send you ideas.

> It is unclear for me when the CommtiFest:Nov is finished, but it is sure
> we don't have enough days.

This is the last commit fest for 8.4 so it will probably go well into
December, perhaps January.

> In my guess, I'll be able to complete to put a flag within TupleDesc to
> control security field of HeapTupleHeader by tomorrow. And I have a plan
> to check its behavior in this weekend before merge them into my tree.

Nice.  I will look over your newest version as soon as I can.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

21 November 2008, 09:47:35

Bruce Momjian wrote:
> KaiGai Kohei wrote:
>>>> Please consider SELinux/SE-PostgreSQL requires various kind of objects
>>>> (including database objects) to be labeled properly at the initial state.
>>>> If it allows clients to turn on row-level security feature, it means many
>>>> "unlabeled" tuples appear suddenly. In generally, these have to be labeled
>>>> before the system get being available.
>>> I was thinking more about people are using the SQL-level row
>>> permissions.  Are they going to want it for all tables for all
>>> databases, or not at all?
>> We don't have a reason why the SQL-level row permissions should be toggled
>> in global only. It it designed based on DAC policy, so it is quite natural
>> to be controled by owner of resources.
>> (However, it is not implemented yet.)
> 
> Yes, that was my question --- how will SQL-level row permissions be
> controlled by the user.

When the given tuple has no ACL, it applies the default behavior which
allows anything for public. This behavior intends to keep compatible
works compared to the vanilla PostgreSQL.

We can have two state for "no ACLs". The first one is when tuples don't
have fixed 4-bytes security attributes. It can be happen when PostgreSQL
with SQL-level row-permissions mount compatible database files created
by vanilla PostgreSQL. The other is the fixed 4-byte security attribute
indicates an entry of "no ACLs". It seems unnecessary consumption, but
we cannot determine whether the user tries to set ACLs when heap_form_tuple().

One considerable solution is to add an RowACL specific table option to
disable row-level ACLs whole of the tables. It can be suitable for security
design because the option is provided by the resource owner.

For example:
  CRATE TABLE t (      a int,      b text  ) WITH (row_acl=disable);

If the reloptions contains an information to determine whether a new tuple
need the security field, or not, we can reflect it at heap_form_tuple().

>>> Actually, you called it "sepostgresql_mode", SE*, but how are we going
>>> to control the SQL-level row permissions?  Are we using this name?  I
>>> didn't think so because it seems so associated withe SE-Linux, and if
>>> not, how would one say whether a table has SQL-level row permissions?
>> A new GUC variable of "row_acl_is_enabled" allows us to toggle the SQL-level
>> row permission feature, when the binary is built with "--enable-row-acl".
> 
> I assumed we would always enable SQL-level row permissions.  Why would
> it be a compile-time option?

The SQL-level row permission feature is implemented as a guest of PGACE
security framework, so we need to select one of the guest mehanism when
compile-time.
The "--enable-selinux" and "--enable-row-acl" are exclusive option.

>> In my guess, I'll be able to complete to put a flag within TupleDesc to
>> control security field of HeapTupleHeader by tomorrow. And I have a plan
>> to check its behavior in this weekend before merge them into my tree.
> 
> Nice.  I will look over your newest version as soon as I can.

Today, I could complete to implement the newer version which passes
standard regression tests except for two cases expected.

Simon,
In the result of "pgbench -i -s 10", the "sepostgresql_row_level=true"
case consumed 152MB of storage, and the "sepostgresql_row_level=false"
case consumed 148MB of storage.

[kaigai@saba sepgsql]$ du -hs $PGDATA/base/16384
152M    /opt/sepgsql/base/16384
[kaigai@saba sepgsql]$ du -hs $PGDATA/base/16385
148M    /opt/sepgsql/base/16385

Now I'm under preparation to submit the newest patch set. ToDo:  - Check behavior for SE-PostgreSQL specific
operations.   (like security context updates)  - Implementation of new table options to disable row-acl.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Bruce Momjian

Date:

21 November 2008, 13:57:49

KaiGai Kohei wrote:
> Bruce Momjian wrote:
> > KaiGai Kohei wrote:
> >>>> Please consider SELinux/SE-PostgreSQL requires various kind of objects
> >>>> (including database objects) to be labeled properly at the initial state.
> >>>> If it allows clients to turn on row-level security feature, it means many
> >>>> "unlabeled" tuples appear suddenly. In generally, these have to be labeled
> >>>> before the system get being available.
> >>> I was thinking more about people are using the SQL-level row
> >>> permissions.  Are they going to want it for all tables for all
> >>> databases, or not at all?
> >> We don't have a reason why the SQL-level row permissions should be toggled
> >> in global only. It it designed based on DAC policy, so it is quite natural
> >> to be controled by owner of resources.
> >> (However, it is not implemented yet.)
> > 
> > Yes, that was my question --- how will SQL-level row permissions be
> > controlled by the user.
> 
> When the given tuple has no ACL, it applies the default behavior which
> allows anything for public. This behavior intends to keep compatible
> works compared to the vanilla PostgreSQL.
> 
> We can have two state for "no ACLs". The first one is when tuples don't
> have fixed 4-bytes security attributes. It can be happen when PostgreSQL
> with SQL-level row-permissions mount compatible database files created
> by vanilla PostgreSQL. The other is the fixed 4-byte security attribute
> indicates an entry of "no ACLs". It seems unnecessary consumption, but
> we cannot determine whether the user tries to set ACLs when heap_form_tuple().
> 
> One considerable solution is to add an RowACL specific table option to
> disable row-level ACLs whole of the tables. It can be suitable for security
> design because the option is provided by the resource owner.
> 
> For example:
> 
>    CRATE TABLE t (
>        a int,
>        b text
>    ) WITH (row_acl=disable);
> 
> If the reloptions contains an information to determine whether a new tuple
> need the security field, or not, we can reflect it at heap_form_tuple().

What I am saying is for the default compile, SQL-level ACLs should be
possible because, since the ACL field has optional storage, there is no
downside to have it be available by default.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

22 November 2008, 04:39:07

Bruce Momjian wrote:
> KaiGai Kohei wrote:
>> Bruce Momjian wrote:
>>> KaiGai Kohei wrote:
>>>>>> Please consider SELinux/SE-PostgreSQL requires various kind of objects
>>>>>> (including database objects) to be labeled properly at the initial state.
>>>>>> If it allows clients to turn on row-level security feature, it means many
>>>>>> "unlabeled" tuples appear suddenly. In generally, these have to be labeled
>>>>>> before the system get being available.
>>>>> I was thinking more about people are using the SQL-level row
>>>>> permissions.  Are they going to want it for all tables for all
>>>>> databases, or not at all?
>>>> We don't have a reason why the SQL-level row permissions should be toggled
>>>> in global only. It it designed based on DAC policy, so it is quite natural
>>>> to be controled by owner of resources.
>>>> (However, it is not implemented yet.)
>>> Yes, that was my question --- how will SQL-level row permissions be
>>> controlled by the user.
>> When the given tuple has no ACL, it applies the default behavior which
>> allows anything for public. This behavior intends to keep compatible
>> works compared to the vanilla PostgreSQL.
>>
>> We can have two state for "no ACLs". The first one is when tuples don't
>> have fixed 4-bytes security attributes. It can be happen when PostgreSQL
>> with SQL-level row-permissions mount compatible database files created
>> by vanilla PostgreSQL. The other is the fixed 4-byte security attribute
>> indicates an entry of "no ACLs". It seems unnecessary consumption, but
>> we cannot determine whether the user tries to set ACLs when heap_form_tuple().
>>
>> One considerable solution is to add an RowACL specific table option to
>> disable row-level ACLs whole of the tables. It can be suitable for security
>> design because the option is provided by the resource owner.
>>
>> For example:
>>
>>    CRATE TABLE t (
>>        a int,
>>        b text
>>    ) WITH (row_acl=disable);
>>
>> If the reloptions contains an information to determine whether a new tuple
>> need the security field, or not, we can reflect it at heap_form_tuple().
> 
> What I am saying is for the default compile, SQL-level ACLs should be
> possible because, since the ACL field has optional storage, there is no
> downside to have it be available by default.

I think it is a possible and desirable desicion from the viewpoint of
security folks.

However, I think we have a few issues, and it makes unclear whether
we can make an agreement in the community.
The one is a cost of security hooks. They consume a bit more CPU steps
when a security mechanism is enabled. The other is prevention to override
a few hooks (ExecutorRun_hook and planner_hook), because they assume
standard implementations to be executed.

Which is more desirable option in the default?

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

22 November 2008, 08:45:32

On Fri, 2008-11-21 at 19:47 +0900, KaiGai Kohei wrote:

> In the result of "pgbench -i -s 10", the "sepostgresql_row_level=true"
> case consumed 152MB of storage, and the "sepostgresql_row_level=false"
> case consumed 148MB of storage.

Sounds good. It may not sound much to you, but it is all good news.

This allows us to include the feature in the main release package, which
was my main objective.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Bruce Momjian

Date:

22 November 2008, 11:59:57

KaiGai Kohei wrote:
> > What I am saying is for the default compile, SQL-level ACLs should be
> > possible because, since the ACL field has optional storage, there is no
> > downside to have it be available by default.
> 
> I think it is a possible and desirable desicion from the viewpoint of
> security folks.
> 
> However, I think we have a few issues, and it makes unclear whether
> we can make an agreement in the community.
> The one is a cost of security hooks. They consume a bit more CPU steps
> when a security mechanism is enabled. The other is prevention to override
> a few hooks (ExecutorRun_hook and planner_hook), because they assume
> standard implementations to be executed.
> 
> Which is more desirable option in the default?

Well, my assumption is that if a table doesn't have SQL-level row
permissions then there is no overhead becaues there are no permissions
to check.  If it does have  SQL-level row permissions then the user who
created the table has accepted the performance impact of SQL-level row
permissions.  I would think it would be pretty easy to see quickly if
any table in a query has SQL-level row permissions and then take the
performance hit.

For example, I might want to put SQL-level row permissions on an audit
table, but none of my other tables, and in that case I assume there is
only a performance impact on queries that use the audit table.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Tom Lane

Date:

22 November 2008, 16:42:37

KaiGai Kohei <kaigai@kaigai.gr.jp> writes:
> However, I think we have a few issues, and it makes unclear whether
> we can make an agreement in the community.
> The one is a cost of security hooks. They consume a bit more CPU steps
> when a security mechanism is enabled. The other is prevention to override
> a few hooks (ExecutorRun_hook and planner_hook), because they assume
> standard implementations to be executed.

I think your chances of taking those hooks away are zero.  It would
cripple a lot of other facilities that people are more interested in
than they are in SEPostgres.  In any case, the only way to use those
hooks is to load C code into the backend, and anyone who can do that
already has the keys to the kingdom.  I hope you are not suffering
from any illusions about being able to defend against arbitrary add-on
C code.
        regards, tom lane

Updates of SE-PostgreSQL 8.4devel patches (r1244)

From

KaiGai Kohei

Date:

24 November 2008, 11:58:48

I updated the patch set of SE-PostgreSQL (revision 1244).

[1/6] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1244.patch
[2/6] http://sepgsql.googlecode.com/files/sepostgresql-pg_dump-8.4devel-3-r1244.patch
[3/6] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1244.patch
[4/6] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1244.patch
[5/6] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1244.patch
[6/6] http://sepgsql.googlecode.com/files/sepostgresql-row_acl-8.4devel-3-r1244.patch

Draft of the SE-PostgreSQL documentation is here: http://wiki.postgresql.org/wiki/SEPostgreSQL

This revision contains some fixes required by some persons.
(Thanks for Simon, Bruce and Tom.)

List of updates:
- Rebase to the latest CVS HEAD.

- The fixed length security field of HeapTupleHeader becomes optimal. It enables enhanced security mechanism to control
itsallocation on heap_form_tuple(), and to reduce unnecessary storage consumption. The TupleDesc structure got a new
variableof "tdhassecurity". When it is true, heap_form_tuple() allocates an additional field to store security
identifier.The enhanced security mechanism can control value of the flag via a new hook: pgaceTupleDescHasSecurity().
 

- SE-PostgreSQL got a new GUC variable: "sepostgresql_row_level". When it turned off, SE-PostgreSQL does not apply its
row-levelaccess controls, and does not assign per-tuple security context.
 

- The following two hooks are removed:   * pgaceIsAllowPlannerHook()   * pgaceIsAllowExecutorRunHook() And, the
followinghook is added   * pgaceGramRelationOption() This hook gives a chance to handle relation options.
 

- The row-level acl got two new relation options:   * row_level_acl=on|off     When it is tuened off, the row-level
accesscontrols are     not applied, and security field is not allocated.   * default_row_acl='...'     It enables to
specifya default for newly inserted tuples.
 

- pg_security system catalog is added to the regression test of sanity_check.

- Code cleanups related to module installation checks.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

24 November 2008, 12:09:16

Tom Lane wrote:
> KaiGai Kohei <kaigai@kaigai.gr.jp> writes:
>> However, I think we have a few issues, and it makes unclear whether
>> we can make an agreement in the community.
>> The one is a cost of security hooks. They consume a bit more CPU steps
>> when a security mechanism is enabled. The other is prevention to override
>> a few hooks (ExecutorRun_hook and planner_hook), because they assume
>> standard implementations to be executed.
> 
> I think your chances of taking those hooks away are zero.  It would
> cripple a lot of other facilities that people are more interested in
> than they are in SEPostgres.  In any case, the only way to use those
> hooks is to load C code into the backend, and anyone who can do that
> already has the keys to the kingdom.  I hope you are not suffering
> from any illusions about being able to defend against arbitrary add-on
> C code.

I removed the two hooks at the r1244 patch set.
As you said, it is fundamentally danger to load uncertain binary modules.
Thus, what we should do is checks on module loading.

The default security policy requires loadable modules to be labeled as
'lib_t' type which means shared library files installed correctly.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

24 November 2008, 12:18:33

Bruce Momjian wrote:
> KaiGai Kohei wrote:
>>> What I am saying is for the default compile, SQL-level ACLs should be
>>> possible because, since the ACL field has optional storage, there is no
>>> downside to have it be available by default.
>> I think it is a possible and desirable desicion from the viewpoint of
>> security folks.
>>
>> However, I think we have a few issues, and it makes unclear whether
>> we can make an agreement in the community.
>> The one is a cost of security hooks. They consume a bit more CPU steps
>> when a security mechanism is enabled. The other is prevention to override
>> a few hooks (ExecutorRun_hook and planner_hook), because they assume
>> standard implementations to be executed.
>>
>> Which is more desirable option in the default?
> 
> Well, my assumption is that if a table doesn't have SQL-level row
> permissions then there is no overhead becaues there are no permissions
> to check.

When the binary is built with the SQL-level row permissions and scanned
table does not activated it, all it does is checking a flag variable
at Relation->rd_options. I guess it will be acceptable cost.

In this case, DBA disables row-level permission on the table, so
no additional security field is required.

> For example, I might want to put SQL-level row permissions on an audit
> table, but none of my other tables, and in that case I assume there is
> only a performance impact on queries that use the audit table.

It is not a zero, but tiny as far as we can ignore it in my opinion.

Thanks.
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Simon Riggs

Date:

25 November 2008, 22:59:51

On Mon, 2008-11-24 at 22:09 +0900, KaiGai Kohei wrote:

> I removed the two hooks at the r1244 patch set.
> As you said, it is fundamentally danger to load uncertain binary modules.
> Thus, what we should do is checks on module loading.
> 
> The default security policy requires loadable modules to be labeled as
> 'lib_t' type which means shared library files installed correctly.

We definitely want to include add-in modules with high security systems,
e.g. GIS and oracle compatibility functions.

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

KaiGai Kohei

Date:

25 November 2008, 23:45:44

Simon Riggs wrote:
> On Mon, 2008-11-24 at 22:09 +0900, KaiGai Kohei wrote:
> 
>> I removed the two hooks at the r1244 patch set.
>> As you said, it is fundamentally danger to load uncertain binary modules.
>> Thus, what we should do is checks on module loading.
>>
>> The default security policy requires loadable modules to be labeled as
>> 'lib_t' type which means shared library files installed correctly.
> 
> We definitely want to include add-in modules with high security systems,
> e.g. GIS and oracle compatibility functions.

Yes, it is possible.
SELinux assigns 'lib_t' type for modules stored in '/usr/lib/pgsql/' in default.

like:
[kaigai@saba ~]$ ls -Z /usr/lib/pgsql
-rwxr-xr-x  root root system_u:object_r:lib_t          ascii_and_mic.so
-rwxr-xr-x  root root system_u:object_r:lib_t          cyrillic_and_mic.so
-rwxr-xr-x  root root system_u:object_r:lib_t          dict_snowball.so
-rwxr-xr-x  root root system_u:object_r:lib_t          euc_cn_and_mic.so
-rwxr-xr-x  root root system_u:object_r:lib_t          euc_jis_2004_and_shift_jis_2004.so
-rwxr-xr-x  root root system_u:object_r:lib_t          euc_jp_and_sjis.so
-rwxr-xr-x  root root system_u:object_r:lib_t          euc_kr_and_mic.so    - snip -
(*) "-Z" option enables to show the security context of files.

SE-PostgreSQL does not prevent to load them. It means we want to allow to load library
files stored by database administrators properly, not a uncertain files.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1197)

From

Bruce Momjian

Date:

29 November 2008, 14:42:35

KaiGai Kohei wrote:
> Bruce Momjian wrote:
> > KaiGai Kohei wrote:
> >>> What I am saying is for the default compile, SQL-level ACLs should be
> >>> possible because, since the ACL field has optional storage, there is no
> >>> downside to have it be available by default.
> >> I think it is a possible and desirable desicion from the viewpoint of
> >> security folks.
> >>
> >> However, I think we have a few issues, and it makes unclear whether
> >> we can make an agreement in the community.
> >> The one is a cost of security hooks. They consume a bit more CPU steps
> >> when a security mechanism is enabled. The other is prevention to override
> >> a few hooks (ExecutorRun_hook and planner_hook), because they assume
> >> standard implementations to be executed.
> >>
> >> Which is more desirable option in the default?
> > 
> > Well, my assumption is that if a table doesn't have SQL-level row
> > permissions then there is no overhead becaues there are no permissions
> > to check.
> 
> When the binary is built with the SQL-level row permissions and scanned
> table does not activated it, all it does is checking a flag variable
> at Relation->rd_options. I guess it will be acceptable cost.
> 
> In this case, DBA disables row-level permission on the table, so
> no additional security field is required.
> 
> > For example, I might want to put SQL-level row permissions on an audit
> > table, but none of my other tables, and in that case I assume there is
> > only a performance impact on queries that use the audit table.
> 
> It is not a zero, but tiny as far as we can ignore it in my opinion.

Yes.  It is good to have SQL-level row security available by default in
every binary, even if it requires a few checks in C.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

02 December 2008, 05:00:14

I updated the patch set of SE-PostgreSQL (revision 1268).

[1/6] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1268.patch
[2/6] http://sepgsql.googlecode.com/files/sepostgresql-pg_dump-8.4devel-3-r1268.patch
[3/6] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1268.patch
[4/6] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1268.patch
[5/6] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1268.patch
[6/6] http://sepgsql.googlecode.com/files/sepostgresql-row_acl-8.4devel-3-r1268.patch

Draft of the SE-PostgreSQL documentation is here: http://wiki.postgresql.org/wiki/SEPostgreSQL

List of updates:
- The patches are rebased to the CVS HEAD.
- RelOptions related hooks are cleaned up.
- The Row-level ACL feature is chosen in default.
- rowacl_table_default() is added to show the default ACL of the table.
- The initial revision of regression test for Row-level ACL is added.

If you have anything to comment for the patches, could you disclose it?
It is not necessary to be a comprehensive one. Don't hesitate to submit.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

04 December 2008, 02:13:49

KaiGai Kohei wrote:
> I updated the patch set of SE-PostgreSQL (revision 1268).
> 
> [1/6] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1268.patch
> [2/6] http://sepgsql.googlecode.com/files/sepostgresql-pg_dump-8.4devel-3-r1268.patch
> [3/6] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1268.patch
> [4/6] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1268.patch
> [5/6] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1268.patch
> [6/6] http://sepgsql.googlecode.com/files/sepostgresql-row_acl-8.4devel-3-r1268.patch
> 
> Draft of the SE-PostgreSQL documentation is here:
>   http://wiki.postgresql.org/wiki/SEPostgreSQL
> 
> List of updates:
> - The patches are rebased to the CVS HEAD.
> - RelOptions related hooks are cleaned up.
> - The Row-level ACL feature is chosen in default.
> - rowacl_table_default() is added to show the default ACL of the table.
> - The initial revision of regression test for Row-level ACL is added.
> 
> If you have anything to comment for the patches, could you disclose it?
> It is not necessary to be a comprehensive one. Don't hesitate to submit.

I looked over the patch and was wondering why you chose to have a
configure option to disable row-level ACLs.  I assume that could just be
always enabled.  In fact, perhaps that is the first part of the patch
that should be applied because it doesn't rely on SE-Linux.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

04 December 2008, 04:02:13

Bruce Momjian wrote:
> KaiGai Kohei wrote:
>> I updated the patch set of SE-PostgreSQL (revision 1268).
>>
>> [1/6] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1268.patch
>> [2/6] http://sepgsql.googlecode.com/files/sepostgresql-pg_dump-8.4devel-3-r1268.patch
>> [3/6] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1268.patch
>> [4/6] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1268.patch
>> [5/6] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1268.patch
>> [6/6] http://sepgsql.googlecode.com/files/sepostgresql-row_acl-8.4devel-3-r1268.patch
>>
>> Draft of the SE-PostgreSQL documentation is here:
>>   http://wiki.postgresql.org/wiki/SEPostgreSQL
>>
>> List of updates:
>> - The patches are rebased to the CVS HEAD.
>> - RelOptions related hooks are cleaned up.
>> - The Row-level ACL feature is chosen in default.
>> - rowacl_table_default() is added to show the default ACL of the table.
>> - The initial revision of regression test for Row-level ACL is added.
>>
>> If you have anything to comment for the patches, could you disclose it?
>> It is not necessary to be a comprehensive one. Don't hesitate to submit.
> 
> I looked over the patch and was wondering why you chose to have a
> configure option to disable row-level ACLs.

There are no explicit reasons.
I thought it was natural, as if we can build Linux kernel without any
enhanced security features (SELinux, SMACK and so on).

I don't oppose to elimination of "--disable-row-acl" options, however,
it is not clear for me whether it should be unavoidable selection in
the future, or not.

> I assume that could just be always enabled.

It is not "always" enabled. When we build it with SE-PostgreSQL feature,
rest of enhanced security features (includes the row-level ACL) are
disabled automatically, as we discussed before.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

05 December 2008, 01:29:22

KaiGai Kohei wrote:
> >> If you have anything to comment for the patches, could you disclose it?
> >> It is not necessary to be a comprehensive one. Don't hesitate to submit.
> > 
> > I looked over the patch and was wondering why you chose to have a
> > configure option to disable row-level ACLs.
> 
> There are no explicit reasons.
> I thought it was natural, as if we can build Linux kernel without any
> enhanced security features (SELinux, SMACK and so on).
> 
> I don't oppose to elimination of "--disable-row-acl" options, however,
> it is not clear for me whether it should be unavoidable selection in
> the future, or not.

Look at the existing configure options;  we don't remove features via
configure unless it is for some platform-specific reason.  Please remove
the configure option and make it always enabled.  The way it should work
is that the feature is always enabled, and some SQL-level commands
should throw an appropriate error if SE-Linux is enabled in the build.

> > I assume that could just be always enabled.
> 
> It is not "always" enabled. When we build it with SE-PostgreSQL feature,
> rest of enhanced security features (includes the row-level ACL) are
> disabled automatically, as we discussed before.

Oh.  Is that because we use SE-Linux row-level security when
SE-PostgreSQL is enabled?  I guess that makes sense.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

05 December 2008, 02:33:38

>> I don't oppose to elimination of "--disable-row-acl" options, however,
>> it is not clear for me whether it should be unavoidable selection in
>> the future, or not.
> 
> Look at the existing configure options;  we don't remove features via
> configure unless it is for some platform-specific reason.  Please remove
> the configure option and make it always enabled.

OK, I'll update it in the next patch set.

>>> I assume that could just be always enabled.
>> It is not "always" enabled. When we build it with SE-PostgreSQL feature,
>> rest of enhanced security features (includes the row-level ACL) are
>> disabled automatically, as we discussed before.
> 
> Oh.  Is that because we use SE-Linux row-level security when
> SE-PostgreSQL is enabled?  I guess that makes sense.

Yes, when SE-PostgreSQL is enabled, it provides row-level security,
and the per-tuple security field is used to show its security context.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

05 December 2008, 02:41:24

KaiGai Kohei wrote:
> >> I don't oppose to elimination of "--disable-row-acl" options, however,
> >> it is not clear for me whether it should be unavoidable selection in
> >> the future, or not.
> > 
> > Look at the existing configure options;  we don't remove features via
> > configure unless it is for some platform-specific reason.  Please remove
> > the configure option and make it always enabled.
> 
> OK, I'll update it in the next patch set.

Good.  I assume the SQL-row security patch is not testable alone with
out the rest of the patches, right?

> >>> I assume that could just be always enabled.
> >> It is not "always" enabled. When we build it with SE-PostgreSQL feature,
> >> rest of enhanced security features (includes the row-level ACL) are
> >> disabled automatically, as we discussed before.
> > 
> > Oh.  Is that because we use SE-Linux row-level security when
> > SE-PostgreSQL is enabled?  I guess that makes sense.
> 
> Yes, when SE-PostgreSQL is enabled, it provides row-level security,
> and the per-tuple security field is used to show its security context.

Yes, that seems fine.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

05 December 2008, 02:51:12

Bruce Momjian wrote:
> KaiGai Kohei wrote:
>>>> I don't oppose to elimination of "--disable-row-acl" options, however,
>>>> it is not clear for me whether it should be unavoidable selection in
>>>> the future, or not.
>>> Look at the existing configure options;  we don't remove features via
>>> configure unless it is for some platform-specific reason.  Please remove
>>> the configure option and make it always enabled.
>> OK, I'll update it in the next patch set.
> 
> Good.  I assume the SQL-row security patch is not testable alone with
> out the rest of the patches, right?

The minimum requirements are the 1st and 2nd patches.
The first provides security hooks to PostgreSQL server program, and
the other provides ones to pg_dump command.
The 3rd, 4th and 5th are not necessary for the test purpose.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Updates of SE-PostgreSQL 8.4devel patches (r1280)

From

KaiGai Kohei

Date:

05 December 2008, 08:44:40

I updated the patch set of SE-PostgreSQL and related (r1280)

[1/6] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1280.patch
[2/6] http://sepgsql.googlecode.com/files/sepostgresql-pg_dump-8.4devel-3-r1280.patch
[3/6] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1280.patch
[4/6] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1280.patch
[5/6] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1280.patch
[6/6] http://sepgsql.googlecode.com/files/sepostgresql-row_acl-8.4devel-3-r1280.patch

Draft of the SE-PostgreSQL documentation is here: http://wiki.postgresql.org/wiki/SEPostgreSQL

List of updates:
- The patches are rebased to the current CVS HEAD
- '--disable-row-acl' is removed. This feature is implicitly enabled when we don't activate any other enhanced security
features.From this change, SECURITY_SYSATTR_NAME got being always defined, so some of unnecessary #ifdef ... #endif
blockhas gone.
 
- bugfix: it caused a crash on COPY when we specify a system attribute on FORCE QUATE clause.
- cleanup: some warnings to notice unused variables are eliminated.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

06 December 2008, 22:21:25

KaiGai Kohei wrote:
> Bruce Momjian wrote:
> > KaiGai Kohei wrote:
> >>>> I don't oppose to elimination of "--disable-row-acl" options, however,
> >>>> it is not clear for me whether it should be unavoidable selection in
> >>>> the future, or not.
> >>> Look at the existing configure options;  we don't remove features via
> >>> configure unless it is for some platform-specific reason.  Please remove
> >>> the configure option and make it always enabled.
> >> OK, I'll update it in the next patch set.
> > 
> > Good.  I assume the SQL-row security patch is not testable alone with
> > out the rest of the patches, right?
> 
> The minimum requirements are the 1st and 2nd patches.
> The first provides security hooks to PostgreSQL server program, and
> the other provides ones to pg_dump command.
> The 3rd, 4th and 5th are not necessary for the test purpose.

First, let me say you have done an amazing job of producing patches for
us, and your code quality is very high, especially considering the
complexity of this code and your newness to our development process. My
compliments to NEC, your employer.

Also, I personally am excited about this code and what it will add to
Postgres 8.4.

I hate to ask for something else from you, but I am trying to figure out
how we can proceed in reviewing and applying your additions.  I am
wondering if you can produce a patch that has the SE-Linux part separate
so I can review the non-SE-Linux parts of the patch alone --- right now
I am not 100% clear on what parts are always active as row-level SQL
security and what needs SE-Linux to operate.  I know this is an
additional burden on you and if it is too much to ask, please tell me.

Thanks.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

08 December 2008, 00:37:25

Bruce Momjian wrote:
> KaiGai Kohei wrote:
>> Bruce Momjian wrote:
>>> KaiGai Kohei wrote:
>>>>>> I don't oppose to elimination of "--disable-row-acl" options, however,
>>>>>> it is not clear for me whether it should be unavoidable selection in
>>>>>> the future, or not.
>>>>> Look at the existing configure options;  we don't remove features via
>>>>> configure unless it is for some platform-specific reason.  Please remove
>>>>> the configure option and make it always enabled.
>>>> OK, I'll update it in the next patch set.
>>> Good.  I assume the SQL-row security patch is not testable alone with
>>> out the rest of the patches, right?
>> The minimum requirements are the 1st and 2nd patches.
>> The first provides security hooks to PostgreSQL server program, and
>> the other provides ones to pg_dump command.
>> The 3rd, 4th and 5th are not necessary for the test purpose.
> 
> First, let me say you have done an amazing job of producing patches for
> us, and your code quality is very high, especially considering the
> complexity of this code and your newness to our development process. My
> compliments to NEC, your employer.
> 
> Also, I personally am excited about this code and what it will add to
> Postgres 8.4.
> 
> I hate to ask for something else from you, but I am trying to figure out
> how we can proceed in reviewing and applying your additions.  I am
> wondering if you can produce a patch that has the SE-Linux part separate
> so I can review the non-SE-Linux parts of the patch alone --- right now
> I am not 100% clear on what parts are always active as row-level SQL
> security and what needs SE-Linux to operate.  I know this is an
> additional burden on you and if it is too much to ask, please tell me.

All the SELinux specific part is stored within: - src/include/security/sepgsq.h - src/backend/security/sepgsql/* -
Blocksenclosed by "#if defined(HAVE_SELINUX)"   in src/include/security/pgace.h
 

SELinux related codes are never invoked without pgaceXXXX() hooks,
so you can simply ignore the above files/parts when you are under
the reviewing to non-SELinux parts.
Rest of changes are commonly needed to manage security attribute
and to inject security hooks.

In all honesty, I hesitate to separate the patch again into two
parts to be integrated later. I would be happy, if you suggested
it a half year ago, because this feature was suggested as two
separated patches in CommitFest:May. :(

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

08 December 2008, 00:54:24

KaiGai Kohei wrote:
> > I hate to ask for something else from you, but I am trying to figure out
> > how we can proceed in reviewing and applying your additions.  I am
> > wondering if you can produce a patch that has the SE-Linux part separate
> > so I can review the non-SE-Linux parts of the patch alone --- right now
> > I am not 100% clear on what parts are always active as row-level SQL
> > security and what needs SE-Linux to operate.  I know this is an
> > additional burden on you and if it is too much to ask, please tell me.
> 
> All the SELinux specific part is stored within:
>   - src/include/security/sepgsq.h
>   - src/backend/security/sepgsql/*
>   - Blocks enclosed by "#if defined(HAVE_SELINUX)"
>     in src/include/security/pgace.h
> 
> SELinux related codes are never invoked without pgaceXXXX() hooks,
> so you can simply ignore the above files/parts when you are under
> the reviewing to non-SELinux parts.
> Rest of changes are commonly needed to manage security attribute
> and to inject security hooks.
> 
> In all honesty, I hesitate to separate the patch again into two
> parts to be integrated later. I would be happy, if you suggested
> it a half year ago, because this feature was suggested as two
> separated patches in CommitFest:May. :(

Thanks, that's what I needed to know.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Tom Lane

Date:

08 December 2008, 15:05:35

KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
> Bruce Momjian wrote:
>> I assume that could just be always enabled.

> It is not "always" enabled. When we build it with SE-PostgreSQL feature,
> rest of enhanced security features (includes the row-level ACL) are
> disabled automatically, as we discussed before.

It seems like a pretty awful idea to have enabling sepostgres take away
a feature that exists in the default build.
        regards, tom lane

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

08 December 2008, 17:33:32

Tom Lane wrote:
> KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
>> Bruce Momjian wrote:
>>> I assume that could just be always enabled.
> 
>> It is not "always" enabled. When we build it with SE-PostgreSQL feature,
>> rest of enhanced security features (includes the row-level ACL) are
>> disabled automatically, as we discussed before.
> 
> It seems like a pretty awful idea to have enabling sepostgres take away
> a feature that exists in the default build.

Why?

The PGACE security framework allows one or no enhanced security
mechanism at most. It is quite natural that the default selection
is overrided when an alternative option is chosen explicitly.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Simon Riggs

Date:

08 December 2008, 18:02:56

On Tue, 2008-12-09 at 03:33 +0900, KaiGai Kohei wrote:
> Tom Lane wrote:
> > KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
> >> Bruce Momjian wrote:
> >>> I assume that could just be always enabled.
> > 
> >> It is not "always" enabled. When we build it with SE-PostgreSQL feature,
> >> rest of enhanced security features (includes the row-level ACL) are
> >> disabled automatically, as we discussed before.
> > 
> > It seems like a pretty awful idea to have enabling sepostgres take away
> > a feature that exists in the default build.
> 
> Why?
> 
> The PGACE security framework allows one or no enhanced security
> mechanism at most. It is quite natural that the default selection
> is overrided when an alternative option is chosen explicitly.

I'm finding these discussions very confusing to follow, sorry about
that.

We now have a parameter option that allows you to have row level
security in non-mandatory mode, which is good. But in order to get that
we need to build the server with a special configure option.

My previous objective was to remove the need for a configure option, so
we can enable row-level security in the default distribution of
Postgres. Are we going to enable that option in all normal distros? If
yes, why is it a configure option (at all)?

-- Simon Riggs           www.2ndQuadrant.comPostgreSQL Training, Services and Support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

09 December 2008, 00:27:04

Simon Riggs wrote:
> On Tue, 2008-12-09 at 03:33 +0900, KaiGai Kohei wrote:
>> Tom Lane wrote:
>>> KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
>>>> Bruce Momjian wrote:
>>>>> I assume that could just be always enabled.
>>>> It is not "always" enabled. When we build it with SE-PostgreSQL feature,
>>>> rest of enhanced security features (includes the row-level ACL) are
>>>> disabled automatically, as we discussed before.
>>> It seems like a pretty awful idea to have enabling sepostgres take away
>>> a feature that exists in the default build.
>> Why?
>>
>> The PGACE security framework allows one or no enhanced security
>> mechanism at most. It is quite natural that the default selection
>> is overrided when an alternative option is chosen explicitly.
> 
> I'm finding these discussions very confusing to follow, sorry about
> that.
> 
> We now have a parameter option that allows you to have row level
> security in non-mandatory mode, which is good. But in order to get that
> we need to build the server with a special configure option.

We need to distinguish a selection of enhanced security mechanism
and options provided by the mechanism chosen.

The PGACE security framework allows one or no enhanced security
mechanism at most when it is built. Thus, we have to determine
what mechanism to be activated. Currently, we have two enhanced
security mechanism for v8.4. The one is SE-PostgreSQL, and the
other is Row-level ACLs.

SE-PostgreSQL is a MAC-based feature which provides column/row
level granularity and collaboration with operating system.
It provides two GUC parameter options as follows:- sepostgresql = (default|enforcing|permissive|disabled)-
sepostgresql_row_level= (on|off)

Row-level ACLs is a DAC-based feature which provides row level
granularity and works independently from operating system security.
It provides two table options as follows:- row_level_acl = (on|off)- default_row_acl = '<ACL text>'

> My previous objective was to remove the need for a configure option, so
> we can enable row-level security in the default distribution of
> Postgres. Are we going to enable that option in all normal distros? If
> yes, why is it a configure option (at all)?

The purpose of configure options is to choose a enhanced security
mechanism implemented on the PGACE security framework.
It is unclear for me what means the "row-level security" in this
context. Is it provided by SE-PostgreSQL? Row-level ACLs?

We currently have no option to disable all the enhanced security
mechanism, so one mechanism has to be choosen at least.
Both of the two mechanisms being available now provide row-level
granularity, so "row-level security" will be always enabled *in the
result*. Please note that it can be provided by different security
mechanism which is designed with independent security model.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

10 December 2008, 10:13:44

Simon Riggs wrote:
> 
> On Tue, 2008-12-09 at 03:33 +0900, KaiGai Kohei wrote:
> > Tom Lane wrote:
> > > KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
> > >> Bruce Momjian wrote:
> > >>> I assume that could just be always enabled.
> > > 
> > >> It is not "always" enabled. When we build it with SE-PostgreSQL feature,
> > >> rest of enhanced security features (includes the row-level ACL) are
> > >> disabled automatically, as we discussed before.
> > > 
> > > It seems like a pretty awful idea to have enabling sepostgres take away
> > > a feature that exists in the default build.
> > 
> > Why?
> > 
> > The PGACE security framework allows one or no enhanced security
> > mechanism at most. It is quite natural that the default selection
> > is overrided when an alternative option is chosen explicitly.
> 
> I'm finding these discussions very confusing to follow, sorry about
> that.

It isn't you;  it _is_ hard to follow.  ;-)  No one is at fault;  the
topic is just complex.

> We now have a parameter option that allows you to have row level
> security in non-mandatory mode, which is good. But in order to get that
> we need to build the server with a special configure option.

No, that was removed at my request so it is always available.

> My previous objective was to remove the need for a configure option, so
> we can enable row-level security in the default distribution of
> Postgres. Are we going to enable that option in all normal distros? If
> yes, why is it a configure option (at all)?

Option is gone in the most recent patch and it is always enabled.  We
still have a configure option to enable SE-Linux features.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

10 December 2008, 10:16:25

Tom Lane wrote:
> KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
> > Bruce Momjian wrote:
> >> I assume that could just be always enabled.
> 
> > It is not "always" enabled. When we build it with SE-PostgreSQL feature,
> > rest of enhanced security features (includes the row-level ACL) are
> > disabled automatically, as we discussed before.
> 
> It seems like a pretty awful idea to have enabling sepostgres take away
> a feature that exists in the default build.

Agreed.  The problem is that the security column used for SQL-level row
security is reused to hold the SE-Linux ACL when SE-Linux is enabled.  I
suppose the only way to enable them both in an SE-Linux build would be
to use a new optional column for SE-Linux and keep the SQL-level row
security optional column unchanged.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

10 December 2008, 11:13:29

Bruce Momjian wrote:
> Tom Lane wrote:
>> KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
>>> Bruce Momjian wrote:
>>>> I assume that could just be always enabled.
>>> It is not "always" enabled. When we build it with SE-PostgreSQL feature,
>>> rest of enhanced security features (includes the row-level ACL) are
>>> disabled automatically, as we discussed before.
>> It seems like a pretty awful idea to have enabling sepostgres take away
>> a feature that exists in the default build.
> 
> Agreed.

I don't agree. What is the reason why? It has been unclear for me.

The PGACE security framework is designed to allow users to choose
an enhanced security mechanism from some of provided options.
(Currently, we have sepgsql and rowacl.)
It is quite natural that one is disabled when the other is enabled.

If a specific enhanced security mechanism has a privileged position,
it should not be a guest of the security framwork, and be hardcoded
like existing table-level database ACLs.

Again, I don't oppose the Row-level ACLs to be the default selection.
However, it should be a selectable option.

Thanks,

> The problem is that the security column used for SQL-level row
> security is reused to hold the SE-Linux ACL when SE-Linux is enabled.  I
> suppose the only way to enable them both in an SE-Linux build would be
> to use a new optional column for SE-Linux and keep the SQL-level row
> security optional column unchanged.
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Greg Stark

Date:

10 December 2008, 11:20:39

Is there any reason it's easier to do as a configure option instead of  
an initdb option or better yet a per-database option?

The reason we're shying away from configure options for functionality  
changes is that more and more users are getting pistgres from binary  
distributions. Which option should redhat ship for example?

Also, a configure option is kind if weird since that's a property of  
the binary not the data: what happens if you build a database with  
selinux and the switch binaries to a non-we build? Is that option in  
pg_control?

-- 
Greg


On 10 Dec 2008, at 12:13, KaiGai Kohei <kaigai@kaigai.gr.jp> wrote:

> Bruce Momjian wrote:
>> Tom Lane wrote:
>>> KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
>>>> Bruce Momjian wrote:
>>>>> I assume that could just be always enabled.
>>>> It is not "always" enabled. When we build it with SE-PostgreSQL  
>>>> feature,
>>>> rest of enhanced security features (includes the row-level ACL) are
>>>> disabled automatically, as we discussed before.
>>> It seems like a pretty awful idea to have enabling sepostgres take  
>>> away
>>> a feature that exists in the default build.
>> Agreed.
>
> I don't agree. What is the reason why? It has been unclear for me.
>
> The PGACE security framework is designed to allow users to choose
> an enhanced security mechanism from some of provided options.
> (Currently, we have sepgsql and rowacl.)
> It is quite natural that one is disabled when the other is enabled.
>
> If a specific enhanced security mechanism has a privileged position,
> it should not be a guest of the security framwork, and be hardcoded
> like existing table-level database ACLs.
>
> Again, I don't oppose the Row-level ACLs to be the default selection.
> However, it should be a selectable option.
>
> Thanks,
>
>> The problem is that the security column used for SQL-level row
>> security is reused to hold the SE-Linux ACL when SE-Linux is  
>> enabled.  I
>> suppose the only way to enable them both in an SE-Linux build would  
>> be
>> to use a new optional column for SE-Linux and keep the SQL-level row
>> security optional column unchanged.
> -- 
> KaiGai Kohei <kaigai@kaigai.gr.jp>
>
> -- 
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

10 December 2008, 11:28:35

KaiGai Kohei wrote:
> Bruce Momjian wrote:
> > Tom Lane wrote:
> >> KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
> >>> Bruce Momjian wrote:
> >>>> I assume that could just be always enabled.
> >>> It is not "always" enabled. When we build it with SE-PostgreSQL feature,
> >>> rest of enhanced security features (includes the row-level ACL) are
> >>> disabled automatically, as we discussed before.
> >> It seems like a pretty awful idea to have enabling sepostgres take away
> >> a feature that exists in the default build.
> > 
> > Agreed.
> 
> I don't agree. What is the reason why? It has been unclear for me.
> 
> The PGACE security framework is designed to allow users to choose
> an enhanced security mechanism from some of provided options.
> (Currently, we have sepgsql and rowacl.)
> It is quite natural that one is disabled when the other is enabled.
> 
> If a specific enhanced security mechanism has a privileged position,
> it should not be a guest of the security framwork, and be hardcoded
> like existing table-level database ACLs.
> 
> Again, I don't oppose the Row-level ACLs to be the default selection.
> However, it should be a selectable option.

I understand, but imagine how this is going to interact for users.  What
happens if I install an SE-Linux binary and point it at a /data
directory that was not created by SE-Linxu binary.  How is the SE-Linux
binary going to interpret the security field?  What happens if I load a
non-SE-Linux data dump into a SE-Linux binary?  Do I lose my security
settings?

I am starting to think we should have two optional security fields, one
for SQL and one for SE-Linux. The big downside of that is that we are
back to the case of the having lots of SE-Linux-specific code to handle
that SE-Linux field, rather than reusing the SQL-row-level security
field.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

10 December 2008, 11:34:45

Greg Stark wrote:
> Is there any reason it's easier to do as a configure option instead of 
> an initdb option or better yet a per-database option?

SE-PostgreSQL needs "libselinux" to communicate with SELinux,
however, it is not available for non-SELinux'ed platforms.

> The reason we're shying away from configure options for functionality 
> changes is that more and more users are getting pistgres from binary 
> distributions. Which option should redhat ship for example?

As we're doing on Fedora, SE-PostgreSQL will be distributed as
a separated package.
If a user want to start SE- version, he will install another
binary package, and stop vanilla PostgreSQl, and start SE-PostgreSQL.

> Also, a configure option is kind if weird since that's a property of the 
> binary not the data: what happens if you build a database with selinux 
> and the switch binaries to a non-we build? Is that option in pg_control?

In this case, the vanilla PostgreSQL should be simply ignore the
data which is generated by SE-PostgreSQL.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Peter Eisentraut

Date:

10 December 2008, 11:42:26

KaiGai Kohei wrote:
> I don't agree. What is the reason why? It has been unclear for me.
> 
> The PGACE security framework is designed to allow users to choose
> an enhanced security mechanism from some of provided options.
> (Currently, we have sepgsql and rowacl.)
> It is quite natural that one is disabled when the other is enabled.

As a general rule, mutually exclusive features as compile-time option 
should be avoided at all costs.  Since most people use binary packages, 
forcing the packager to make such a choice will always make a lot of 
people unhappy, or alternatively cause one of the features to bitrot.

As a secondary rule, mutually exclusive features should be avoided at 
all, without a compelling reason.  I don't see such a reason here.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

10 December 2008, 11:48:08

Peter Eisentraut wrote:
> KaiGai Kohei wrote:
> > I don't agree. What is the reason why? It has been unclear for me.
> > 
> > The PGACE security framework is designed to allow users to choose
> > an enhanced security mechanism from some of provided options.
> > (Currently, we have sepgsql and rowacl.)
> > It is quite natural that one is disabled when the other is enabled.
> 
> As a general rule, mutually exclusive features as compile-time option 
> should be avoided at all costs.  Since most people use binary packages, 
> forcing the packager to make such a choice will always make a lot of 
> people unhappy, or alternatively cause one of the features to bitrot.
> 
> As a secondary rule, mutually exclusive features should be avoided at 
> all, without a compelling reason.  I don't see such a reason here.

I think there is a reason to have SE-Linux be compile-time because there
is no way to know at run time if the OS has the SE-Linux libraries,
right?  I assume this is similar to how we do LDAP.

But your larger point is that SQL-row-level security should always be
available, which I just posted about.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

10 December 2008, 11:55:15

Bruce Momjian wrote:
> KaiGai Kohei wrote:
>> Bruce Momjian wrote:
>>> Tom Lane wrote:
>>>> KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
>>>>> Bruce Momjian wrote:
>>>>>> I assume that could just be always enabled.
>>>>> It is not "always" enabled. When we build it with SE-PostgreSQL feature,
>>>>> rest of enhanced security features (includes the row-level ACL) are
>>>>> disabled automatically, as we discussed before.
>>>> It seems like a pretty awful idea to have enabling sepostgres take away
>>>> a feature that exists in the default build.
>>> Agreed.
>> I don't agree. What is the reason why? It has been unclear for me.
>>
>> The PGACE security framework is designed to allow users to choose
>> an enhanced security mechanism from some of provided options.
>> (Currently, we have sepgsql and rowacl.)
>> It is quite natural that one is disabled when the other is enabled.
>>
>> If a specific enhanced security mechanism has a privileged position,
>> it should not be a guest of the security framwork, and be hardcoded
>> like existing table-level database ACLs.
>>
>> Again, I don't oppose the Row-level ACLs to be the default selection.
>> However, it should be a selectable option.
> 
> I understand, but imagine how this is going to interact for users.  What
> happens if I install an SE-Linux binary and point it at a /data
> directory that was not created by SE-Linxu binary.  How is the SE-Linux
> binary going to interpret the security field?

When SE-PostgreSQL binary fetch a tuple without its security attribute,
it considers the tuple has an alternative one called as "unlabeled_t".
This behavior is same as when we mount an unlabled filesystem on SELinux
system.
> What happens if I load a non-SE-Linux data dump into a SE-Linux binary?> Do I lose my security settings?

It is same as normal INSERT case. When user gives a data without specific
security context, SE-PostgreSQL assigns it a default security context.
In the default security policy, it is "sepgsql_table_t".

> I am starting to think we should have two optional security fields, one
> for SQL and one for SE-Linux. The big downside of that is that we are
> back to the case of the having lots of SE-Linux-specific code to handle
> that SE-Linux field, rather than reusing the SQL-row-level security
> field.

It is just an idea. If Row-level ACL feature is *hardcoded* (not a guest
of PGACE), is it considerable a hidden attribute typed as "aclitem[]"?

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

10 December 2008, 12:36:23

Bruce Momjian wrote:
> Peter Eisentraut wrote:
>> KaiGai Kohei wrote:
>>> I don't agree. What is the reason why? It has been unclear for me.
>>>
>>> The PGACE security framework is designed to allow users to choose
>>> an enhanced security mechanism from some of provided options.
>>> (Currently, we have sepgsql and rowacl.)
>>> It is quite natural that one is disabled when the other is enabled.
>> As a general rule, mutually exclusive features as compile-time option 
>> should be avoided at all costs.  Since most people use binary packages, 
>> forcing the packager to make such a choice will always make a lot of 
>> people unhappy, or alternatively cause one of the features to bitrot.
>>
>> As a secondary rule, mutually exclusive features should be avoided at 
>> all, without a compelling reason.  I don't see such a reason here.
> 
> I think there is a reason to have SE-Linux be compile-time because there
> is no way to know at run time if the OS has the SE-Linux libraries,
> right?  I assume this is similar to how we do LDAP.

Yes, the libselinux is a factor it to be a compile-time option.

> But your larger point is that SQL-row-level security should always be
> available, which I just posted about.

If so, it should be hardcoded on somewhere, no need to be implemented
as a guest of PGACE security framework. Its purpose is to implement
enhanced security mechanisms with minimum impact to core facilities.

If you intend to implement is as a hardcoded feature, I can agree.
Please wait for a few days, I'll try to implement it.
So, ignore the 6th patch during the days and make progress to review
the rest of patches.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

10 December 2008, 18:04:30

KaiGai Kohei wrote:
> > But your larger point is that SQL-row-level security should always be
> > available, which I just posted about.
> 
> If so, it should be hardcoded on somewhere, no need to be implemented
> as a guest of PGACE security framework. Its purpose is to implement
> enhanced security mechanisms with minimum impact to core facilities.
> 
> If you intend to implement is as a hardcoded feature, I can agree.
> Please wait for a few days, I'll try to implement it.
> So, ignore the 6th patch during the days and make progress to review
> the rest of patches.

Hold, please don't code yet.  You might have already read our
developer's FAQ:
http://wiki.postgresql.org/wiki/Developer_FAQ#What_do_I_do_after_choosing_an_item_to_work_on.3F

but I feel we need to follow part of it to prevent you from repeatedly
having to adjust your patch, which I think has happened in the past.

Let's decide exactly what configure options, GUC options, system catalog
changes, and user-visible SQL command syntax we are going to use for the
patch before you recode anything.

For example, what configure flags are we going to have?  I assume just
'--enable-selinux', right?

Second, system catalogs; are you going to have separate columns for
SQL-level row security and SE-Linux security?  If so, is the SE-Linux
column only going to be created by the --enable-selinux build?  If so,
that is going to mean that the /data directory is going to be affected
by the --enable-selinux flag, which is not ideal --- it would be good if
someone could switch to an SE-Linux binary without to dump/reload.  One
nice idea would be to have one "security" column and allow both
SQL-level and SE-Linux security values to be stored in the column,
perhaps at the same time;  that way there is no new column needed for
SE-Linux.

I know I suggested the security column act like the system oid column,
but now I am thinking I was wrong.  The system oid column stores an
auto-generated value so it was necessary to have it be specified at
CREATE TABLE time because it is guaranteed to take storage space.  For
the security column, in most cases it will be null, meaning it would not
take any storage space becuase we use a null bitmap for null column
values.  So, perhaps the security column should be in every table and we
can just make it default to null;  I think that would give us much more
flexibility to add security to existing tables.

Finally, I am wondering what other things should be adjusted that I have
not thought of yet.  I would like to make KaiGai's job as easy as
possible.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

10 December 2008, 23:48:29

Bruce Momjian wrote:
> Second, system catalogs; are you going to have separate columns for
> SQL-level row security and SE-Linux security?  If so, is the SE-Linux
> column only going to be created by the --enable-selinux build?  If so,
> that is going to mean that the /data directory is going to be affected
> by the --enable-selinux flag, which is not ideal --- it would be good if
> someone could switch to an SE-Linux binary without to dump/reload.  One
> nice idea would be to have one "security" column and allow both
> SQL-level and SE-Linux security values to be stored in the column,
> perhaps at the same time;  that way there is no new column needed for
> SE-Linux.
> 
> I know I suggested the security column act like the system oid column,
> but now I am thinking I was wrong.  The system oid column stores an
> auto-generated value so it was necessary to have it be specified at
> CREATE TABLE time because it is guaranteed to take storage space.  For
> the security column, in most cases it will be null, meaning it would not
> take any storage space becuase we use a null bitmap for null column
> values.  So, perhaps the security column should be in every table and we
> can just make it default to null;  I think that would give us much more
> flexibility to add security to existing tables.

Sorry, I am confusing system columns, like pg_class.relname, with
user-table system columns, like mytable.xmin.  I don't think we have the
ability of having user-table system columns nullable, right?

We do have a per-row HEAP_HASOID bit, so I wonder if we can have a
HEAP_HASSEC bit too.  Right now the HEAP_HASOID is controlled by the
CREATE/ALTER table;  I am unclear how hard it would be to allow it to be
controlled via INSERT, meaning it would be present only if the row had a
SEC value supplied.

Here is our OID column on some rows but not others:
test=> CREATE TABLE test (x INT) WITH oids;CREATE TABLEtest=> INSERT INTO test VALUES (1);INSERT 16392 1test=> ALTER
TABLEtest SET WITHOUT OIDS;ALTER TABLEtest=> INSERT INTO test VALUES(2);INSERT 0 1test=> SELECT oid, * FROM test;ERROR:
COLUMN "oid" does not existLINE 1: SELECT oid, * FROM test;

but it has per-table granularity;  the oid value for '1' is still
stored, but is no longer visible;  that would not work for the security
value.

The complexity is that the column would always exist (unlike OID), but
would be NULL if not assigned to.  This would allow any table to have
security by default, and have no overhead if security is not defined for
the row.  Now, I realize SE-Linux would have security defined for each
row, but SQL-level row security might not, and this would allow the
feature to be very useful, perhaps with a GUC that required security for
SE-Linux, and /data would be the same for SE-Linux and non-SE-Linux.  If
we could make the row behave like this we might use separate columns for
SQL-level and SE-Linux security because there would be no storage
overhead unless security was used.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

11 December 2008, 00:07:01

Bruce Momjian wrote:
> KaiGai Kohei wrote:
>>> But your larger point is that SQL-row-level security should always be
>>> available, which I just posted about.
>> If so, it should be hardcoded on somewhere, no need to be implemented
>> as a guest of PGACE security framework. Its purpose is to implement
>> enhanced security mechanisms with minimum impact to core facilities.
>>
>> If you intend to implement is as a hardcoded feature, I can agree.
>> Please wait for a few days, I'll try to implement it.
>> So, ignore the 6th patch during the days and make progress to review
>> the rest of patches.
> 
> Hold, please don't code yet.  You might have already read our
> developer's FAQ:
> 
>     http://wiki.postgresql.org/wiki/Developer_FAQ#What_do_I_do_after_choosing_an_item_to_work_on.3F
> 
> but I feel we need to follow part of it to prevent you from repeatedly
> having to adjust your patch, which I think has happened in the past.
> 
> Let's decide exactly what configure options, GUC options, system catalog
> changes, and user-visible SQL command syntax we are going to use for the
> patch before you recode anything.

The guest of PGACE security framework should be chosen by configure option.
It also means any other facilities except for the guest of PGACE can be
enabled/disabled by other kind of options.

If the Row-level ACLs should be always enabled (independent from SE-PostgreSQL),
I think it should be hardcoded outside of PGACE, and enabled/disabled by
any other way. Table option is a candidate, like:
  CREATE TABLE t (      a   int,      b   text  ) WITH (ROW_LEVEL_ACL=ON);

> For example, what configure flags are we going to have?  I assume just
> '--enable-selinux', right?

Yes, currently.

> Second, system catalogs; are you going to have separate columns for
> SQL-level row security and SE-Linux security?

Yes, I think these different properties are stored within separate
columns, as if a file has both of permission masks (-rwxr-x---) and
security context of SELinux.

> If so, is the SE-Linux
> column only going to be created by the --enable-selinux build?  If so,
> that is going to mean that the /data directory is going to be affected
> by the --enable-selinux flag, which is not ideal --- it would be good if
> someone could switch to an SE-Linux binary without to dump/reload.  One
> nice idea would be to have one "security" column and allow both
> SQL-level and SE-Linux security values to be stored in the column,
> perhaps at the same time;  that way there is no new column needed for
> SE-Linux.

As you noticed, a "conditional system column" can be open to discuss.
If someone switch his binary to SE- version, the tables already defined
in $PGDATA don't have "security_context" system column. SE-PostgreSQL
handles tuples stored within the table as "unlabeled" one, but it does
not allow users to access the attribute due to lack of proper system
column.

In addition, we may have to refer security context of tuples via
"tuple_acl" system column, when someone switch his binary from
the Row-level ACL to SE-PostgreSQL. :)

The naming is a difficult matter. It seems to me "security" is too
general term. How do you think "security_label" or "security_attribute"?

> I know I suggested the security column act like the system oid column,
> but now I am thinking I was wrong.  The system oid column stores an
> auto-generated value so it was necessary to have it be specified at
> CREATE TABLE time because it is guaranteed to take storage space.  For
> the security column, in most cases it will be null, meaning it would not
> take any storage space becuase we use a null bitmap for null column
> values.  So, perhaps the security column should be in every table and we
> can just make it default to null;  I think that would give us much more
> flexibility to add security to existing tables.

Here is a differences in frequency of valid value in security column
between SE-PostgreSQL and Row-level ACLs.
SE-PostgreSQL requires all the tuple should be labeled properly.
(And, I guess any other upcoming feature to support another secure OS
has similar requirement.)
But we assume the Row-level ACLs is enabled on the limited number of
tables, so it need to specify an option to activate on CREATE TABLE.

Please note that I distinguish between "security column" and "acl column"
here. In my opinion, the security column should be a system column, but
the acl column can be implemented as a regular column that is appended
depending on user's needs.

When we represent a NULL of "security column", it requires HEAP_HASSECURITY
to be set on t_infomask. Here is no additional storage consumption.
If it is represented as NULL bitmaps, it requires a tuple has its null bitmaps
which need 4-bytes at least, so it can make additional storage consumption.
Thus, NULL-bitmap approach is not suitable to represent "security column"
because it has to be defined for any tables. But, it can be suitable for
"acl column".

The following proposal is my idea:

- The Row-level ACLs is implemented as a hardcoded feature, not a guest of  PGACE security framework.
- It can be enabled/disabled via table options as:      CREATE TABLE t (          a   int,          b   text      )
WITH(ROW_LEVEL_ACL=ON);

- When the Row-level ACLs is enabled on CREATE TABLE, it implicitly add  a column to represent Row-level ACLs. This
columnis defined as aclitem[].

- The "acl column" is not expanded via "SELECT * FROM ...", but we can specify  it explicitly like "SELECT tuple_acl, *
FROM..." as system column doing.

- The "acl column" is allowed to update by the table owner or superuser.
- If a table has Row-level ACLs enabled, ExecScan() checks visibility of  fetched tuples, and ignore violated tuples.

> Finally, I am wondering what other things should be adjusted that I have
> not thought of yet.  I would like to make KaiGai's job as easy as
> possible.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

11 December 2008, 00:29:43

Bruce Momjian wrote:
> Bruce Momjian wrote:
>> Second, system catalogs; are you going to have separate columns for
>> SQL-level row security and SE-Linux security?  If so, is the SE-Linux
>> column only going to be created by the --enable-selinux build?  If so,
>> that is going to mean that the /data directory is going to be affected
>> by the --enable-selinux flag, which is not ideal --- it would be good if
>> someone could switch to an SE-Linux binary without to dump/reload.  One
>> nice idea would be to have one "security" column and allow both
>> SQL-level and SE-Linux security values to be stored in the column,
>> perhaps at the same time;  that way there is no new column needed for
>> SE-Linux.
>>
>> I know I suggested the security column act like the system oid column,
>> but now I am thinking I was wrong.  The system oid column stores an
>> auto-generated value so it was necessary to have it be specified at
>> CREATE TABLE time because it is guaranteed to take storage space.  For
>> the security column, in most cases it will be null, meaning it would not
>> take any storage space becuase we use a null bitmap for null column
>> values.  So, perhaps the security column should be in every table and we
>> can just make it default to null;  I think that would give us much more
>> flexibility to add security to existing tables.
> 
> Sorry, I am confusing system columns, like pg_class.relname, with
> user-table system columns, like mytable.xmin.  I don't think we have the
> ability of having user-table system columns nullable, right?

Currently, it assumes system columns should not be NULL.
heap_getsysattr() always set the given 'isnull' 'false'.

> We do have a per-row HEAP_HASOID bit, so I wonder if we can have a
> HEAP_HASSEC bit too.  Right now the HEAP_HASOID is controlled by the
> CREATE/ALTER table;

The current patch add HEAP_HASSECURITY bit to t_infomask. :-)
When it is false, its security field is not available and not allocated.
> I am unclear how hard it would be to allow it to be> controlled via INSERT, meaning it would be present only if the
rowhad a> SEC value supplied.
 

It is impossible, and not suitable for SE-PostgreSQL.
The HeapTuple is allocated prior to fetch INSERT'ed value.
In addition, SE-PostgreSQL assigns its default security context when no
security attribute is given.

> Here is our OID column on some rows but not others:
> 
>     test=> CREATE TABLE test (x INT) WITH oids;
>     CREATE TABLE
>     test=> INSERT INTO test VALUES (1);
>     INSERT 16392 1
>     test=> ALTER TABLE test SET WITHOUT OIDS;
>     ALTER TABLE
>     test=> INSERT INTO test VALUES(2);
>     INSERT 0 1
>     test=> SELECT oid, * FROM test;
>     ERROR:  COLUMN "oid" does not exist
>     LINE 1: SELECT oid, * FROM test;
> 
> but it has per-table granularity;  the oid value for '1' is still
> stored, but is no longer visible;  that would not work for the security
> value.

If we can control the Row-level ACLs via table options, what should be
happen when we turn off the feature?
IMO, it is natural to ignore Row-level ACLs independent from whether
the stored tuple actually has ACLs, or not.

> The complexity is that the column would always exist (unlike OID), but
> would be NULL if not assigned to.  This would allow any table to have
> security by default, and have no overhead if security is not defined for
> the row.  Now, I realize SE-Linux would have security defined for each
> row, but SQL-level row security might not, and this would allow the
> feature to be very useful, perhaps with a GUC that required security for
> SE-Linux, and /data would be the same for SE-Linux and non-SE-Linux.  If
> we could make the row behave like this we might use separate columns for
> SQL-level and SE-Linux security because there would be no storage
> overhead unless security was used.

I basically agree for the idea of separate columns.
My preference is "acl column" as general column (not a system column)
which is defined at limited number of tables with ROW_LEVEL_ACL=ON.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

11 December 2008, 01:44:48

KaiGai Kohei wrote:
> > Let's decide exactly what configure options, GUC options, system catalog
> > changes, and user-visible SQL command syntax we are going to use for the
> > patch before you recode anything.
> 
> The guest of PGACE security framework should be chosen by configure option.
> It also means any other facilities except for the guest of PGACE can be
> enabled/disabled by other kind of options.
> 
> If the Row-level ACLs should be always enabled (independent from SE-PostgreSQL),
> I think it should be hardcoded outside of PGACE, and enabled/disabled by
> any other way. Table option is a candidate, like:
> 
>    CREATE TABLE t (
>        a   int,
>        b   text
>    ) WITH (ROW_LEVEL_ACL=ON);

As I mentioned below, this might not be necessary, meaning that row
security is enabled for rows where you set the row security value and
does not need to be turned on at create table time.

> > Second, system catalogs; are you going to have separate columns for
> > SQL-level row security and SE-Linux security?
> 
> Yes, I think these different properties are stored within separate
> columns, as if a file has both of permission masks (-rwxr-x---) and
> security context of SELinux.

OK.

> > If so, is the SE-Linux
> > column only going to be created by the --enable-selinux build?  If so,
> > that is going to mean that the /data directory is going to be affected
> > by the --enable-selinux flag, which is not ideal --- it would be good if
> > someone could switch to an SE-Linux binary without to dump/reload.  One
> > nice idea would be to have one "security" column and allow both
> > SQL-level and SE-Linux security values to be stored in the column,
> > perhaps at the same time;  that way there is no new column needed for
> > SE-Linux.
> 
> As you noticed, a "conditional system column" can be open to discuss.
> If someone switch his binary to SE- version, the tables already defined
> in $PGDATA don't have "security_context" system column. SE-PostgreSQL
> handles tuples stored within the table as "unlabeled" one, but it does
> not allow users to access the attribute due to lack of proper system
> column.

Right, if the "security_context" always exists, but is null, moving to
SE-Linux would allow them to add security without dump/reload.

> In addition, we may have to refer security context of tuples via
> "tuple_acl" system column, when someone switch his binary from
> the Row-level ACL to SE-PostgreSQL. :)
> 
> The naming is a difficult matter. It seems to me "security" is too
> general term. How do you think "security_label" or "security_attribute"?

So "security_context" has to be the PGACE column name.  I would say
security_acl or just secacl because I think it is going to match our
existing ACL system pretty closely.

> > I know I suggested the security column act like the system oid column,
> > but now I am thinking I was wrong.  The system oid column stores an
> > auto-generated value so it was necessary to have it be specified at
> > CREATE TABLE time because it is guaranteed to take storage space.  For
> > the security column, in most cases it will be null, meaning it would not
> > take any storage space because we use a null bitmap for null column
> > values.  So, perhaps the security column should be in every table and we
> > can just make it default to null;  I think that would give us much more
> > flexibility to add security to existing tables.
> 
> Here is a differences in frequency of valid value in security column
> between SE-PostgreSQL and Row-level ACLs.
> SE-PostgreSQL requires all the tuple should be labeled properly.
> (And, I guess any other upcoming feature to support another secure OS
> has similar requirement.)
> But we assume the Row-level ACLs is enabled on the limited number of
> tables, so it need to specify an option to activate on CREATE TABLE.
> 
> Please note that I distinguish between "security column" and "acl column"
> here. In my opinion, the security column should be a system column, but
> the acl column can be implemented as a regular column that is appended
> depending on user's needs.

If we can make the column not take any space if it is null then we can
have both columns always present and that way /data is the same for
SE-Linux and non-SE-Linux.  We do have columns like xmin which don't
show in SELECT * and I don't see a problem adding two more.

I imagine we would need to modify COPY and pg_dump to specify if we are
to dump/load secacl; I see you already did "security_context".

> When we represent a NULL of "security column", it requires HEAP_HASSECURITY
> to be set on t_infomask. Here is no additional storage consumption.

Nice!

> If it is represented as NULL bitmaps, it requires a tuple has its null bitmaps
> which need 4-bytes at least, so it can make additional storage consumption.
> Thus, NULL-bitmap approach is not suitable to represent "security column"
> because it has to be defined for any tables. But, it can be suitable for
> "acl column".

Ah, that is a good point, that if we have "security column" which is
usually null then we are requiring the NULL bitmask.

I was thinking of having them be system columns and therefore only the
bitmask would specify if the value exists or not.  I am again thinking
of having both columns always exist, making your code clearer and more
portable for people going to and leaving SE-Postgres.

> The following proposal is my idea:
> 
> - The Row-level ACLs is implemented as a hardcoded feature, not a guest of
>    PGACE security framework.

Yep, good.

> - It can be enabled/disabled via table options as:
>        CREATE TABLE t (
>            a   int,
>            b   text
>        ) WITH (ROW_LEVEL_ACL=ON);

Can we make it just always on?  See above.

> - When the Row-level ACLs is enabled on CREATE TABLE, it implicitly add
>    a column to represent Row-level ACLs. This column is defined as aclitem[].
> - The "acl column" is not expanded via "SELECT * FROM ...", but we can specify
>    it explicitly like "SELECT tuple_acl, * FROM ..." as system column doing.

Right, just like other system columns, e.g. xmin.

> - The "acl column" is allowed to update by the table owner or superuser.

Yep.

> - If a table has Row-level ACLs enabled, ExecScan() checks visibility of
>    fetched tuples, and ignore violated tuples.

Yep.

> > Finally, I am wondering what other things should be adjusted that I have
> > not thought of yet.  I would like to make KaiGai's job as easy as
> > possible.

I feel there must still be open issues that I hope we can address before
you need to do more recoding of the patch.  And hopefully other
community members will be involved as well.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

11 December 2008, 01:55:27

KaiGai Kohei wrote:
> Bruce Momjian wrote:
> > Bruce Momjian wrote:
> >> Second, system catalogs; are you going to have separate columns for
> >> SQL-level row security and SE-Linux security?  If so, is the SE-Linux
> >> column only going to be created by the --enable-selinux build?  If so,
> >> that is going to mean that the /data directory is going to be affected
> >> by the --enable-selinux flag, which is not ideal --- it would be good if
> >> someone could switch to an SE-Linux binary without to dump/reload.  One
> >> nice idea would be to have one "security" column and allow both
> >> SQL-level and SE-Linux security values to be stored in the column,
> >> perhaps at the same time;  that way there is no new column needed for
> >> SE-Linux.
> >>
> >> I know I suggested the security column act like the system oid column,
> >> but now I am thinking I was wrong.  The system oid column stores an
> >> auto-generated value so it was necessary to have it be specified at
> >> CREATE TABLE time because it is guaranteed to take storage space.  For
> >> the security column, in most cases it will be null, meaning it would not
> >> take any storage space becuase we use a null bitmap for null column
> >> values.  So, perhaps the security column should be in every table and we
> >> can just make it default to null;  I think that would give us much more
> >> flexibility to add security to existing tables.
> > 
> > Sorry, I am confusing system columns, like pg_class.relname, with
> > user-table system columns, like mytable.xmin.  I don't think we have the
> > ability of having user-table system columns nullable, right?
> 
> Currently, it assumes system columns should not be NULL.
> heap_getsysattr() always set the given 'isnull' 'false'.

Yep, that is a problem.

> > We do have a per-row HEAP_HASOID bit, so I wonder if we can have a
> > HEAP_HASSEC bit too.  Right now the HEAP_HASOID is controlled by the
> > CREATE/ALTER table;
> 
> The current patch add HEAP_HASSECURITY bit to t_infomask. :-)
> When it is false, its security field is not available and not allocated.

Good.

>  > I am unclear how hard it would be to allow it to be
>  > controlled via INSERT, meaning it would be present only if the row had a
>  > SEC value supplied.
> 
> It is impossible, and not suitable for SE-PostgreSQL.
> The HeapTuple is allocated prior to fetch INSERT'ed value.
> In addition, SE-PostgreSQL assigns its default security context when no
> security attribute is given.

When SE-Linux is enabled, couldn't NULL represent the default security
context?

> > Here is our OID column on some rows but not others:
> > 
> >     test=> CREATE TABLE test (x INT) WITH oids;
> >     CREATE TABLE
> >     test=> INSERT INTO test VALUES (1);
> >     INSERT 16392 1
> >     test=> ALTER TABLE test SET WITHOUT OIDS;
> >     ALTER TABLE
> >     test=> INSERT INTO test VALUES(2);
> >     INSERT 0 1
> >     test=> SELECT oid, * FROM test;
> >     ERROR:  COLUMN "oid" does not exist
> >     LINE 1: SELECT oid, * FROM test;
> > 
> > but it has per-table granularity;  the oid value for '1' is still
> > stored, but is no longer visible;  that would not work for the security
> > value.
> 
> If we can control the Row-level ACLs via table options, what should be
> happen when we turn off the feature?
> IMO, it is natural to ignore Row-level ACLs independent from whether
> the stored tuple actually has ACLs, or not.

Hmmm, sounds like perhaps a GUC is needed there.

> > The complexity is that the column would always exist (unlike OID), but
> > would be NULL if not assigned to.  This would allow any table to have
> > security by default, and have no overhead if security is not defined for
> > the row.  Now, I realize SE-Linux would have security defined for each
> > row, but SQL-level row security might not, and this would allow the
> > feature to be very useful, perhaps with a GUC that required security for
> > SE-Linux, and /data would be the same for SE-Linux and non-SE-Linux.  If
> > we could make the row behave like this we might use separate columns for
> > SQL-level and SE-Linux security because there would be no storage
> > overhead unless security was used.
> 
> I basically agree for the idea of separate columns.
> My preference is "acl column" as general column (not a system column)
> which is defined at limited number of tables with ROW_LEVEL_ACL=ON.

That would certainly be the easiest approach.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

11 December 2008, 02:37:56

Bruce Momjian wrote:
> KaiGai Kohei wrote:
>>> Let's decide exactly what configure options, GUC options, system catalog
>>> changes, and user-visible SQL command syntax we are going to use for the
>>> patch before you recode anything.
>> The guest of PGACE security framework should be chosen by configure option.
>> It also means any other facilities except for the guest of PGACE can be
>> enabled/disabled by other kind of options.
>>
>> If the Row-level ACLs should be always enabled (independent from SE-PostgreSQL),
>> I think it should be hardcoded outside of PGACE, and enabled/disabled by
>> any other way. Table option is a candidate, like:
>>
>>    CREATE TABLE t (
>>        a   int,
>>        b   text
>>    ) WITH (ROW_LEVEL_ACL=ON);
> 
> As I mentioned below, this might not be necessary, meaning that row
> security is enabled for rows where you set the row security value and
> does not need to be turned on at create table time.

The reason why I proposed the ROW_LEVEL_ACL option is to reduce storage
consumption for NULL bitmap. For example, the following INSERT originally
didn't need NULL bitmap, but the "acl column" is implicitly NULL, so the
HeapTuple has HEAP_HASNULL flag and NULL bitmap is allocated.
  INSERT INTO t VALUES (1, 'aaa');

If it is acceptable, I don't think the option is necessary.

>>> If so, is the SE-Linux
>>> column only going to be created by the --enable-selinux build?  If so,
>>> that is going to mean that the /data directory is going to be affected
>>> by the --enable-selinux flag, which is not ideal --- it would be good if
>>> someone could switch to an SE-Linux binary without to dump/reload.  One
>>> nice idea would be to have one "security" column and allow both
>>> SQL-level and SE-Linux security values to be stored in the column,
>>> perhaps at the same time;  that way there is no new column needed for
>>> SE-Linux.
>> As you noticed, a "conditional system column" can be open to discuss.
>> If someone switch his binary to SE- version, the tables already defined
>> in $PGDATA don't have "security_context" system column. SE-PostgreSQL
>> handles tuples stored within the table as "unlabeled" one, but it does
>> not allow users to access the attribute due to lack of proper system
>> column.
> 
> Right, if the "security_context" always exists, but is null, moving to
> SE-Linux would allow them to add security without dump/reload.

Yes, but it is not recommendable in generally. For meaningful access
controls, "unlabeled" objects should be labeled properly. :)

>> In addition, we may have to refer security context of tuples via
>> "tuple_acl" system column, when someone switch his binary from
>> the Row-level ACL to SE-PostgreSQL. :)
>>
>> The naming is a difficult matter. It seems to me "security" is too
>> general term. How do you think "security_label" or "security_attribute"?
> 
> So "security_context" has to be the PGACE column name.  I would say
> security_acl or just secacl because I think it is going to match our
> existing ACL system pretty closely.

Agreed,

>>> I know I suggested the security column act like the system oid column,
>>> but now I am thinking I was wrong.  The system oid column stores an
>>> auto-generated value so it was necessary to have it be specified at
>>> CREATE TABLE time because it is guaranteed to take storage space.  For
>>> the security column, in most cases it will be null, meaning it would not
>>> take any storage space because we use a null bitmap for null column
>>> values.  So, perhaps the security column should be in every table and we
>>> can just make it default to null;  I think that would give us much more
>>> flexibility to add security to existing tables.
>> Here is a differences in frequency of valid value in security column
>> between SE-PostgreSQL and Row-level ACLs.
>> SE-PostgreSQL requires all the tuple should be labeled properly.
>> (And, I guess any other upcoming feature to support another secure OS
>> has similar requirement.)
>> But we assume the Row-level ACLs is enabled on the limited number of
>> tables, so it need to specify an option to activate on CREATE TABLE.
>>
>> Please note that I distinguish between "security column" and "acl column"
>> here. In my opinion, the security column should be a system column, but
>> the acl column can be implemented as a regular column that is appended
>> depending on user's needs.
> 
> If we can make the column not take any space if it is null then we can
> have both columns always present and that way /data is the same for
> SE-Linux and non-SE-Linux.  We do have columns like xmin which don't
> show in SELECT * and I don't see a problem adding two more.
> 
> I imagine we would need to modify COPY and pg_dump to specify if we are
> to dump/load secacl; I see you already did "security_context".

Yes, it has to be also done.

>> If it is represented as NULL bitmaps, it requires a tuple has its null bitmaps
>> which need 4-bytes at least, so it can make additional storage consumption.
>> Thus, NULL-bitmap approach is not suitable to represent "security column"
>> because it has to be defined for any tables. But, it can be suitable for
>> "acl column".
> 
> Ah, that is a good point, that if we have "security column" which is
> usually null then we are requiring the NULL bitmask.
> 
> I was thinking of having them be system columns and therefore only the
> bitmask would specify if the value exists or not.  I am again thinking
> of having both columns always exist, making your code clearer and more
> portable for people going to and leaving SE-Postgres.

The HEAP_HASSECURITY flag looks like a specific purpose NULL-bitmap
which only shows whether the tuple has security value, or not.

>> The following proposal is my idea:
>>
>> - The Row-level ACLs is implemented as a hardcoded feature, not a guest of
>>    PGACE security framework.
> 
> Yep, good.
> 
>> - It can be enabled/disabled via table options as:
>>        CREATE TABLE t (
>>            a   int,
>>            b   text
>>        ) WITH (ROW_LEVEL_ACL=ON);
> 
> Can we make it just always on?  See above.

If we assume users set up Row-level ACLs for specific tables, per-table
option is meaningful for reduction of NULL-bitmap space in the tuple
without any NULL-values on general columns.

>> - When the Row-level ACLs is enabled on CREATE TABLE, it implicitly add
>>    a column to represent Row-level ACLs. This column is defined as aclitem[].
>> - The "acl column" is not expanded via "SELECT * FROM ...", but we can specify
>>    it explicitly like "SELECT tuple_acl, * FROM ..." as system column doing.
> 
> Right, just like other system columns, e.g. xmin.
> 
>> - The "acl column" is allowed to update by the table owner or superuser.
> 
> Yep.
> 
>> - If a table has Row-level ACLs enabled, ExecScan() checks visibility of
>>    fetched tuples, and ignore violated tuples.
> 
> Yep.
> 
>>> Finally, I am wondering what other things should be adjusted that I have
>>> not thought of yet.  I would like to make KaiGai's job as easy as
>>> possible.
> 
> I feel there must still be open issues that I hope we can address before
> you need to do more recoding of the patch.  And hopefully other
> community members will be involved as well.
> 

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

11 December 2008, 02:46:22

>>  > I am unclear how hard it would be to allow it to be
>>  > controlled via INSERT, meaning it would be present only if the row had a
>>  > SEC value supplied.
>>
>> It is impossible, and not suitable for SE-PostgreSQL.
>> The HeapTuple is allocated prior to fetch INSERT'ed value.
>> In addition, SE-PostgreSQL assigns its default security context when no
>> security attribute is given.
> 
> When SE-Linux is enabled, couldn't NULL represent the default security
> context?

Yes, users will get the default security context, as if they insert
a tuple without any specific "oid" but proper value is assigned.

>>> Here is our OID column on some rows but not others:
>>>
>>>     test=> CREATE TABLE test (x INT) WITH oids;
>>>     CREATE TABLE
>>>     test=> INSERT INTO test VALUES (1);
>>>     INSERT 16392 1
>>>     test=> ALTER TABLE test SET WITHOUT OIDS;
>>>     ALTER TABLE
>>>     test=> INSERT INTO test VALUES(2);
>>>     INSERT 0 1
>>>     test=> SELECT oid, * FROM test;
>>>     ERROR:  COLUMN "oid" does not exist
>>>     LINE 1: SELECT oid, * FROM test;
>>>
>>> but it has per-table granularity;  the oid value for '1' is still
>>> stored, but is no longer visible;  that would not work for the security
>>> value.
>> If we can control the Row-level ACLs via table options, what should be
>> happen when we turn off the feature?
>> IMO, it is natural to ignore Row-level ACLs independent from whether
>> the stored tuple actually has ACLs, or not.
> 
> Hmmm, sounds like perhaps a GUC is needed there.

Is it possible to control per-table granuality?
Or, per-table control is not necessary?

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

11 December 2008, 04:07:19

KaiGai Kohei wrote:
> >>    CREATE TABLE t (
> >>        a   int,
> >>        b   text
> >>    ) WITH (ROW_LEVEL_ACL=ON);
> > 
> > As I mentioned below, this might not be necessary, meaning that row
> > security is enabled for rows where you set the row security value and
> > does not need to be turned on at create table time.
> 
> The reason why I proposed the ROW_LEVEL_ACL option is to reduce storage
> consumption for NULL bitmap. For example, the following INSERT originally
> didn't need NULL bitmap, but the "acl column" is implicitly NULL, so the
> HeapTuple has HEAP_HASNULL flag and NULL bitmap is allocated.
> 
>    INSERT INTO t VALUES (1, 'aaa');
> 
> If it is acceptable, I don't think the option is necessary.

I was hoping we could allow the column to be null based only on the
bitmask without having to use the user-column bitmask but maybe that
isn't possible.

> >> As you noticed, a "conditional system column" can be open to discuss.
> >> If someone switch his binary to SE- version, the tables already defined
> >> in $PGDATA don't have "security_context" system column. SE-PostgreSQL
> >> handles tuples stored within the table as "unlabeled" one, but it does
> >> not allow users to access the attribute due to lack of proper system
> >> column.
> > 
> > Right, if the "security_context" always exists, but is null, moving to
> > SE-Linux would allow them to add security without dump/reload.
> 
> Yes, but it is not recommendable in generally. For meaningful access
> controls, "unlabeled" objects should be labeled properly. :)

We could get tricky and use the proper default when that column is null
for user-visible queries.  Again,  I am just throwing out ideas.

> >> If it is represented as NULL bitmaps, it requires a tuple has its null bitmaps
> >> which need 4-bytes at least, so it can make additional storage consumption.
> >> Thus, NULL-bitmap approach is not suitable to represent "security column"
> >> because it has to be defined for any tables. But, it can be suitable for
> >> "acl column".
> > 
> > Ah, that is a good point, that if we have "security column" which is
> > usually null then we are requiring the NULL bitmask.
> > 
> > I was thinking of having them be system columns and therefore only the
> > bitmask would specify if the value exists or not.  I am again thinking
> > of having both columns always exist, making your code clearer and more
> > portable for people going to and leaving SE-Postgres.
> 
> The HEAP_HASSECURITY flag looks like a specific purpose NULL-bitmap
> which only shows whether the tuple has security value, or not.

Yep, that was my hope.

> >> The following proposal is my idea:
> >>
> >> - The Row-level ACLs is implemented as a hardcoded feature, not a guest of
> >>    PGACE security framework.
> > 
> > Yep, good.
> > 
> >> - It can be enabled/disabled via table options as:
> >>        CREATE TABLE t (
> >>            a   int,
> >>            b   text
> >>        ) WITH (ROW_LEVEL_ACL=ON);
> > 
> > Can we make it just always on?  See above.
> 
> If we assume users set up Row-level ACLs for specific tables, per-table
> option is meaningful for reduction of NULL-bitmap space in the tuple
> without any NULL-values on general columns.

Right.  I was hoping there was a way to have HEAP_HASSECACL control if
the value is present or not.

I sure wish others were adding ideas to this discussion.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

11 December 2008, 04:11:16

KaiGai Kohei wrote:
> >>> Here is our OID column on some rows but not others:
> >>>
> >>>     test=> CREATE TABLE test (x INT) WITH oids;
> >>>     CREATE TABLE
> >>>     test=> INSERT INTO test VALUES (1);
> >>>     INSERT 16392 1
> >>>     test=> ALTER TABLE test SET WITHOUT OIDS;
> >>>     ALTER TABLE
> >>>     test=> INSERT INTO test VALUES(2);
> >>>     INSERT 0 1
> >>>     test=> SELECT oid, * FROM test;
> >>>     ERROR:  COLUMN "oid" does not exist
> >>>     LINE 1: SELECT oid, * FROM test;
> >>>
> >>> but it has per-table granularity;  the oid value for '1' is still
> >>> stored, but is no longer visible;  that would not work for the security
> >>> value.
> >> If we can control the Row-level ACLs via table options, what should be
> >> happen when we turn off the feature?
> >> IMO, it is natural to ignore Row-level ACLs independent from whether
> >> the stored tuple actually has ACLs, or not.
> > 
> > Hmmm, sounds like perhaps a GUC is needed there.
> 
> Is it possible to control per-table granuality?
> Or, per-table control is not necessary?

Well, we have a GUC which controls whether tables get an OID column. 
If we do the 'secacl' on a per-table level we should probably have a GUC
which would default to ON for all created tables.

Perhaps someone can chime in here to tell us how hard it would be to
allow system columns to be null and not take any storage space, i.e. use
HeapTupleHeaderData.t_infomask on a per-row basis and not
HeapTupleHeaderData.t_bits.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

11 December 2008, 04:39:26

Bruce Momjian wrote:
> KaiGai Kohei wrote:
> > >>    CREATE TABLE t (
> > >>        a   int,
> > >>        b   text
> > >>    ) WITH (ROW_LEVEL_ACL=ON);

Let me outline the simplest API, assuming we are using table-level
granularity for the security columns.

CREATE TABLE would support
WITH (ROWACL = TRUE/FALSE);

for row-level acl and:
WITH (SECEXT = TRUE/FALSE);

for SE-Linux, with 'SECEXTL' standing for SECurity EXTernal or
SECurity_contEXT.

And then in postgresql.conf we would have:
default_with_rowacl

and
default_with_secext

which would control the default value of ROWACL and SECEXT when CREATE
TABLE does not specify these values.  This is how OIDs works now.

When SE-Linux is enabled, CREATE TABLE would issue an error if SECEXT
was false.  I can't think of a clean way to guarantee that existing
tables have SECEXT though, which means we might need to have a missing
'security_context' column mean default SE-Linux permissions.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

11 December 2008, 04:43:33

Bruce Momjian wrote:
> Let me outline the simplest API, assuming we are using table-level
> granularity for the security columns.
> 
> CREATE TABLE would support
> 
>     WITH (ROWACL = TRUE/FALSE);
> 
> for row-level acl and:
> 
>     WITH (SECEXT = TRUE/FALSE);
> 
> for SE-Linux, with 'SECEXTL' standing for SECurity EXTernal or
> SECurity_contEXT.

FYI, in addition COPY and pg_dump would have to have similar settings.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

11 December 2008, 10:09:53

Bruce Momjian wrote:
> Bruce Momjian wrote:
>> KaiGai Kohei wrote:
>>>>>    CREATE TABLE t (
>>>>>        a   int,
>>>>>        b   text
>>>>>    ) WITH (ROW_LEVEL_ACL=ON);
> 
> Let me outline the simplest API, assuming we are using table-level
> granularity for the security columns.
> 
> CREATE TABLE would support
> 
>     WITH (ROWACL = TRUE/FALSE);
> And then in postgresql.conf we would have:>>     default_with_rowacl

Yes, I agree it.

But SE-PostgreSQL does not need its table option to control
its availability per table granuality due to its security model.

Database ACL is a kind of DAC. It allows resource owners to
set up its access rights. In other hand, SE-PostgreSQL is an
implementation of MAC. It does not allow owners to control its
access rights. This is the role of centralized security policy,

> When SE-Linux is enabled, CREATE TABLE would issue an error if SECEXT
> was false.  I can't think of a clean way to guarantee that existing
> tables have SECEXT though, which means we might need to have a missing
> 'security_context' column mean default SE-Linux permissions.

SE-PostgreSQL stores its security context on the security field of
HeapTupleHeader and set HEAP_HASSECURITY of t_infomask.
The security system column is always available, so it does not make
any matter. When no guest is available on PGACE, HEAP_HASSECURITY of
t_infomask is not set, so security field is not allocated and NULL
bitmask is not polluted.
>> If we assume users set up Row-level ACLs for specific tables, per-table>> option is meaningful for reduction of
NULL-bitmapspace in the tuple>> without any NULL-values on general columns.>> Right.  I was hoping there was a way to
haveHEAP_HASSECACL control if> the value is present or not.>> I sure wish others were adding ideas to this discussion.

I have a plan to add a new field (declared as "int2 relrowacl") into
pg_class to show what column stores its Row-level ACLs.
When we create a table with (ROWACL=TRUE), it implicitly add a column
declared as "security_acl aclitem[]", and its attribute number is
stored within the "pg_class.relrowacl". If it has positive value,
tuples within the table can have its individual ACLs. No-ACL is
represented via the NULL-bitmap. If it is zero, the table does not
have the "security_acl" column, and the row-level controls are simply
ignored.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

11 December 2008, 13:40:01

KaiGai Kohei wrote:
> Bruce Momjian wrote:
> > Bruce Momjian wrote:
> >> KaiGai Kohei wrote:
> >>>>>    CREATE TABLE t (
> >>>>>        a   int,
> >>>>>        b   text
> >>>>>    ) WITH (ROW_LEVEL_ACL=ON);
> > 
> > Let me outline the simplest API, assuming we are using table-level
> > granularity for the security columns.
> > 
> > CREATE TABLE would support
> > 
> >     WITH (ROWACL = TRUE/FALSE);
> 
>  > And then in postgresql.conf we would have:
>  >
>  >     default_with_rowacl
> 
> Yes, I agree it.
> 
> But SE-PostgreSQL does not need its table option to control
> its availability per table granuality due to its security model.
> 
> Database ACL is a kind of DAC. It allows resource owners to
> set up its access rights. In other hand, SE-PostgreSQL is an
> implementation of MAC. It does not allow owners to control its
> access rights. This is the role of centralized security policy,

It is fine if you require SECEXT to be on for SE-Linux, but the option
must be available for non-SE-Linux so you can load dumps from either
Postgres configuration, and /data is compatible with both versions.

> > When SE-Linux is enabled, CREATE TABLE would issue an error if SECEXT
> > was false.  I can't think of a clean way to guarantee that existing
> > tables have SECEXT though, which means we might need to have a missing
> > 'security_context' column mean default SE-Linux permissions.
> 
> SE-PostgreSQL stores its security context on the security field of
> HeapTupleHeader and set HEAP_HASSECURITY of t_infomask.
> The security system column is always available, so it does not make
> any matter. When no guest is available on PGACE, HEAP_HASSECURITY of
> t_infomask is not set, so security field is not allocated and NULL
> bitmask is not polluted.

If you make an SE-Linux dump with security fields, how will that be
loadable in a non-SE-Linux Postgres database?

We are also going to need ALTER TABLE to be able to add/remove these
columns from tables, like OIDs.

>  >> If we assume users set up Row-level ACLs for specific tables, per-table
>  >> option is meaningful for reduction of NULL-bitmap space in the tuple
>  >> without any NULL-values on general columns.
>  >
>  > Right.  I was hoping there was a way to have HEAP_HASSECACL control if
>  > the value is present or not.
>  >
>  > I sure wish others were adding ideas to this discussion.
> 
> I have a plan to add a new field (declared as "int2 relrowacl") into
> pg_class to show what column stores its Row-level ACLs.
> When we create a table with (ROWACL=TRUE), it implicitly add a column
> declared as "security_acl aclitem[]", and its attribute number is
> stored within the "pg_class.relrowacl". If it has positive value,
> tuples within the table can have its individual ACLs. No-ACL is
> represented via the NULL-bitmap. If it is zero, the table does not
> have the "security_acl" column, and the row-level controls are simply
> ignored.

I am confused why we would want this instead of the way we do oids.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

RE: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Zeugswetter Andreas OSB sIT

Date:

11 December 2008, 14:01:19

> > > Ah, that is a good point, that if we have "security column" which is
> > > usually null then we are requiring the NULL bitmask.

Yes, I think that would not be optimal, thus I think "WITH SECURITY_CONTEXT"
is needed.

> I sure wish others were adding ideas to this discussion.

One such idea would be, that the security info is already normalized.
pg_security has one row for each security_context. It is my understanding, that
such a context row may already be a combination of "rights". Thus adding an extra column
per subsystem to the user tables may not be required.

You could have all info for each security subsystem in the pg_security table.
This can eighter be done by having one row in pg_security per
subsystem type and oid, or by having a separate column in pg_security per subsystem.

The imho difficult part is, that currently selecting "security_context" defaults to mapping the
oid to the text representation for selinux. Concern has already been voiced in this regard.
Maybe this is another reason to not do automatic mapping, but require a specified conversion
for text output.

Or is the column name "security_context" and representation a standard ?

This is just an idea, since I do not really think actually using more than one
security subsystem in parallel will be common.

Andreas

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

11 December 2008, 14:04:22

Zeugswetter Andreas OSB sIT wrote:
> 
> > > > Ah, that is a good point, that if we have "security column" which is
> > > > usually null then we are requiring the NULL bitmask.
> 
> Yes, I think that would not be optimal, thus I think "WITH
> SECURITY_CONTEXT" is needed.
> 
> > I sure wish others were adding ideas to this discussion.
> 
> One such idea would be, that the security info is already
> normalized.  pg_security has one row for each security_context.
> It is my understanding, that such a context row may already be
> a combination of "rights". Thus adding an extra column per
> subsystem to the user tables may not be required.  >
> You could have all info for each security subsystem in the
> pg_security table.  This can eighter be done by having one row
> in pg_security per subsystem type and oid, or by having a separate
> column in pg_security per subsystem.
> 
> The imho difficult part is, that currently selecting "security_context"
> defaults to mapping the oid to the text representation for
> selinux. Concern has already been voiced in this regard.  Maybe
> this is another reason to not do automatic mapping, but require
> a specified conversion for text output.
> 
> Or is the column name "security_context" and representation a
> standard ?
> 
> This is just an idea, since I do not really think actually using
> more than one security subsystem in parallel will be common.

We already have this.

The idea is that the security columns will hold an OID and the OID will
point to a row in a table that contains the security rights/ACL for the
column, with multiple rows using the same rights OID.  If you change the
rights on the column the code has to check the existing entries and add
a new one if it doesn't already exist.  This does add the problem of how
to remove security rows that are no longer referenced.

-- Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Tom Lane

Date:

11 December 2008, 14:09:46

Bruce Momjian <bruce@momjian.us> writes:
> Let me outline the simplest API, assuming we are using table-level
> granularity for the security columns.
> CREATE TABLE would support
>     WITH (ROWACL = TRUE/FALSE);
> for row-level acl and:
>     WITH (SECEXT = TRUE/FALSE);
> for SE-Linux, with 'SECEXTL' standing for SECurity EXTernal or
> SECurity_contEXT.

Wait a minute.  The original argument for providing SQL-driven row level
security was that it would help provide a framework for testing the code
and doing something useful with it on non-selinux platforms.  Now we
seem to be proposing two independent implementations --- which, even
if similar, could still suffer different bugs (due to copy-and-pasteos
if nothing else).  So the testing argument goes right out the window.
Also, this is getting even further afield from any capability that
anyone actually asked for.

I think there should be only *one* underlying column and that it should
be manipulable by either SQL commands or selinux.  Otherwise you're
making a lie of the primary argument for having the SQL feature at all.

It's possible that some people would want to insist that only selinux
be used to manipulate the settings, but I think that could be addressed
by a compile-time option to disable the SQL commands.
        regards, tom lane

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Tom Lane

Date:

11 December 2008, 14:11:47

KaiGai Kohei <kaigai@kaigai.gr.jp> writes:
> I have a plan to add a new field (declared as "int2 relrowacl") into
> pg_class to show what column stores its Row-level ACLs.

If you want the column to be "hidden" in the same way that system
columns are, I'm afraid this is a pretty bad idea.  There are way too
many places that would require weird hacks to keep from treating a
positive-numbered column as a regular user column.
        regards, tom lane

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

11 December 2008, 14:19:03

>> But SE-PostgreSQL does not need its table option to control
>> its availability per table granuality due to its security model.
>>
>> Database ACL is a kind of DAC. It allows resource owners to
>> set up its access rights. In other hand, SE-PostgreSQL is an
>> implementation of MAC. It does not allow owners to control its
>> access rights. This is the role of centralized security policy,
> 
> It is fine if you require SECEXT to be on for SE-Linux, but the option
> must be available for non-SE-Linux so you can load dumps from either
> Postgres configuration, and /data is compatible with both versions.

It is unclear for me why you thought this option is necessary to load
dumps generated by vanilla PostgreSQL?
I assumes the security column should be always available, so we don't
need to add a column when SE-PostgreSQL load a vanilla $PGDATA.

>>> When SE-Linux is enabled, CREATE TABLE would issue an error if SECEXT
>>> was false.  I can't think of a clean way to guarantee that existing
>>> tables have SECEXT though, which means we might need to have a missing
>>> 'security_context' column mean default SE-Linux permissions.
>> SE-PostgreSQL stores its security context on the security field of
>> HeapTupleHeader and set HEAP_HASSECURITY of t_infomask.
>> The security system column is always available, so it does not make
>> any matter. When no guest is available on PGACE, HEAP_HASSECURITY of
>> t_infomask is not set, so security field is not allocated and NULL
>> bitmask is not polluted.
> 
> If you make an SE-Linux dump with security fields, how will that be
> loadable in a non-SE-Linux Postgres database?

In this case, user should no dump his database with security field.
The patched pg_dump does not dump security field without
'--security-context' option.

> We are also going to need ALTER TABLE to be able to add/remove these
> columns from tables, like OIDs.

I don't agree.
The "security column" (not a "acl column") should be always exist.
In the vanilla binary, it simply returns NULL or empty string, so there is
no waste of storage consumption.
It is worthful when we run SE- binary with $PGDATA generated by vanilla one.
Any stored tuple does not have security field, but it gives DBAs a chance to
relabel them via the "security column".

>>  >> If we assume users set up Row-level ACLs for specific tables, per-table
>>  >> option is meaningful for reduction of NULL-bitmap space in the tuple
>>  >> without any NULL-values on general columns.
>>  >
>>  > Right.  I was hoping there was a way to have HEAP_HASSECACL control if
>>  > the value is present or not.
>>  >
>>  > I sure wish others were adding ideas to this discussion.
>>
>> I have a plan to add a new field (declared as "int2 relrowacl") into
>> pg_class to show what column stores its Row-level ACLs.
>> When we create a table with (ROWACL=TRUE), it implicitly add a column
>> declared as "security_acl aclitem[]", and its attribute number is
>> stored within the "pg_class.relrowacl". If it has positive value,
>> tuples within the table can have its individual ACLs. No-ACL is
>> represented via the NULL-bitmap. If it is zero, the table does not
>> have the "security_acl" column, and the row-level controls are simply
>> ignored.
> 
> I am confused why we would want this instead of the way we do oids.

It enables to implement the hardcoded row-acl more simple.
It allows to store variable length ACLs using existing mechanism, and
makes unnecessary to translate between ACLs and raw text representation.


Thanks,

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Alvaro Herrera

Date:

11 December 2008, 14:20:04

Bruce Momjian wrote:

> The idea is that the security columns will hold an OID and the OID will
> point to a row in a table that contains the security rights/ACL for the
> column, with multiple rows using the same rights OID.  If you change the
> rights on the column the code has to check the existing entries and add
> a new one if it doesn't already exist.  This does add the problem of how
> to remove security rows that are no longer referenced.

How will it search for existing entries?  Are you saying that it will
seqscan the whole catalog of entries, looking for a match?

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

11 December 2008, 14:33:25

Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > Let me outline the simplest API, assuming we are using table-level
> > granularity for the security columns.
> > CREATE TABLE would support
> >     WITH (ROWACL = TRUE/FALSE);
> > for row-level acl and:
> >     WITH (SECEXT = TRUE/FALSE);
> > for SE-Linux, with 'SECEXTL' standing for SECurity EXTernal or
> > SECurity_contEXT.
> 
> Wait a minute.  The original argument for providing SQL-driven row level
> security was that it would help provide a framework for testing the code
> and doing something useful with it on non-selinux platforms.  Now we
> seem to be proposing two independent implementations --- which, even
> if similar, could still suffer different bugs (due to copy-and-pasteos
> if nothing else).  So the testing argument goes right out the window.
> Also, this is getting even further afield from any capability that
> anyone actually asked for.

Yep, no question.  The two-column idea happened because ignoring the
rowacl value for SE-Linux seemed wrong, but I am fine with it.

> I think there should be only *one* underlying column and that it should
> be manipulable by either SQL commands or selinux.  Otherwise you're
> making a lie of the primary argument for having the SQL feature at all.

True.

> It's possible that some people would want to insist that only selinux
> be used to manipulate the settings, but I think that could be addressed
> by a compile-time option to disable the SQL commands.

Yes, an SE-Linux compile could throw errors for those commands.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

11 December 2008, 14:38:56

At this point I am so confused I don't have any response.

---------------------------------------------------------------------------

KaiGai Kohei wrote:
> >> But SE-PostgreSQL does not need its table option to control
> >> its availability per table granuality due to its security model.
> >>
> >> Database ACL is a kind of DAC. It allows resource owners to
> >> set up its access rights. In other hand, SE-PostgreSQL is an
> >> implementation of MAC. It does not allow owners to control its
> >> access rights. This is the role of centralized security policy,
> > 
> > It is fine if you require SECEXT to be on for SE-Linux, but the option
> > must be available for non-SE-Linux so you can load dumps from either
> > Postgres configuration, and /data is compatible with both versions.
> 
> It is unclear for me why you thought this option is necessary to load
> dumps generated by vanilla PostgreSQL?
> I assumes the security column should be always available, so we don't
> need to add a column when SE-PostgreSQL load a vanilla $PGDATA.
> 
> >>> When SE-Linux is enabled, CREATE TABLE would issue an error if SECEXT
> >>> was false.  I can't think of a clean way to guarantee that existing
> >>> tables have SECEXT though, which means we might need to have a missing
> >>> 'security_context' column mean default SE-Linux permissions.
> >> SE-PostgreSQL stores its security context on the security field of
> >> HeapTupleHeader and set HEAP_HASSECURITY of t_infomask.
> >> The security system column is always available, so it does not make
> >> any matter. When no guest is available on PGACE, HEAP_HASSECURITY of
> >> t_infomask is not set, so security field is not allocated and NULL
> >> bitmask is not polluted.
> > 
> > If you make an SE-Linux dump with security fields, how will that be
> > loadable in a non-SE-Linux Postgres database?
> 
> In this case, user should no dump his database with security field.
> The patched pg_dump does not dump security field without
> '--security-context' option.
> 
> > We are also going to need ALTER TABLE to be able to add/remove these
> > columns from tables, like OIDs.
> 
> I don't agree.
> The "security column" (not a "acl column") should be always exist.
> In the vanilla binary, it simply returns NULL or empty string, so there is
> no waste of storage consumption.
> It is worthful when we run SE- binary with $PGDATA generated by vanilla one.
> Any stored tuple does not have security field, but it gives DBAs a chance to
> relabel them via the "security column".
> 
> >>  >> If we assume users set up Row-level ACLs for specific tables, per-table
> >>  >> option is meaningful for reduction of NULL-bitmap space in the tuple
> >>  >> without any NULL-values on general columns.
> >>  >
> >>  > Right.  I was hoping there was a way to have HEAP_HASSECACL control if
> >>  > the value is present or not.
> >>  >
> >>  > I sure wish others were adding ideas to this discussion.
> >>
> >> I have a plan to add a new field (declared as "int2 relrowacl") into
> >> pg_class to show what column stores its Row-level ACLs.
> >> When we create a table with (ROWACL=TRUE), it implicitly add a column
> >> declared as "security_acl aclitem[]", and its attribute number is
> >> stored within the "pg_class.relrowacl". If it has positive value,
> >> tuples within the table can have its individual ACLs. No-ACL is
> >> represented via the NULL-bitmap. If it is zero, the table does not
> >> have the "security_acl" column, and the row-level controls are simply
> >> ignored.
> > 
> > I am confused why we would want this instead of the way we do oids.
> 
> It enables to implement the hardcoded row-acl more simple.
> It allows to store variable length ACLs using existing mechanism, and
> makes unnecessary to translate between ACLs and raw text representation.
> 
> 
> Thanks,

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Zeugswetter Andreas OSB sIT

Date:

11 December 2008, 14:43:00

> > > > > Ah, that is a good point, that if we have "security
> column" which is
> > > > > usually null then we are requiring the NULL bitmask.
> >
> > Yes, I think that would not be optimal, thus I think "WITH
> > SECURITY_CONTEXT" is needed.
> >
> > > I sure wish others were adding ideas to this discussion.
> >

> > One such idea would be, that the security info is already
> > normalized.

I formulated that sentence badly , sorry :-(
Replace with:
Since the security info is already normalized, one such idea would be:

> > pg_security has one row for each security_context.
> > It is my understanding, that such a context row may already be
> > a combination of "rights". Thus adding an extra column per
> > subsystem to the user tables may not be required.  >
> > You could have all info for each security subsystem in the
> > pg_security table.  This can eighter be done by having one row
> > in pg_security per subsystem type and oid, or by having a separate
> > column in pg_security per subsystem.
> >
> > The imho difficult part is, that currently selecting
> "security_context"
> > defaults to mapping the oid to the text representation for
> > selinux. Concern has already been voiced in this regard.  Maybe
> > this is another reason to not do automatic mapping, but require
> > a specified conversion for text output.
> >
> > Or is the column name "security_context" and representation a
> > standard ?
> >
> > This is just an idea, since I do not really think actually using
> > more than one security subsystem in parallel will be common.
>
> We already have this.
>
> The idea is that the security columns will hold an OID and the OID will
> point to a row in a table that contains the security rights/ACL for the
> column, with multiple rows using the same rights OID.  If you change the
> rights on the column the code has to check the existing entries and add
> a new one if it doesn't already exist.  This does add the problem of how
> to remove security rows that are no longer referenced.

Please reread with above correction,
and I'll also try a little differently:

Since a pg_security row already represents a combination of rights
within selinux, I do not really see why that cannot be extended to "a combination
of rowacl and selinux rights" or more general "one oid represents a unique
combination of rights within different subsystems" ?

A simplified example of pg_security:
oid    rights
1    selinux:secret_read rowacl:ra,rb
2    selinux:unlabeled_t rowacl:ra,rb
3    selinux:secret_read rowacl:ra

Andreas

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

11 December 2008, 14:43:56

> Bruce Momjian <bruce@momjian.us> writes:
>> Let me outline the simplest API, assuming we are using table-level
>> granularity for the security columns.
>> CREATE TABLE would support
>>     WITH (ROWACL = TRUE/FALSE);
>> for row-level acl and:
>>     WITH (SECEXT = TRUE/FALSE);
>> for SE-Linux, with 'SECEXTL' standing for SECurity EXTernal or
>> SECurity_contEXT.
> 
> Wait a minute.  The original argument for providing SQL-driven row level
> security was that it would help provide a framework for testing the code
> and doing something useful with it on non-selinux platforms.

Yes,
In addition, I want folks to remind that the Row-level ACLs are not designed
based on SQL standards. Thus, I called it one of the enhanced securities.

> I think there should be only *one* underlying column and that it should
> be manipulable by either SQL commands or selinux.  Otherwise you're
> making a lie of the primary argument for having the SQL feature at all.
> 
> It's possible that some people would want to insist that only selinux
> be used to manipulate the settings, but I think that could be addressed
> by a compile-time option to disable the SQL commands.

My original opinion is that users should be able to choose what enhanced
security mechanism is available on his system.
In all honesty, I don't understand why the Row-level ACLs has privileged
position in the enhanced securities and it should be always available.

Thanks,
--
KaiGai Kohei

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

11 December 2008, 14:45:18

Zeugswetter Andreas OSB sIT wrote:
> Please reread with above correction, 
> and I'll also try a little differently:
> 
> Since a pg_security row already represents a combination of rights
> within selinux, I do not really see why that cannot be extended to "a combination
> of rowacl and selinux rights" or more general "one oid represents a unique 
> combination of rights within different subsystems" ?
> 
> A simplified example of pg_security:
> oid    rights
> 1    selinux:secret_read rowacl:ra,rb
> 2    selinux:unlabeled_t rowacl:ra,rb
> 3    selinux:secret_read rowacl:ra

Yes, I suggested that but the patch author thought it would be better to
have two columns.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Peter Eisentraut

Date:

11 December 2008, 14:59:40

On Thursday 11 December 2008 04:52:51 Bruce Momjian wrote:
> > > We do have a per-row HEAP_HASOID bit, so I wonder if we can have a
> > > HEAP_HASSEC bit too.  Right now the HEAP_HASOID is controlled by the
> > > CREATE/ALTER table;
> >
> > The current patch add HEAP_HASSECURITY bit to t_infomask. :-)
> > When it is false, its security field is not available and not allocated.
>
> Good.

This is probably OK, but if you want to save a bit or generalize it, it might
be worth considering using the normal null bitmap and nullity everywhere
instead of individual HEAP_HASTHISORTHAT bits for every feature.

Of course, if we expect that most rows will have no security information, this
tradeoff might end up on the wrong side of the equation.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

11 December 2008, 15:05:28

Bruce Momjian wrote:
> At this point I am so confused I don't have any response.

Are you discussing the case when we start a PostgreSQL with $PGDATA
generated by different binary?

At first, please consider the case when we start SE-PostgreSQL with
$PGDATA generated by vanilla binary.
(1) In this case, any stored tuple is not set HEAP_HASSECURITY on its
t_infomask, so SE-PostgreSQL considers it as a "unlabeled_t" one.

(2) In addition, we assumed the security system column was always available
independent from compile-time option. Thus, existing table definition
has security system column within its pg_attribute entries.

(3) Thus, SE-PostgreSQL allows users to access security field of existing
tuples via the security column. Maybe, it returns "unlabeled_t" security
context, so we have to label it properly.

(4) In another point of view, how the security system column work in the
vanilla PostgreSQL? Because it does not available any guest on PGACE,
HEAP_HASSECURITY is not always set. It means we cannot set any value on
the security field via security system column, and reading the column
always returns dummy data (empty text).

In the reversed case, the vanilla PostgreSQL simply ignores security
field independent from HEAP_HASSECURITY. When it inserts/update a tuple,
it does not has its security field and lost its security attribute.
It is quite natural.

Does it help you to understand?

> ---------------------------------------------------------------------------
> 
> KaiGai Kohei wrote:
>>>> But SE-PostgreSQL does not need its table option to control
>>>> its availability per table granuality due to its security model.
>>>>
>>>> Database ACL is a kind of DAC. It allows resource owners to
>>>> set up its access rights. In other hand, SE-PostgreSQL is an
>>>> implementation of MAC. It does not allow owners to control its
>>>> access rights. This is the role of centralized security policy,
>>> It is fine if you require SECEXT to be on for SE-Linux, but the option
>>> must be available for non-SE-Linux so you can load dumps from either
>>> Postgres configuration, and /data is compatible with both versions.
>> It is unclear for me why you thought this option is necessary to load
>> dumps generated by vanilla PostgreSQL?
>> I assumes the security column should be always available, so we don't
>> need to add a column when SE-PostgreSQL load a vanilla $PGDATA.
>>
>>>>> When SE-Linux is enabled, CREATE TABLE would issue an error if SECEXT
>>>>> was false.  I can't think of a clean way to guarantee that existing
>>>>> tables have SECEXT though, which means we might need to have a missing
>>>>> 'security_context' column mean default SE-Linux permissions.
>>>> SE-PostgreSQL stores its security context on the security field of
>>>> HeapTupleHeader and set HEAP_HASSECURITY of t_infomask.
>>>> The security system column is always available, so it does not make
>>>> any matter. When no guest is available on PGACE, HEAP_HASSECURITY of
>>>> t_infomask is not set, so security field is not allocated and NULL
>>>> bitmask is not polluted.
>>> If you make an SE-Linux dump with security fields, how will that be
>>> loadable in a non-SE-Linux Postgres database?
>> In this case, user should no dump his database with security field.
>> The patched pg_dump does not dump security field without
>> '--security-context' option.
>>
>>> We are also going to need ALTER TABLE to be able to add/remove these
>>> columns from tables, like OIDs.
>> I don't agree.
>> The "security column" (not a "acl column") should be always exist.
>> In the vanilla binary, it simply returns NULL or empty string, so there is
>> no waste of storage consumption.
>> It is worthful when we run SE- binary with $PGDATA generated by vanilla one.
>> Any stored tuple does not have security field, but it gives DBAs a chance to
>> relabel them via the "security column".
>>
>>>>  >> If we assume users set up Row-level ACLs for specific tables, per-table
>>>>  >> option is meaningful for reduction of NULL-bitmap space in the tuple
>>>>  >> without any NULL-values on general columns.
>>>>  >
>>>>  > Right.  I was hoping there was a way to have HEAP_HASSECACL control if
>>>>  > the value is present or not.
>>>>  >
>>>>  > I sure wish others were adding ideas to this discussion.
>>>>
>>>> I have a plan to add a new field (declared as "int2 relrowacl") into
>>>> pg_class to show what column stores its Row-level ACLs.
>>>> When we create a table with (ROWACL=TRUE), it implicitly add a column
>>>> declared as "security_acl aclitem[]", and its attribute number is
>>>> stored within the "pg_class.relrowacl". If it has positive value,
>>>> tuples within the table can have its individual ACLs. No-ACL is
>>>> represented via the NULL-bitmap. If it is zero, the table does not
>>>> have the "security_acl" column, and the row-level controls are simply
>>>> ignored.
>>> I am confused why we would want this instead of the way we do oids.
>> It enables to implement the hardcoded row-acl more simple.
>> It allows to store variable length ACLs using existing mechanism, and
>> makes unnecessary to translate between ACLs and raw text representation.
>>
>>
>> Thanks,
>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Peter Eisentraut

Date:

11 December 2008, 15:08:47

On Thursday 11 December 2008 17:43:38 KaiGai Kohei wrote:
> In addition, I want folks to remind that the Row-level ACLs are not
> designed based on SQL standards. Thus, I called it one of the enhanced
> securities.

We have a lot of things in our code that are nonstandard, beyond the standard, 
enhanced, or pretty cool.  We don't call any of those interfaces "enhanced" 
and lay out our code based on that.  So I don't reall buy the "PGACE" 
premise.  Either your code is so modular that it is a separate plugin, which 
is probably not appropriate here, or it is really built-in.  Someone, I 
forgot who, recently wrote, a patch should make the code look as if the patch 
had been in there all along.  Yes.  Clear internal interfaces are also great, 
but don't lay out your code to create hooks or interfaces just because your 
are afraid to paste your code in someone else's source file.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

"Robert Haas"

Date:

11 December 2008, 15:15:46

> I think there should be only *one* underlying column and that it should
> be manipulable by either SQL commands or selinux.  Otherwise you're
> making a lie of the primary argument for having the SQL feature at all.

I agree that we're getting pretty far afield from the original
proposal, but I don't think it's a good idea to foreclose the option
of ever supporting MAC and DAC in the same executable.  Whichever one
the vendor decides to ship, I have to recompile if I want the other.
There's a good chance that most people will use NEITHER feature, but
it isn't nice if one of the two is easily available and the other is
much harder.

...Robert

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Peter Eisentraut

Date:

11 December 2008, 15:16:19

On Thursday 11 December 2008 17:09:25 Tom Lane wrote:
> I think there should be only *one* underlying column and that it should
> be manipulable by either SQL commands or selinux.  Otherwise you're
> making a lie of the primary argument for having the SQL feature at all.

Well, an SQL-manipulated row security column will probably have a content like
   {joe=rw/bob,staff=r/bob}

An SELinux-aware row security column will probably have a content like
  blah_t:foo_t:quux_t

And a Solaris TX-aware security column will probably have a content like
  Classified

How can we stick all of these in the same column at the same time?

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

11 December 2008, 15:25:03

Peter Eisentraut wrote:
> On Thursday 11 December 2008 17:09:25 Tom Lane wrote:
>> I think there should be only *one* underlying column and that it should
>> be manipulable by either SQL commands or selinux.  Otherwise you're
>> making a lie of the primary argument for having the SQL feature at all.
> 
> Well, an SQL-manipulated row security column will probably have a content like
> 
>     {joe=rw/bob,staff=r/bob}
> 
> An SELinux-aware row security column will probably have a content like
> 
>    blah_t:foo_t:quux_t
> 
> And a Solaris TX-aware security column will probably have a content like
> 
>    Classified
> 
> How can we stick all of these in the same column at the same time?

To choose it on compile-time option is the most simple approach.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Peter Eisentraut

Date:

11 December 2008, 15:29:15

On Thursday 11 December 2008 17:04:05 Bruce Momjian wrote:
> The idea is that the security columns will hold an OID and the OID will
> point to a row in a table that contains the security rights/ACL for the
> column, with multiple rows using the same rights OID.

That sounds somewhat scary for a number of reasons:

1. Running out of OIDs, the main reason why we got rid of OIDs in user tables 
by default.  This would essentially put them back.

2. You are implying some kind of ACL unification algorithm, to combine 
identical ACLs under one ID.  How will that work, and how will it be managed?

3. The performance impact of having to look somewhere else for every row 
fetched.  If you propose a cache, note that this cache has potentially one 
possible entry for every row in the database.  That would need significant 
thought and tuning.

4. Size scalability of the whole thing.  When using IDs as references is being 
proposed, somewhere in there is a total size limitation for a row-security 
enabled database.

Even if you manage to solve #2, is this cleanup feasible to run on a database 
that has run into the limits of #4?

I suppose that SELinux in the kernel addresses these issues somehow (e.g. 
caching), but what would the SQL-only solution do?

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Tom Lane

Date:

11 December 2008, 15:33:00

Peter Eisentraut <peter_e@gmx.net> writes:
> On Thursday 11 December 2008 17:09:25 Tom Lane wrote:
>> I think there should be only *one* underlying column and that it should
>> be manipulable by either SQL commands or selinux.  Otherwise you're
>> making a lie of the primary argument for having the SQL feature at all.

> Well, an SQL-manipulated row security column will probably have a content like

>     {joe=rw/bob,staff=r/bob}

> An SELinux-aware row security column will probably have a content like

>    blah_t:foo_t:quux_t

> And a Solaris TX-aware security column will probably have a content like

>    Classified

> How can we stick all of these in the same column at the same time?

Why would we want to?  I think one column that can hold any of these
ought to be sufficient.  I certainly don't care for the idea that we
might invent still a third column for Solaris TX at some future time.
        regards, tom lane

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Peter Eisentraut

Date:

11 December 2008, 15:39:04

On Thursday 11 December 2008 18:24:54 KaiGai Kohei wrote:
> Peter Eisentraut wrote:
> > On Thursday 11 December 2008 17:09:25 Tom Lane wrote:
> >> I think there should be only *one* underlying column and that it should
> >> be manipulable by either SQL commands or selinux.  Otherwise you're
> >> making a lie of the primary argument for having the SQL feature at all.
> >
> > Well, an SQL-manipulated row security column will probably have a content
> > like
> >
> >     {joe=rw/bob,staff=r/bob}
> >
> > An SELinux-aware row security column will probably have a content like
> >
> >    blah_t:foo_t:quux_t
> >
> > And a Solaris TX-aware security column will probably have a content like
> >
> >    Classified
> >
> > How can we stick all of these in the same column at the same time?
>
> To choose it on compile-time option is the most simple approach.

As mentioned before, compile-time options to choose between these variants in 
a mutually exlusive manner is not acceptable.

Plus, using two of these together, or even three, is certainly useful and 
reasonable in some uses.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

11 December 2008, 15:53:48

Peter Eisentraut wrote:
> On Thursday 11 December 2008 17:04:05 Bruce Momjian wrote:
> > The idea is that the security columns will hold an OID and the OID will
> > point to a row in a table that contains the security rights/ACL for the
> > column, with multiple rows using the same rights OID.
> 
> That sounds somewhat scary for a number of reasons:
> 
> 1. Running out of OIDs, the main reason why we got rid of OIDs in user tables 
> by default.  This would essentially put them back.
> 
> 2. You are implying some kind of ACL unification algorithm, to combine 
> identical ACLs under one ID.  How will that work, and how will it be managed?
> 
> 3. The performance impact of having to look somewhere else for every row 
> fetched.  If you propose a cache, note that this cache has potentially one 
> possible entry for every row in the database.  That would need significant 
> thought and tuning.
> 
> 4. Size scalability of the whole thing.  When using IDs as references is being 
> proposed, somewhere in there is a total size limitation for a row-security 
> enabled database.
> 
> Even if you manage to solve #2, is this cleanup feasible to run on a database 
> that has run into the limits of #4?
> 
> I suppose that SELinux in the kernel addresses these issues somehow (e.g. 
> caching), but what would the SQL-only solution do?

Agreed.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Peter Eisentraut

Date:

11 December 2008, 16:43:25

On Thursday 11 December 2008 18:32:50 Tom Lane wrote:
> > How can we stick all of these in the same column at the same time?
>
> Why would we want to?

Because we want to use SQL-based row access control and SELinux-based row 
access control at the same time.  Isn't this exactly one of the objections 
upthread?  Both must be available at the same time.

We can debate the merits of having, say, SELinux plus Solaris TX at the same 
time, but if we can have two as per previous paragraph, we should design for 
several.

> I think one column that can hold any of these 
> ought to be sufficient.  I certainly don't care for the idea that we
> might invent still a third column for Solaris TX at some future time.

Yes, it is certainly more appealing to have one column describing all access 
rights.

In fact, if we extend the ACL storage structure to store external access 
control information, we might also consider using that for system object 
access.  So instead of adding a column to pg_class for SELinux-controlled 
access to tables, we just reused relacl.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Tom Lane

Date:

11 December 2008, 17:32:36

Peter Eisentraut <peter_e@gmx.net> writes:
> On Thursday 11 December 2008 18:32:50 Tom Lane wrote:
>>> How can we stick all of these in the same column at the same time?
>> 
>> Why would we want to?

> Because we want to use SQL-based row access control and SELinux-based row 
> access control at the same time.  Isn't this exactly one of the objections 
> upthread?  Both must be available at the same time.

Well, the objection I was raising is that they should control the same
thing.  Otherwise we are simply inventing an invasive, high-cost,
nonstandard(*) feature that we have had zero field demand for.
        regards, tom lane

(*) Worse than nonstandard: it actively breaks semantics demanded by
the standard.  If I had my druthers we would flat out reject row-level
security filtering of any kind.  I don't want us to expend a lot of
effort implementing multiple kinds.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Gregory Stark

Date:

11 December 2008, 18:44:51

Peter Eisentraut <peter_e@gmx.net> writes:

> On Thursday 11 December 2008 18:32:50 Tom Lane wrote:
>> > How can we stick all of these in the same column at the same time?
>>
>> Why would we want to?
>
> Because we want to use SQL-based row access control and SELinux-based row 
> access control at the same time.  Isn't this exactly one of the objections 
> upthread?  Both must be available at the same time.

Well I don't think anyone would actually want them *at the same time*.
Combining multiple security models would mean you aren't actually following
any security model.

But I don't like the idea of making it a compile-time switch. Having to ship
separate packages for different compile-time options is really an awful
solution from the distribution's point of view. And it doesn't scale either --
if we got another such option they would have 2^n combinations.

Distributions like to set distribution-wide policies like "compile with X
support". It doesn't mean you can't run those programs without actually using
that support, as in "emacs -nw". It would be nice to have the option at
run-time of whether to use selinux or row-acl support instead.

I think we need to separate out the --enable-selinux which would merely
compile in the support for selinux from the switch to control whether we
actually have selinux turned on. Make that either an initdb option or a
per-database option like we have with collation/encoding.

Then users can install a single package and decide whether they want to use
selinux or row-acls. If their distribution decides not to compile in selinux
support they just have one choice, row-acls (or nothing).

--  Gregory Stark EnterpriseDB          http://www.enterprisedb.com Get trained by Bruce Momjian - ask me about
EnterpriseDB'sPostgreSQL training!

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Aidan Van Dyk

Date:

11 December 2008, 18:58:00

* Gregory Stark <stark@enterprisedb.com> [081211 14:47]:
> Peter Eisentraut <peter_e@gmx.net> writes:
> 
> > On Thursday 11 December 2008 18:32:50 Tom Lane wrote:
> >> > How can we stick all of these in the same column at the same time?
> >>
> >> Why would we want to?
> >
> > Because we want to use SQL-based row access control and SELinux-based row 
> > access control at the same time.  Isn't this exactly one of the objections 
> > upthread?  Both must be available at the same time.
> 
> Well I don't think anyone would actually want them *at the same time*.
> Combining multiple security models would mean you aren't actually following
> any security model.

Actually, I think people (or rather, systems) will.  No "application" is
going to want to use SE-linux, but "OS controllers" are...

Just like now, the actual people/apps/etc rely on plain unix
permissions, yet the distro still provides an SElinux policy that is
"more restrictive yet"...

Simlarly, SElinux is going to be used *on top* of any application that's
out there, to try and enfoce the "no data coming in from a secure input"
leaves through a "less secure output", irrespective of what app level
security (and in this case, app-level being the SQL/SCHEMA/row-level)
does itself...

So, if row-level access comes to PG in any sql form, apps and others
will use it (if only a few of them)...  And se-linux on top of that will
be used to try and enforce that the app hasn't made a mistake...

a.

-- 
Aidan Van Dyk                                             Create like a god,
aidan@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Peter Eisentraut

Date:

11 December 2008, 20:03:37

On Thursday 11 December 2008 20:32:25 Tom Lane wrote:
> Well, the objection I was raising is that they should control the same
> thing.  Otherwise we are simply inventing an invasive, high-cost,
> nonstandard(*) feature that we have had zero field demand for.

There is certainly a rather big field demand for row-level security.  I'm not 
sure about SELinux integration, though, or which one of the two you were 
referring to.

The trick, of course, is to make it work well.  That would usually require the 
polyinstantiation approach, and I am disappointed that that was apparently 
not chosen here.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Peter Eisentraut

Date:

11 December 2008, 20:06:09

On Thursday 11 December 2008 21:44:39 Gregory Stark wrote:
> > Because we want to use SQL-based row access control and SELinux-based row
> > access control at the same time.  Isn't this exactly one of the
> > objections upthread?  Both must be available at the same time.
>
> Well I don't think anyone would actually want them *at the same time*.
> Combining multiple security models would mean you aren't actually following
> any security model.

That doesn't follow.  Using DAC and MAC together is quite standard.  Even if
your kernel is SELinux-enabled and has a policy, you'd still want to use
normal permission bits.  Same difference here.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

11 December 2008, 22:52:31

Peter Eisentraut wrote:
> On Thursday 11 December 2008 18:24:54 KaiGai Kohei wrote:
>> Peter Eisentraut wrote:
>>> On Thursday 11 December 2008 17:09:25 Tom Lane wrote:
>>>> I think there should be only *one* underlying column and that it should
>>>> be manipulable by either SQL commands or selinux.  Otherwise you're
>>>> making a lie of the primary argument for having the SQL feature at all.
>>> Well, an SQL-manipulated row security column will probably have a content
>>> like
>>>
>>>     {joe=rw/bob,staff=r/bob}
>>>
>>> An SELinux-aware row security column will probably have a content like
>>>
>>>    blah_t:foo_t:quux_t
>>>
>>> And a Solaris TX-aware security column will probably have a content like
>>>
>>>    Classified
>>>
>>> How can we stick all of these in the same column at the same time?
>> To choose it on compile-time option is the most simple approach.
> 
> As mentioned before, compile-time options to choose between these variants in 
> a mutually exlusive manner is not acceptable.
> 
> Plus, using two of these together, or even three, is certainly useful and 
> reasonable in some uses.

Sorry, we have been deep midnight for the last several hours.

Packing two or more stuff into one field gives us several unignorable pains.
I cannot agree this approach. One field should be hold one value.
However, I found out it is an independent issue whether the feature should
be enabled/disabled on compile-time.

* A pain for user-interface

Currently, we allow users to update security context of tuples, as follows:
  UPDATE t SET security_context = 'system_u:object_r:sepgsql_table_t' WHERE a < 10;

But, if the "security_context" hold both of security context and row-level acls,
the above operation means setting a new security context and clear its acls
because it does not contain acl part.
It requires users to pay carefull mention to keep integrity of security attribute
for both security mechanism.
It is the heaviest pain which is never acceptable.

* A pain for performance

SE-PostgreSQL assigns a default security context for newly inserted tuples.
It is defined at in-kernel security policy, and preliminarily translated to
security identifier on userspace cache. SE-PostgreSQL fetch the cache entry
with O(1) steps, so we can decide what security identifier should be attached
so quickly.
Row-level ACL can assign default ACLs for newly inserted tuples. It is also
held as a security identifier, not a text representation. So, we can decide
what security identifier should be attached so quickly.

But, if two or more security mechanism shares a single data field, it is not
simple. When SE-PostgreSQL and Row-level ACL work at same time, it has to
extract these default security identifier to text format, combine it, lookup
pg_security, and assign matched security identifier.
It will give us awful pain.

* A pain for code complexity
No need to say.

-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

11 December 2008, 23:06:31

KaiGai Kohei wrote:
> Peter Eisentraut wrote:
>> On Thursday 11 December 2008 18:24:54 KaiGai Kohei wrote:
>>> Peter Eisentraut wrote:
>>>> On Thursday 11 December 2008 17:09:25 Tom Lane wrote:
>>>>> I think there should be only *one* underlying column and that it 
>>>>> should
>>>>> be manipulable by either SQL commands or selinux.  Otherwise you're
>>>>> making a lie of the primary argument for having the SQL feature at 
>>>>> all.
>>>> Well, an SQL-manipulated row security column will probably have a 
>>>> content
>>>> like
>>>>
>>>>     {joe=rw/bob,staff=r/bob}
>>>>
>>>> An SELinux-aware row security column will probably have a content like
>>>>
>>>>    blah_t:foo_t:quux_t
>>>>
>>>> And a Solaris TX-aware security column will probably have a content 
>>>> like
>>>>
>>>>    Classified
>>>>
>>>> How can we stick all of these in the same column at the same time?
>>> To choose it on compile-time option is the most simple approach.
>>
>> As mentioned before, compile-time options to choose between these 
>> variants in a mutually exlusive manner is not acceptable.
>>
>> Plus, using two of these together, or even three, is certainly useful 
>> and reasonable in some uses.
> 
> Sorry, we have been deep midnight for the last several hours.
> 
> Packing two or more stuff into one field gives us several unignorable 
> pains.
> I cannot agree this approach. One field should be hold one value.
> However, I found out it is an independent issue whether the feature should
> be enabled/disabled on compile-time.

If we cannot agree the compile-time enable/disable approach, I can prepare
to suggest a compromise draft.

This idea allows to compile two or more security mechanism in the same binary,
and adds a configuration parameter to choose a security mechanism on its startup
time. So, a single security mechanism chosen works in same time, but multiple
security mechanisms are built in compile time.
  For example, at $PGDATA/postgresql.conf    pgace_security = selinux    # for SE-PostgreSQL    pgace_security = rowacl
   # for row-level acl    pgace_security = solaristx  # for upcoming Trusted Solaris?

As Peter mentioned before, the reason why compile-time enable/disable approach
is not recommendable is packaging issue. It requires to deliver a single version
of binary package as far as possible. This idea does not conflict to the policy.

Again, I cannot think it is a good idea to pack several values into a field.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

11 December 2008, 23:18:41

Peter Eisentraut wrote:
> On Thursday 11 December 2008 18:32:50 Tom Lane wrote:
>>> How can we stick all of these in the same column at the same time?
>> Why would we want to?
> 
> Because we want to use SQL-based row access control and SELinux-based row 
> access control at the same time.  Isn't this exactly one of the objections 
> upthread?  Both must be available at the same time.

Please make clear the meaning of "use".
As you said, if your concern is based on packaging/distributing issue,
I suggested an alternative proposal which allows to compile multiple
security mechanism and to choose one of them on runtime.

> We can debate the merits of having, say, SELinux plus Solaris TX at the same 
> time, but if we can have two as per previous paragraph, we should design for 
> several.

What platform is available for both of SELinux and Solaris TX?
I think it is exactly compile-time issue.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

11 December 2008, 23:33:17

Aidan Van Dyk wrote:
> Simlarly, SElinux is going to be used *on top* of any application that's
> out there, to try and enfoce the "no data coming in from a secure input"
> leaves through a "less secure output", irrespective of what app level
> security (and in this case, app-level being the SQL/SCHEMA/row-level)
> does itself...

It is incorrect.
SELinux works as a security server which provides access control decisions
for other subsystems. In this model, the kernel is also considered as one
of the subsystems.

Currently, X-window system has SELinux support because it manages window
objects in userspace, and we can use them as a method to communicate
other processes. (Please imagine copy&paste buffer.)

This slide will help your understand: http://selinux-symposium.org/2007/slides/03-xorg.pdf

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

11 December 2008, 23:38:22

Peter Eisentraut wrote:
> On Thursday 11 December 2008 20:32:25 Tom Lane wrote:
>> Well, the objection I was raising is that they should control the same
>> thing.  Otherwise we are simply inventing an invasive, high-cost,
>> nonstandard(*) feature that we have had zero field demand for.
> 
> There is certainly a rather big field demand for row-level security.  I'm not 
> sure about SELinux integration, though, or which one of the two you were 
> referring to.

SELinux integration is also strong demand, no need to say.

> The trick, of course, is to make it work well.  That would usually require the 
> polyinstantiation approach, and I am disappointed that that was apparently 
> not chosen here.

As we discussed before, I don't have a plan to implement polyinstantiation
feature. It is too much requirement for enterprise class system, and can be
estimate its implementation so complex.
(Please note that major commercial RDBMSs don't implement it also.)

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

11 December 2008, 23:43:01

Gregory Stark wrote:
> Peter Eisentraut <peter_e@gmx.net> writes:
> 
>> On Thursday 11 December 2008 18:32:50 Tom Lane wrote:
>>>> How can we stick all of these in the same column at the same time?
>>> Why would we want to?
>> Because we want to use SQL-based row access control and SELinux-based row 
>> access control at the same time.  Isn't this exactly one of the objections 
>> upthread?  Both must be available at the same time.
> 
> Well I don't think anyone would actually want them *at the same time*.
> Combining multiple security models would mean you aren't actually following
> any security model.
> 
> But I don't like the idea of making it a compile-time switch. Having to ship
> separate packages for different compile-time options is really an awful
> solution from the distribution's point of view. And it doesn't scale either --
> if we got another such option they would have 2^n combinations.
> 
> Distributions like to set distribution-wide policies like "compile with X
> support". It doesn't mean you can't run those programs without actually using
> that support, as in "emacs -nw". It would be nice to have the option at
> run-time of whether to use selinux or row-acl support instead.
> 
> I think we need to separate out the --enable-selinux which would merely
> compile in the support for selinux from the switch to control whether we
> actually have selinux turned on. Make that either an initdb option or a
> per-database option like we have with collation/encoding.
> 
> Then users can install a single package and decide whether they want to use
> selinux or row-acls. If their distribution decides not to compile in selinux
> support they just have one choice, row-acls (or nothing).

I agree the opinion.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

12 December 2008, 00:20:33

KaiGai Kohei wrote:
> Gregory Stark wrote:
>> Peter Eisentraut <peter_e@gmx.net> writes:
>>
>>> On Thursday 11 December 2008 18:32:50 Tom Lane wrote:
>>>>> How can we stick all of these in the same column at the same time?
>>>> Why would we want to?
>>> Because we want to use SQL-based row access control and SELinux-based 
>>> row access control at the same time.  Isn't this exactly one of the 
>>> objections upthread?  Both must be available at the same time.
>>
>> Well I don't think anyone would actually want them *at the same time*.
>> Combining multiple security models would mean you aren't actually 
>> following
>> any security model.
>>
>> But I don't like the idea of making it a compile-time switch. Having 
>> to ship
>> separate packages for different compile-time options is really an awful
>> solution from the distribution's point of view. And it doesn't scale 
>> either --
>> if we got another such option they would have 2^n combinations.
>>
>> Distributions like to set distribution-wide policies like "compile with X
>> support". It doesn't mean you can't run those programs without 
>> actually using
>> that support, as in "emacs -nw". It would be nice to have the option at
>> run-time of whether to use selinux or row-acl support instead.
>>
>> I think we need to separate out the --enable-selinux which would merely
>> compile in the support for selinux from the switch to control whether we
>> actually have selinux turned on. Make that either an initdb option or a
>> per-database option like we have with collation/encoding.

I prefer an initdb option, because SE-PostgreSQL assigns its security
context for inserted tuples during bootstraping mode also, so we need
to make clear what security mechanism should be activate on initdb time
at least.
It will be possible to switch via $PGDATA/postgresql.conf. I assume
this change should not be happen so frequently.

Proposed initdb option, like:  $ initdb --pgace-security=(selinux|rowacl)

And, the setting is written out to $PGDATA/postgresql.conf

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

12 December 2008, 01:00:20

I understood objections for "enable/disable something on compile-time"
approach. The following items are my revised proposal.
It does not conflicts to the policy Peter said before in this thread.

Items to be revised:

- It allows to compile multiple security mechanis within a single binary.

- It provides "--enable-selinux" configure option to build SELinux support,  but the Row-level ACLs is always built
becauseit does not require any  platform specific libraries and so on.
 

- User choose one (or no) security mechanism on initdb and postmaster  starting up via $PGDATA/postgresql.conf. The
currentavailable options  are selinux, rowacl and nothing.  (*) What should be the default? rowacl? or, nothing?
 
  The modified initdb interface is as follows:    $ initdb --pgace-security=(selinux|rowacl|nothing)

- It add a new system column for security purpose. It enables to store  a security attribute of tuple, so working
securitymechanism has to  handle it properly. For example, SE-PostgreSQL handles a tuple without  valid security
contextas a "unlabeled_t" tuple.
 

- The security mechanism can control whether the new tuple has its  security field, or not. If no security field is
necessary,t_infomask  is not set HEAP_HASSECURITY bit, and it does not consume any additional  storage space.
 

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

12 December 2008, 01:04:52

KaiGai Kohei wrote:
> I understood objections for "enable/disable something on compile-time"
> approach. The following items are my revised proposal.
> It does not conflicts to the policy Peter said before in this thread.
> 
> Items to be revised:
> 
> - It allows to compile multiple security mechanis within a single binary.
> 
> - It provides "--enable-selinux" configure option to build SELinux support,
>   but the Row-level ACLs is always built because it does not require any
>   platform specific libraries and so on.
> 
> - User choose one (or no) security mechanism on initdb and postmaster
>   starting up via $PGDATA/postgresql.conf. The current available options
>   are selinux, rowacl and nothing.
>   (*) What should be the default? rowacl? or, nothing?
> 
>   The modified initdb interface is as follows:
>     $ initdb --pgace-security=(selinux|rowacl|nothing)
> 
> - It add a new system column for security purpose. It enables to store
>   a security attribute of tuple, so working security mechanism has to
>   handle it properly. For example, SE-PostgreSQL handles a tuple without
>   valid security context as a "unlabeled_t" tuple.

In addition,

- The new security conlumn has common name for each security mechanisms.  Candidates are "security_attr",
"security_attribute","sec_attr" or  "security". At first, I start it with "security_attr".
 

> - The security mechanism can control whether the new tuple has its
>   security field, or not. If no security field is necessary, t_infomask
>   is not set HEAP_HASSECURITY bit, and it does not consume any additional
>   storage space.
> 
> Thanks,


-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

"Robert Haas"

Date:

12 December 2008, 02:12:00

> This idea allows to compile two or more security mechanism in the same binary,
> and adds a configuration parameter to choose a security mechanism on its startup
> time. So, a single security mechanism chosen works in same time, but multiple
> security mechanisms are built in compile time.

This is a good idea.

> Again, I cannot think it is a good idea to pack several values into a field.

I don't either.  I think we need two fields.  I can't imagine anyone
making a serious argument that we need to simultaneously support more
than one MAC system: you pick EITHER SELinux or Trusted Solaris, not
both.

But I can sure imagine someone wanting both MAC and DAC.

...Robert

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

12 December 2008, 02:13:40

Peter Eisentraut wrote:
> On Thursday 11 December 2008 20:32:25 Tom Lane wrote:
> > Well, the objection I was raising is that they should control the same
> > thing.  Otherwise we are simply inventing an invasive, high-cost,
> > nonstandard(*) feature that we have had zero field demand for.
> 
> There is certainly a rather big field demand for row-level security.  I'm not 
> sure about SELinux integration, though, or which one of the two you were 
> referring to.
> 
> The trick, of course, is to make it work well.  That would usually require the 
> polyinstantiation approach, and I am disappointed that that was apparently 
> not chosen here.

Polyinstantiation was just too complex to do at this point:
http://en.wikipedia.org/wiki/Polyinstantiation

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

12 December 2008, 02:51:13

In general, I am concerned that the SE-Linux patch is not converging on
a community consensus in terms of user interface or implementation
details.  This suggests that the patch might not succeed in being
included in Postgres 8.4, which is a shame.

We are going to need to come up with specific answers to these issues
soon.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

12 December 2008, 02:59:20

We have had discussion on whether we want one or two security columns; 
there have been comments on both sides.

Have we decided if we are going to use some type of integer on every row
that points to a pg_security row or put the value right in the row?

If we use some type of integer, I suggest using this structure for
pg_security:
CREATE TABLE pg_security(    relid oid,     secid int2,     secacl aclitem[],     secext TEXT);

This allows the per-row value to be a simple int2.  It also improves
maintenance because rows are associated only with a specific table;
unused values can then be removed more easily.  And it allows both
secacl and secext security to be specified.

I am unsure how an insert into a 'security_context' column would
automatically insert into pg_security however.  I am also unclear how
COPY would work.

Also, having the per-row value always be present in the row and
controlled by the bitmask seems ideal;  it avoids having to add a CREATE
TABLE option.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

12 December 2008, 03:46:42

Bruce Momjian wrote:
> We have had discussion on whether we want one or two security columns; 
> there have been comments on both sides.

It is quite natural end-user should be able to choose one of provided
security mechanisms, in my opinion. It means user (administrator) allows
his prefered security mechanism to use security column and security field.

It makes the implementation keep simple, and avoid some of pains.

How frequently does someone want to *work* (not compile) DAC and MAC
in same time? I could not believe it is a generic demand.
I expect that a user who explicitly enables SE-PostgreSQL by hand,
he focuses on MAC feature.

> Have we decided if we are going to use some type of integer on every row
> that points to a pg_security row or put the value right in the row?

The reason why I decided to put an integer value on HeapTupleHeader,
as "oid" doing, is that it has to be modified after simple_heap_insert()
or ExecInsert(). It has to be fixed length and not a user data.

> If we use some type of integer, I suggest using this structure for
> pg_security:
> 
>     CREATE TABLE pg_security(
>         relid oid, 
>         secid int2, 
>         secacl aclitem[], 
>         secext TEXT
>     );
> 
> This allows the per-row value to be a simple int2.  It also improves
> maintenance because rows are associated only with a specific table;
> unused values can then be removed more easily.  And it allows both
> secacl and secext security to be specified.

How does the approach resolve the pain of user interface?
I don't think packing two or more values into one field is not a right way.

> I am unsure how an insert into a 'security_context' column would
> automatically insert into pg_security however.  I am also unclear how
> COPY would work.

When a pgaceHeapTupleInsert() hook is invoked without specific security
context, SE-PostgreSQL does... 1. Compute a default security context. It ask for the security policy,    if necessary.
2.It lookup pg_security via SysCache. If the required text representation    is on the pg_security, SE-PostgreSQL put
itsoid as the security    identifier of given tuple.    (pg_security.oid is primary key.) 3. When the finding text is
notfound, it insert a new tuple into    pg_security, and put its oid as the security identifier of given tuple.

COPY statement also invokes heap_insert(), so SE-PostgreSQL assign
a proper security context in same way.

> Also, having the per-row value always be present in the row and
> controlled by the bitmask seems ideal;  it avoids having to add a CREATE
> TABLE option.

Sorry, I don't understand why it related to a CREATE TABLE option.
System columns are always allocated for any tables?

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

12 December 2008, 04:00:27

Bruce Momjian wrote:
> In general, I am concerned that the SE-Linux patch is not converging on
> a community consensus in terms of user interface or implementation
> details.  This suggests that the patch might not succeed in being
> included in Postgres 8.4, which is a shame.

It is extreamly worst.

I have worked for this feature more than 2 years, and has waited for
opening the merge window of v8.4, because it was just after the feature
freeze of v8.3 when I first proposed SE-PostgreSQL on the list.
(I was not good at development process in the community.)
Yes, we got active discussion recent days which I waited for long term.
I don't want to wait for any more!

> We are going to need to come up with specific answers to these issues
> soon.

The origion of issue is simple.

Whether we should support to activate (not only compile) two or more security
mechanism in same time, or not.
In my opinion, it is not a frequent situation, and it gives us several big
pains, but benefit is smaller than the pains.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

12 December 2008, 04:25:37

KaiGai Kohei wrote:
> Bruce Momjian wrote:
> > In general, I am concerned that the SE-Linux patch is not converging on
> > a community consensus in terms of user interface or implementation
> > details.  This suggests that the patch might not succeed in being
> > included in Postgres 8.4, which is a shame.
> 
> It is extreamly worst.

Yes, agreed.

> I have worked for this feature more than 2 years, and has waited for
> opening the merge window of v8.4, because it was just after the feature
> freeze of v8.3 when I first proposed SE-PostgreSQL on the list.
> (I was not good at development process in the community.)
> Yes, we got active discussion recent days which I waited for long term.
> I don't want to wait for any more!
> 
> > We are going to need to come up with specific answers to these issues
> > soon.
> 
> The origion of issue is simple.
> 
> Whether we should support to activate (not only compile) two or more security
> mechanism in same time, or not.
> In my opinion, it is not a frequent situation, and it gives us several big
> pains, but benefit is smaller than the pains.

Yes.  My concern is what other issues are going to come that we have not
thought of yet.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

12 December 2008, 04:36:38

KaiGai Kohei wrote:
> Bruce Momjian wrote:
> > We have had discussion on whether we want one or two security columns; 
> > there have been comments on both sides.
> 
> It is quite natural end-user should be able to choose one of provided
> security mechanisms, in my opinion. It means user (administrator) allows
> his prefered security mechanism to use security column and security field.
> 
> It makes the implementation keep simple, and avoid some of pains.

Agreed.  I think if the community wants too many features we will not
get this done in time for 8.4.

> How frequently does someone want to *work* (not compile) DAC and MAC
> in same time? I could not believe it is a generic demand.
> I expect that a user who explicitly enables SE-PostgreSQL by hand,
> he focuses on MAC feature.
> 
> > Have we decided if we are going to use some type of integer on every row
> > that points to a pg_security row or put the value right in the row?
> 
> The reason why I decided to put an integer value on HeapTupleHeader,
> as "oid" doing, is that it has to be modified after simple_heap_insert()
> or ExecInsert(). It has to be fixed length and not a user data.

Ah, so you found a way so the column can be optionally used and does not
have to be specified for the entire table using a CREATE TABLE option? 
Great.

> > If we use some type of integer, I suggest using this structure for
> > pg_security:
> > 
> >     CREATE TABLE pg_security(
> >         relid oid, 
> >         secid int2, 
> >         secacl aclitem[], 
> >         secext TEXT
> >     );
> > 
> > This allows the per-row value to be a simple int2.  It also improves
> > maintenance because rows are associated only with a specific table;
> > unused values can then be removed more easily.  And it allows both
> > secacl and secext security to be specified.
> 
> How does the approach resolve the pain of user interface?
> I don't think packing two or more values into one field is not a right way.

Well, having relid allows us to use a smaller int2 and allows us to more
easily remove pg_security rows that are no longer referenced in the user
table.  It is just an idea.

Perhaps the idea of two security columns is too much and we should
consider that in a later release if we find we need it.  Shipping 8.4
with only one active security mechanism at a time is certainly
acceptable and wise, or we might get nothing in 8.4.
> 
> > I am unsure how an insert into a 'security_context' column would
> > automatically insert into pg_security however.  I am also unclear how
> > COPY would work.
> 
> When a pgaceHeapTupleInsert() hook is invoked without specific security
> context, SE-PostgreSQL does...
>   1. Compute a default security context. It ask for the security policy,
>      if necessary.
>   2. It lookup pg_security via SysCache. If the required text representation
>      is on the pg_security, SE-PostgreSQL put its oid as the security
>      identifier of given tuple.
>      (pg_security.oid is primary key.)
>   3. When the finding text is not found, it insert a new tuple into
>      pg_security, and put its oid as the security identifier of given tuple.

Nice, I did not realize you had coded that already.  Good approach.

> COPY statement also invokes heap_insert(), so SE-PostgreSQL assign
> a proper security context in same way.

OK, good.

> > Also, having the per-row value always be present in the row and
> > controlled by the bitmask seems ideal;  it avoids having to add a CREATE
> > TABLE option.
> 
> Sorry, I don't understand why it related to a CREATE TABLE option.
> System columns are always allocated for any tables?

Does a table use storage for the security column if no SQL-level
security value is supplied for a given row?

Also what patch version should I be looking at.  How will that version
need to be changed?

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

12 December 2008, 05:09:56

Bruce Momjian wrote:
>>> Have we decided if we are going to use some type of integer on every row
>>> that points to a pg_security row or put the value right in the row?
>> The reason why I decided to put an integer value on HeapTupleHeader,
>> as "oid" doing, is that it has to be modified after simple_heap_insert()
>> or ExecInsert(). It has to be fixed length and not a user data.
> 
> Ah, so you found a way so the column can be optionally used and does not
> have to be specified for the entire table using a CREATE TABLE option? 
> Great.

The security system column is always generated.
But it is independent from whether the tuple allocate its security field
on the HeapTupleHeader, or not.

The Row-level ACL stores a flag within pg_class.reloptions, and it shows
whether the Roe-level ACL is available, or not.
When we refer the security system column on the table being unavailable,
it returns a dummy data (empty text) because tuples does not have security
field.

SE-PostgreSQL does not allow owner to control it per-table due to its
security design.

Is it appropriate for the answer?

>>> Also, having the per-row value always be present in the row and
>>> controlled by the bitmask seems ideal;  it avoids having to add a CREATE
>>> TABLE option.
>> Sorry, I don't understand why it related to a CREATE TABLE option.
>> System columns are always allocated for any tables?
> 
> Does a table use storage for the security column if no SQL-level
> security value is supplied for a given row?

When Row-level ACL is enabled on the table and user suppies a tuple
without any specific ACLs, it requires security field, because the
length of HeapTuple is decided at heap_form_tuple() which is invoked
prior to fetching the user supplied ACLs.

When Row-level ACL is disabled (by pg_class.reloptions) on the table,
the storage for security field is not necessary.

> Also what patch version should I be looking at.  How will that version
> need to be changed?

As I promised, the patch ready for reviewing must allow to compile multiple
security mechanisms, and to choose one of them on startup.
(The current latest r1280 use approach to enable/disable on compile-time.)

Could you wait for a few days to provide newer version.

The following list is the items to be revised:  http://archives.postgresql.org/message-id/4941C690.60803@ak.jp.nec.com

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Peter Eisentraut

Date:

12 December 2008, 06:40:18

KaiGai Kohei wrote:
> Peter Eisentraut wrote:
>> On Thursday 11 December 2008 18:32:50 Tom Lane wrote:
>>>> How can we stick all of these in the same column at the same time?
>>> Why would we want to?
>>
>> Because we want to use SQL-based row access control and SELinux-based 
>> row access control at the same time.  Isn't this exactly one of the 
>> objections upthread?  Both must be available at the same time.
> 
> Please make clear the meaning of "use".
> As you said, if your concern is based on packaging/distributing issue,
> I suggested an alternative proposal which allows to compile multiple
> security mechanism and to choose one of them on runtime.

I would like to be able to assign SQL-level ACLs and SELinux labels to 
the same row at the same time in the same build, and have the system 
enforce both on top of each other.

In my mind, this is completely analogous to being able to assign 
SQL-level ACLs and SELinux labels to the same 
table/function/tablespace/etc. at the same time, and I think that 
behavior is undisputed.

>> We can debate the merits of having, say, SELinux plus Solaris TX at 
>> the same time, but if we can have two as per previous paragraph, we 
>> should design for several.
> 
> What platform is available for both of SELinux and Solaris TX?

Well, Solaris, if you believe various rumours.  I agree the case for 
this might be weak, though.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Peter Eisentraut

Date:

12 December 2008, 06:54:21

Bruce Momjian wrote:
> Have we decided if we are going to use some type of integer on every row
> that points to a pg_security row or put the value right in the row?

This would effectively create a limit on the number of rows per table.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Peter Eisentraut

Date:

12 December 2008, 06:56:09

KaiGai Kohei wrote:
> How frequently does someone want to *work* (not compile) DAC and MAC
> in same time?

My expectation would be Always.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Peter Eisentraut

Date:

12 December 2008, 07:04:17

KaiGai Kohei wrote:
>> We are going to need to come up with specific answers to these issues
>> soon.
> 
> The origion of issue is simple.
> 
> Whether we should support to activate (not only compile) two or more 
> security
> mechanism in same time, or not.
> In my opinion, it is not a frequent situation, and it gives us several big
> pains, but benefit is smaller than the pains.

With all respect, you were asked to divide up the issues so we don't 
have to deal with them all at once.

For instance, a separate patch that implements SQL-level row level 
security would be fairly uncontroversial and issue-free at this point, 
but it would be completely useful on its own and it would build 
confidence in the developer community about your other plans.

Most committers have expressed the viewpoint in one way or another that 
having this available is a prerequisite for accepting further work. 
Yet, I am not aware of even an interface proposal for this.

Meanwhile, we are busy worrying about what system columns the follow-up 
features will have.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

12 December 2008, 07:18:45

Peter Eisentraut wrote:
> KaiGai Kohei wrote:
>> Peter Eisentraut wrote:
>>> On Thursday 11 December 2008 18:32:50 Tom Lane wrote:
>>>>> How can we stick all of these in the same column at the same time?
>>>> Why would we want to?
>>>
>>> Because we want to use SQL-based row access control and SELinux-based 
>>> row access control at the same time.  Isn't this exactly one of the 
>>> objections upthread?  Both must be available at the same time.
>>
>> Please make clear the meaning of "use".
>> As you said, if your concern is based on packaging/distributing issue,
>> I suggested an alternative proposal which allows to compile multiple
>> security mechanism and to choose one of them on runtime.
> 
> I would like to be able to assign SQL-level ACLs and SELinux labels to 
> the same row at the same time in the same build, and have the system 
> enforce both on top of each other.

In my opinion, it makes more pains (user-interface, performance, complexity
of implementation and so on) than its benefit which allows to support MAC
and DAC concurrently.


>>> We can debate the merits of having, say, SELinux plus Solaris TX at 
>>> the same time, but if we can have two as per previous paragraph, we 
>>> should design for several.
>>
>> What platform is available for both of SELinux and Solaris TX?
> 
> Well, Solaris, if you believe various rumours.  I agree the case for 
> this might be weak, though.

Are you saying about Solaris FMAC project?
It is a different platform from Trusted Solaris.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Peter Eisentraut

Date:

12 December 2008, 12:10:48

KaiGai Kohei wrote:
>> I would like to be able to assign SQL-level ACLs and SELinux labels to 
>> the same row at the same time in the same build, and have the system 
>> enforce both on top of each other.
> 
> In my opinion, it makes more pains (user-interface, performance, complexity
> of implementation and so on) than its benefit which allows to support MAC
> and DAC concurrently.

I am a bit surprised.  I'd consider the ability to do DAC and MAC 
concurrently to be absolutely essential, for several reasons:

1. DAC is managed by users, MAC by administrators.

2. They address different but concurrent use cases.

3. Transitioning to MAC will be a lot easier if it doesn't require you 
to drop the DAC configuration all at once.

4. You don't propose to drop table ACLs if you enable SELinux, do you? 
Same issue.

5. It's possible!

>>>> We can debate the merits of having, say, SELinux plus Solaris TX at 
>>>> the same time, but if we can have two as per previous paragraph, we 
>>>> should design for several.
>>>
>>> What platform is available for both of SELinux and Solaris TX?
>>
>> Well, Solaris, if you believe various rumours.  I agree the case for 
>> this might be weak, though.
> 
> Are you saying about Solaris FMAC project?
> It is a different platform from Trusted Solaris.

Trusted Solaris was a separate fork of Solaris, which is now legacy. 
The current thing is Solaris Trusted Extensions (TX), which is 
integrated in normal Solaris.  So when the FMAC project produces 
something, it should conceivably be available in parallel to the current 
TX stuff.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Zeugswetter Andreas OSB sIT

Date:

12 December 2008, 12:22:00

> If we use some type of integer, I suggest using this structure for
> pg_security:
>
>     CREATE TABLE pg_security(
>         relid oid,
>         secid int2,
>         secacl aclitem[],
>         secext TEXT
>     );
>
> This allows the per-row value to be a simple int2.  It also improves
> maintenance because rows are associated only with a specific table;
> unused values can then be removed more easily.  And it allows both
> secacl and secext security to be specified.

I do not expect that the number of unique combinations of "rights"
strongly varies between the tables. Thus I think creating pg_security rows per table
would vastly increase the size of pg_security.
The expected size of pg_security is small in the current implementation.

Example: security_context = "top_secret_t"
With above schema you need one row in pg_security for each table that has "top_secret_t" rows.
The current implementation only needs one row for this, which is imho better.

CREATE TABLE pg_security(    secid serial,     secacl aclitem[],     secext TEXT);

May be ok, but I am with KaiGai, that it is not obvious how to update the
security context syntactically when using 2 subsystems simultaneously.
But using, restricting and selecting is easy.

Andreas

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

12 December 2008, 13:59:07

Peter Eisentraut wrote:
> KaiGai Kohei wrote:
>>> I would like to be able to assign SQL-level ACLs and SELinux labels 
>>> to the same row at the same time in the same build, and have the 
>>> system enforce both on top of each other.
>>
>> In my opinion, it makes more pains (user-interface, performance, 
>> complexity
>> of implementation and so on) than its benefit which allows to support MAC
>> and DAC concurrently.
> 
> I am a bit surprised.  I'd consider the ability to do DAC and MAC 
> concurrently to be absolutely essential, for several reasons:
> 
> 1. DAC is managed by users, MAC by administrators.
> 
> 2. They address different but concurrent use cases.

Yes,
Please note that I don't say it is something worthless.
However, when we implement it with a single security system column,
its demerit is unacceptable.

> 3. Transitioning to MAC will be a lot easier if it doesn't require you 
> to drop the DAC configuration all at once.
> 
> 4. You don't propose to drop table ACLs if you enable SELinux, do you? 
> Same issue.

Yes, management of security attribute is a major work of security features.
Thus, when we switch the active security feature, it does not managed
correctly. Please imagine what is happen when we remount ext3 filesystem
without "acl" flag, or we boot a SELinux'ed system with selinux=0.

> 5. It's possible!

Indeed, it is possible, but it is fact we also have some of trade-offs.
The biggest matter is user-interfaces to modify security attribute is
unclear.

If we have two system column, "security_acl" for DAC and "security_label"
for MAC, it allows to implement the feature without remarkable pains.
Well, we can simply update the "security_acl" to update per tuple ACLs.
However, again, we return to start of the discussion.

Tom said:> Wait a minute.  The original argument for providing SQL-driven row level> security was that it would help
providea framework for testing the code> and doing something useful with it on non-selinux platforms.

I think it is not a issue which has perfect answer, so we need to decide
a way to implement it. My preference is 1 security system column and
1 security feature design.

I tried to summarize the proposed options, as follows:

o : merit    x : demerit    X : unacceptable demerit

* 1 security system column, 1 security feature (DAC or MAC) o It suitable for a single security system column
implementation.x If a user want to use both of DAC and MAC concurrently, he has   to choose one of them. o It allows
allthe security feature on the common framework,   suitable for the original Row-level ACLs purpose.

* 2 security system column, 2 security feature (DAC and MAC) o It allows both of DAC and MAC consurrently, without
remarkable  regressions. x It needs two new security system columns. x What is the purpose of the Row-level security in
original?

* 1 security system column, 2 security feature X It gives us catastrophic regression in user-interface, performance
andcode complexity. Its merit is trivial compared to its demerit.

--
KaiGai Kohei

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

"Robert Haas"

Date:

12 December 2008, 15:37:08

> I tried to summarize the proposed options, as follows:
>
> o : merit    x : demerit    X : unacceptable demerit
>
> * 1 security system column, 1 security feature (DAC or MAC)
>  o It suitable for a single security system column implementation.
>  x If a user want to use both of DAC and MAC concurrently, he has
>   to choose one of them.
>  o It allows all the security feature on the common framework,
>   suitable for the original Row-level ACLs purpose.
>
> * 2 security system column, 2 security feature (DAC and MAC)
>  o It allows both of DAC and MAC consurrently, without remarkable
>   regressions.
>  x It needs two new security system columns.
>  x What is the purpose of the Row-level security in original?
>
> * 1 security system column, 2 security feature
>  X It gives us catastrophic regression in user-interface, performance
>   and code complexity. Its merit is trivial compared to its demerit.

Obviously sandwhiching two values in one column is not going to work.
The only question here is whether it's important to simultaneously
support both DAC and MAC.  As far as I can see, KaiGai is the only one
arguing that we don't need to do that (except for Tom, who doesn't
like either feature).  If anyone else agrees with his position, now
would be a good time to speak up.

Peter made an excellent point a few emails upthread: there seemed to
be consensus in the September CommitFest that we needed SQL-level
support for row and column level security before we talked about
implementing those features as part of SELinux.  I don't see that
we're any closer to that goal than we were then.  There has been some
progress made on column-level permissions, but the patch is back in
"waiting for author" limbo, and the only alternatives for SQL-level
row-level permissions is to have them INSTEAD OF SELinux-based
row-level permissions.  That's not the same thing at all, and I think
it's also the underlying reason behind Bruce's complaint here:

http://archives.postgresql.org/pgsql-hackers/2008-12/msg00863.php

...Robert

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Gregory Stark

Date:

12 December 2008, 15:57:40

"Robert Haas" <robertmhaas@gmail.com> writes:

> Obviously sandwhiching two values in one column is not going to work.
> The only question here is whether it's important to simultaneously
> support both DAC and MAC.  As far as I can see, KaiGai is the only one
> arguing that we don't need to do that (except for Tom, who doesn't
> like either feature).  If anyone else agrees with his position, now
> would be a good time to speak up.

Well maybe I'm not following. I think it's strange to need two different
row-based security methods. Can you give some examples of how you would use
these two mechanisms together?

--  Gregory Stark EnterpriseDB          http://www.enterprisedb.com Ask me about EnterpriseDB's PostGIS support!

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Alvaro Herrera

Date:

12 December 2008, 16:09:56

Robert Haas escribió:

> Peter made an excellent point a few emails upthread: there seemed to
> be consensus in the September CommitFest that we needed SQL-level
> support for row and column level security before we talked about
> implementing those features as part of SELinux.  I don't see that
> we're any closer to that goal than we were then.  There has been some
> progress made on column-level permissions, but the patch is back in
> "waiting for author" limbo, and the only alternatives for SQL-level
> row-level permissions is to have them INSTEAD OF SELinux-based
> row-level permissions.

I don't understand -- why wouldn't we just have two columns, one for
plain row-level security and another for whatever security system the
platforms happens to offer?  If we were to follow that route, we could
have row-level security first, extracting the feature from the current
patch; and the rest of PGACE could be a much smaller patch implementing
the rest of the stuff, with SELinux support for now with an eye to
implementing Solaris TX or whatever.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

"Robert Haas"

Date:

12 December 2008, 16:20:47

>> Peter made an excellent point a few emails upthread: there seemed to
>> be consensus in the September CommitFest that we needed SQL-level
>> support for row and column level security before we talked about
>> implementing those features as part of SELinux.  I don't see that
>> we're any closer to that goal than we were then.  There has been some
>> progress made on column-level permissions, but the patch is back in
>> "waiting for author" limbo, and the only alternatives for SQL-level
>> row-level permissions is to have them INSTEAD OF SELinux-based
>> row-level permissions.
>
> I don't understand -- why wouldn't we just have two columns, one for
> plain row-level security and another for whatever security system the
> platforms happens to offer?  If we were to follow that route, we could
> have row-level security first, extracting the feature from the current
> patch; and the rest of PGACE could be a much smaller patch implementing
> the rest of the stuff, with SELinux support for now with an eye to
> implementing Solaris TX or whatever.

Well, I think we should do exactly what you're proposing, so don't ask me.

...Robert

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

"Robert Haas"

Date:

12 December 2008, 16:31:17

On Fri, Dec 12, 2008 at 11:57 AM, Gregory Stark <stark@enterprisedb.com> wrote:
>> Obviously sandwhiching two values in one column is not going to work.
>> The only question here is whether it's important to simultaneously
>> support both DAC and MAC.  As far as I can see, KaiGai is the only one
>> arguing that we don't need to do that (except for Tom, who doesn't
>> like either feature).  If anyone else agrees with his position, now
>> would be a good time to speak up.
>
> Well maybe I'm not following. I think it's strange to need two different
> row-based security methods. Can you give some examples of how you would use
> these two mechanisms together?

Not really.  I'm not an SELinux expert.  But typically the two do
exist alongside one another.  For example, installing SELinux (MAC)
does on your system does not make "chmod g+w file" (DAC) stop working;
it merely performs an ADDITIONAL security check before allowing access
to the file.  You have to satisfy BOTH SELinux AND the ordinary
filesystem permissions system in order to perform an operation on a
file.

Similarly, we have SQL statements "GRANT {SELECT|INSERT|DELETE|etc} ON
table..." and hopefully soon a column-level variant of that same
syntax.  Those permissions aren't going to be ignored just because you
also enable SELinux to control access to those tables or columns.  Or
at least I certainly hope they aren't.

The contention of the author of this patch is that row-level access is
somehow different - that even though we have two sets of checks for
files, tables, and (assuming Stephen Frost's patch is accepted)
columns, we will only have one set of checks for rows, and you can
pick which one you want.

...Robert

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

12 December 2008, 16:58:07

KaiGai Kohei wrote:
> > If we use some type of integer, I suggest using this structure for
> > pg_security:
> > 
> >     CREATE TABLE pg_security(
> >         relid oid, 
> >         secid int2, 
> >         secacl aclitem[], 
> >         secext TEXT
> >     );
> > 
> > This allows the per-row value to be a simple int2.  It also improves
> > maintenance because rows are associated only with a specific table;
> > unused values can then be removed more easily.  And it allows both
> > secacl and secext security to be specified.
> 
> How does the approach resolve the pain of user interface?
> I don't think packing two or more values into one field is not a right way.

I see later emails that say we have to have both security methods
available at the same time, and the table above does that.  There would
be one security oid on every row and it would refer to this table.  

pg_security would contain every _unique_ combination of secacl and
secext. On INSERT the code looks to see if the secacl/secext exists in
pg_security, and if it does it reuses the same oid, if not it adds a new
row.  (There is no method for cleaning out unreferenced pg_security rows
(relid was supposed to help with that but it also bloats pg_security)).

Some people didn't like it was per-table so I removed the relid column:
CREATE TABLE pg_security(    secid oid,     secacl aclitem[],     secext TEXT);

pg_dump and COPY would dump the per-row oid and pg_security so there
should be only one flag to dump security info, even though it supports
two security methods.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Bruce Momjian

Date:

12 December 2008, 16:59:57

KaiGai Kohei wrote:
> >>> Also, having the per-row value always be present in the row and
> >>> controlled by the bitmask seems ideal;  it avoids having to add a CREATE
> >>> TABLE option.
> >> Sorry, I don't understand why it related to a CREATE TABLE option.
> >> System columns are always allocated for any tables?
> > 
> > Does a table use storage for the security column if no SQL-level
> > security value is supplied for a given row?
> 
> When Row-level ACL is enabled on the table and user suppies a tuple
> without any specific ACLs, it requires security field, because the
> length of HeapTuple is decided at heap_form_tuple() which is invoked
> prior to fetching the user supplied ACLs.
> 
> When Row-level ACL is disabled (by pg_class.reloptions) on the table,
> the storage for security field is not necessary.

It is possible to re-call heap_form_tuple() once we know we need a
security field;  I talked Tom about that.  We can worry about it later.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

13 December 2008, 01:24:58

Robert Haas wrote:
>> I tried to summarize the proposed options, as follows:
>>
>> o : merit    x : demerit    X : unacceptable demerit
>>
>> * 1 security system column, 1 security feature (DAC or MAC)
>>  o It suitable for a single security system column implementation.
>>  x If a user want to use both of DAC and MAC concurrently, he has
>>   to choose one of them.
>>  o It allows all the security feature on the common framework,
>>   suitable for the original Row-level ACLs purpose.
>>
>> * 2 security system column, 2 security feature (DAC and MAC)
>>  o It allows both of DAC and MAC consurrently, without remarkable
>>   regressions.
>>  x It needs two new security system columns.
>>  x What is the purpose of the Row-level security in original?
>>
>> * 1 security system column, 2 security feature
>>  X It gives us catastrophic regression in user-interface, performance
>>   and code complexity. Its merit is trivial compared to its demerit.
> 
> Obviously sandwhiching two values in one column is not going to work.
> The only question here is whether it's important to simultaneously
> support both DAC and MAC.  As far as I can see, KaiGai is the only one
> arguing that we don't need to do that (except for Tom, who doesn't
> like either feature).  If anyone else agrees with his position, now
> would be a good time to speak up.

The reason why I can never agree the simultaneous support both of
DAC and MAC is that it assumes we have one security system column
and one security field on HeapTupleHeader.

If we can agree the second approach (2 security system columns and
2 security features), I don't oppose to the simultaneous support.

However, again, the third approach give us too much pains much more
than its merits, so I can never agree. Giving up simultaneous support
is much better than compounding two values into one security field.

> Peter made an excellent point a few emails upthread: there seemed to
> be consensus in the September CommitFest that we needed SQL-level
> support for row and column level security before we talked about
> implementing those features as part of SELinux.

Yes, I remember.
It is the reason why I also implemented the Row-level ACLs feature.
However, please remember the purpose of my 2 years activity is to
upstream the "fine-grained mandatory access controls with operating
syste collaboration", not a just "fine-grained (discretionary) access
controls".

> I don't see that
> we're any closer to that goal than we were then.  There has been some
> progress made on column-level permissions, but the patch is back in
> "waiting for author" limbo, and the only alternatives for SQL-level
> row-level permissions is to have them INSTEAD OF SELinux-based
> row-level permissions.  That's not the same thing at all, and I think
> it's also the underlying reason behind Bruce's complaint here:

As I noted above, I don't oppose to simultaneous support both of
DAC and MAC, if we have *2 security system columns*.
If you can agrre this point, the next patch set will support them
simultaneously.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

13 December 2008, 01:52:16

Robert Haas wrote:
>>> Peter made an excellent point a few emails upthread: there seemed to
>>> be consensus in the September CommitFest that we needed SQL-level
>>> support for row and column level security before we talked about
>>> implementing those features as part of SELinux.  I don't see that
>>> we're any closer to that goal than we were then.  There has been some
>>> progress made on column-level permissions, but the patch is back in
>>> "waiting for author" limbo, and the only alternatives for SQL-level
>>> row-level permissions is to have them INSTEAD OF SELinux-based
>>> row-level permissions.
>> I don't understand -- why wouldn't we just have two columns, one for
>> plain row-level security and another for whatever security system the
>> platforms happens to offer?  If we were to follow that route, we could
>> have row-level security first, extracting the feature from the current
>> patch; and the rest of PGACE could be a much smaller patch implementing
>> the rest of the stuff, with SELinux support for now with an eye to
>> implementing Solaris TX or whatever.
> 
> Well, I think we should do exactly what you're proposing, so don't ask me.

As I noted the previous message, the "two security system column and
two security feature" is a reasonable option which I can agree.
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

13 December 2008, 01:58:26

Bruce Momjian wrote:
> KaiGai Kohei wrote:
>>> If we use some type of integer, I suggest using this structure for
>>> pg_security:
>>>
>>>     CREATE TABLE pg_security(
>>>         relid oid, 
>>>         secid int2, 
>>>         secacl aclitem[], 
>>>         secext TEXT
>>>     );
>>>
>>> This allows the per-row value to be a simple int2.  It also improves
>>> maintenance because rows are associated only with a specific table;
>>> unused values can then be removed more easily.  And it allows both
>>> secacl and secext security to be specified.
>> How does the approach resolve the pain of user interface?
>> I don't think packing two or more values into one field is not a right way.
> 
> I see later emails that say we have to have both security methods
> available at the same time, and the table above does that.  There would
> be one security oid on every row and it would refer to this table.  

I understand the voice for simultaneous support both of DAC and MAC.
If so, I strongly desire two security column and two security field
which can store its own attribute independently.

Sorry, I cannot think compounding two values into one field is
a reasonable solution.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

13 December 2008, 02:24:14

Bruce Momjian wrote:
> KaiGai Kohei wrote:
>>>>> Also, having the per-row value always be present in the row and
>>>>> controlled by the bitmask seems ideal;  it avoids having to add a CREATE
>>>>> TABLE option.
>>>> Sorry, I don't understand why it related to a CREATE TABLE option.
>>>> System columns are always allocated for any tables?
>>> Does a table use storage for the security column if no SQL-level
>>> security value is supplied for a given row?
>> When Row-level ACL is enabled on the table and user suppies a tuple
>> without any specific ACLs, it requires security field, because the
>> length of HeapTuple is decided at heap_form_tuple() which is invoked
>> prior to fetching the user supplied ACLs.
>>
>> When Row-level ACL is disabled (by pg_class.reloptions) on the table,
>> the storage for security field is not necessary.
> 
> It is possible to re-call heap_form_tuple() once we know we need a
> security field;  I talked Tom about that.  We can worry about it later.

Is it really possible?

It seems to me the simple_heap_insert() allows to put a value on
the preallocated fixed length field (like "oid"), but does not assume
a situation that whole of the HeapTuple is replaced. Thus, when PGACE
hook is invoked from inside the simple_heap_insert() and it replaces
the given tuple, the caller does not know it and believes the older
tuple was inserted. Does it makes a problematic behavior, isn't it?

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Peter Eisentraut

Date:

13 December 2008, 09:27:43

On Friday 12 December 2008 19:09:26 Alvaro Herrera wrote:
> I don't understand -- why wouldn't we just have two columns, one for
> plain row-level security and another for whatever security system the
> platforms happens to offer?  If we were to follow that route, we could
> have row-level security first, extracting the feature from the current
> patch; and the rest of PGACE could be a much smaller patch implementing
> the rest of the stuff, with SELinux support for now with an eye to
> implementing Solaris TX or whatever.

Exactly.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

Peter Eisentraut

Date:

13 December 2008, 09:37:36

On Friday 12 December 2008 19:31:11 Robert Haas wrote:
> Not really.  I'm not an SELinux expert.  But typically the two do
> exist alongside one another.  For example, installing SELinux (MAC)
> does on your system does not make "chmod g+w file" (DAC) stop working;
> it merely performs an ADDITIONAL security check before allowing access
> to the file.  You have to satisfy BOTH SELinux AND the ordinary
> filesystem permissions system in order to perform an operation on a
> file.

The MAC permissions are usually set up globally (in some cryptic file) and 
apply mandatorily (= M).  So a rule might say, a file named topsecret.pdf can 
only be stored in a certain place, can only be read by certain people, can 
only be opened by a special viewer, cannot be copied and pasted out of, etc.  
And there is nothing you can do about it, even if you own the file (short of 
changing the global policy).

The DAC permissions are set up by the object owner at their discretion (= D).  
So if you write a draft.odt and want your group to look at it, you do a chgrp 
g+r or whatever, as you want.  It would be silly in this case to have to 
request a global MAC policy change for every such step.

> The contention of the author of this patch is that row-level access is
> somehow different - that even though we have two sets of checks for
> files, tables, and (assuming Stephen Frost's patch is accepted)
> columns, we will only have one set of checks for rows, and you can
> pick which one you want.

Yes, that is the part that is puzzling me as well.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1268)

From

KaiGai Kohei

Date:

15 December 2008, 00:00:30

Peter Eisentraut wrote:
> On Friday 12 December 2008 19:09:26 Alvaro Herrera wrote:
>> I don't understand -- why wouldn't we just have two columns, one for
>> plain row-level security and another for whatever security system the
>> platforms happens to offer?  If we were to follow that route, we could
>> have row-level security first, extracting the feature from the current
>> patch; and the rest of PGACE could be a much smaller patch implementing
>> the rest of the stuff, with SELinux support for now with an eye to
>> implementing Solaris TX or whatever.
> 
> Exactly.

It seems to me most of people (including me) can agree on the "2 security
feature and 2 security system columns" approach.
Now, I started to work the implementation based on the way here:
  http://code.google.com/p/sepgsql/source/browse/#svn/trunk/sepgsql-test

It enables to support a plain row-level DAC and a selectable MAC.
So, it does not require more than two security system columns, in future also.

Please wait for a few days to see the revised version of patches.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

KaiGai Kohei

Date:

17 December 2008, 07:22:50

I updated the patch set of SE-PostgreSQL and related ones (r1324).

[1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1324.patch
[2/5] http://sepgsql.googlecode.com/files/sepostgresql-utils-8.4devel-3-r1324.patch
[3/5] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1324.patch
[4/5] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1324.patch
[5/5] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1324.patch

Draft of the SE-PostgreSQL documentation is here: http://wiki.postgresql.org/wiki/SEPostgreSQL

List of updates:
- The patches are rebased to the latest CVS HEAD.

- Now the "sepostgresql-sepgsql-8.4devel-3-r1324.patch" contains PGACE security framework, SE-PostgreSQL and Row-level
ACLs.So, the 6th patch has gone.
 

- It enables to compile multiple security features within a single binary. The Row-level ACLs feature is always
available,and SE-PostgreSQL is available when we build it with "--enable-selinux" option.
 

- Two new system columns ("security_acl" and "security_label") are added. The first one is for the Row-level ACLs, and
theother is for the guest of PGACE security framework which is chosen by user.
 

- Some of interfaces are changed: * initdb got a new option "--pgace-feature" which enables to specify   one or no MAC
featureon initialization of $PGDATA.   e.g)     $ initdb --pgace-feature=selinux * pg_dump got two new options
(--security-acland --security-label)   to dump row-level ACLs and security contexts. * $PGDATA/postgresql.conf has a
newparameter of "pgace_feature".   It enables users to choose an enhanced security mechanism from   candidates.
Currently,SE-PostgreSQL is the only candidate.
 

- Todo item * Documentation updates.   The "sepostgresql-docs-8.4devel-3-r1324.patch" is not uptodate,   because higher
priorityshould be given to provide the patch   set for reviewers. So, I'll update the src/doc/* from now.
 

Thanks,

===[ Example ]========================================================
postgres=# CREATE TABLE t1 (a int, b text) WITH (row_level_acl=on);
CREATE TABLE
postgres=# INSERT INTO t1 VALUES (1, 'aaa'), (2, 'bbb'), (3, 'ccc');
INSERT 0 3
postgres=# SELECT security_label, security_acl, * FROM t1;             security_label              |  security_acl  | a
| b
 
------------------------------------------+----------------+---+-----unconfined_u:object_r:sepgsql_table_t:s0 |
{=rwdx/kaigai}| 1 | aaaunconfined_u:object_r:sepgsql_table_t:s0 | {=rwdx/kaigai} | 2 |
bbbunconfined_u:object_r:sepgsql_table_t:s0| {=rwdx/kaigai} | 3 | ccc
 
(3 rows)

postgres=# INSERT INTO t1 (security_acl, a, b) VALUES ('{kaigai=rw/kaigai}', 4, 'ddd');
INSERT 0 1
postgres=# INSERT INTO t1 (security_label, security_acl, a, b)       VALUES ('system_u:object_r:sepgsql_ro_table_t:s0',
'{kaigai=rx/kaigai}',5, 'eee');
 
INSERT 0 1
postgres=# SELECT security_label, security_acl, * FROM t1;             security_label              |    security_acl
|a |  b
 
------------------------------------------+--------------------+---+-----unconfined_u:object_r:sepgsql_table_t:s0 |
{=rwdx/kaigai}    | 1 | aaaunconfined_u:object_r:sepgsql_table_t:s0 | {=rwdx/kaigai}     | 2 |
bbbunconfined_u:object_r:sepgsql_table_t:s0| {=rwdx/kaigai}     | 3 | cccunconfined_u:object_r:sepgsql_table_t:s0 |
{kaigai=rw/kaigai}| 4 | dddsystem_u:object_r:sepgsql_ro_table_t:s0  | {kaigai=rx/kaigai} | 5 | eee
 
(5 rows)

postgres=# UPDATE t1 SET security_label = sepgsql_set_user(security_label, 'system_u'), security_acl =
'{kaigai=r/kaigai}';
UPDATE 5
postgres=# SELECT security_label, security_acl, * FROM t1;            security_label              |   security_acl    |
a|  b
 
-----------------------------------------+-------------------+---+-----system_u:object_r:sepgsql_table_t:s0    |
{kaigai=r/kaigai}| 1 | aaasystem_u:object_r:sepgsql_table_t:s0    | {kaigai=r/kaigai} | 2 |
bbbsystem_u:object_r:sepgsql_table_t:s0   | {kaigai=r/kaigai} | 3 | cccsystem_u:object_r:sepgsql_table_t:s0    |
{kaigai=r/kaigai}| 4 | dddsystem_u:object_r:sepgsql_ro_table_t:s0 | {kaigai=r/kaigai} | 5 | eee
 
(5 rows)

postgres=#

-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

Bruce Momjian

Date:

18 December 2008, 16:34:15

KaiGai Kohei wrote:
> - Two new system columns ("security_acl" and "security_label") are added.
>   The first one is for the Row-level ACLs, and the other is for the guest
>   of PGACE security framework which is chosen by user.

This is certainly an impressive patch.  I see you went with storing the
values inline rather than using pg_security, which is wise, I think.
"security_acl" is 'aclitem' (12 bytes) and "security_label" is 'text'.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

KaiGai Kohei

Date:

18 December 2008, 21:39:06

Bruce Momjian wrote:
> KaiGai Kohei wrote:
>> - Two new system columns ("security_acl" and "security_label") are added.
>>   The first one is for the Row-level ACLs, and the other is for the guest
>>   of PGACE security framework which is chosen by user.
> 
> This is certainly an impressive patch.  I see you went with storing the
> values inline rather than using pg_security, which is wise, I think.
> "security_acl" is 'aclitem' (12 bytes) and "security_label" is 'text'.

No, the "security_acl" also uses pg_security to translate between
security id (4bytes) and "aclitem[]" (variable length).

The Row-level ACLs facility internally translate the given acl array
into its own text representation, and stores it within pg_security.
We can use a common facility here to manage both of "acl" and "label".

Thanks,
--
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

Bruce Momjian

Date:

18 December 2008, 22:15:30

KaiGai Kohei wrote:
> Bruce Momjian wrote:
> > KaiGai Kohei wrote:
> >> - Two new system columns ("security_acl" and "security_label") are added.
> >>   The first one is for the Row-level ACLs, and the other is for the guest
> >>   of PGACE security framework which is chosen by user.
> > 
> > This is certainly an impressive patch.  I see you went with storing the
> > values inline rather than using pg_security, which is wise, I think.
> > "security_acl" is 'aclitem' (12 bytes) and "security_label" is 'text'.
> 
> No, the "security_acl" also uses pg_security to translate between
> security id (4bytes) and "aclitem[]" (variable length).
> 
> The Row-level ACLs facility internally translate the given acl array
> into its own text representation, and stores it within pg_security.
> We can use a common facility here to manage both of "acl" and "label".

OK, then I am confused.  I see this added to
src/include/catalog/pg_attribute.h:

+ DATA(insert ( 1247 security_acl   1034 0 -1  -8 1 -1 -1 f x i t f f t 0));
+ DATA(insert ( 1247 security_label   25 0 -1  -9 0 -1 -1 f x i t f f t 0));

1034 and 25 are the oids for 'acllist' and 'text' and they are being
added to system tables.  Are you saying system tables don't use
pg_security but other tables do?

I do see pg_security being defined:
+ CATALOG(pg_security,3400) BKI_SHARED_RELATION+ {+   text        seclabel;       /* text representation of security
label*/+ } FormData_pg_security;
 

and I assume both security columns reference that.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

KaiGai Kohei

Date:

18 December 2008, 22:53:56

Bruce Momjian wrote:
> KaiGai Kohei wrote:
>> Bruce Momjian wrote:
>>> KaiGai Kohei wrote:
>>>> - Two new system columns ("security_acl" and "security_label") are added.
>>>>   The first one is for the Row-level ACLs, and the other is for the guest
>>>>   of PGACE security framework which is chosen by user.
>>> This is certainly an impressive patch.  I see you went with storing the
>>> values inline rather than using pg_security, which is wise, I think.
>>> "security_acl" is 'aclitem' (12 bytes) and "security_label" is 'text'.
>> No, the "security_acl" also uses pg_security to translate between
>> security id (4bytes) and "aclitem[]" (variable length).
>>
>> The Row-level ACLs facility internally translate the given acl array
>> into its own text representation, and stores it within pg_security.
>> We can use a common facility here to manage both of "acl" and "label".
> 
> OK, then I am confused.  I see this added to
> src/include/catalog/pg_attribute.h:
> 
> + DATA(insert ( 1247 security_acl   1034 0 -1  -8 1 -1 -1 f x i t f f t 0));
> + DATA(insert ( 1247 security_label   25 0 -1  -9 0 -1 -1 f x i t f f t 0));
> 
> 1034 and 25 are the oids for 'acllist' and 'text' and they are being
> added to system tables.  Are you saying system tables don't use
> pg_security but other tables do?

It means users can refer the "security_acl" and "security_label",
as if they have proper types. However, in actually, they are stroed
as security identifiers.

When user refers "security_acl", the patched heap_getsysattr() invokes
rowaclHeapGetSecurityAclSysattr() to translate the security identifier
of Row-level ACLs into an array of ACLs. User will see the translated
representation, as if there is a variable length array, not an oid.

The rowaclHeapGetSecurityAclSysattr() fetch a tuple for the given
security identifier from pg_security. It contains a text representation
of ACLs, so the function exchange it into Acl *.

rowaclHeapGetSecurityAclSysattr() -> rowaclSidToSecurityAcl()     -> pgaceLookupSecurityLabel() ... It fetch a tuple
frompg_security     -> rawAclTextToAclArray() ... It translate from TEXT -> AclItem[]

> I do see pg_security being defined:
> 
>     + CATALOG(pg_security,3400) BKI_SHARED_RELATION
>     + {
>     +   text        seclabel;       /* text representation of security label */
>     + } FormData_pg_security;
> 
> and I assume both security columns reference that.

Yes, AclItem[] is internally translated into text representation to store
it on pg_security.

Thanks,
--
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

Bruce Momjian

Date:

18 December 2008, 23:39:04

Ah, very interesting.  Thanks.

---------------------------------------------------------------------------

KaiGai Kohei wrote:
> > 1034 and 25 are the oids for 'acllist' and 'text' and they are being
> > added to system tables.  Are you saying system tables don't use
> > pg_security but other tables do?
> 
> It means users can refer the "security_acl" and "security_label",
> as if they have proper types. However, in actually, they are stroed
> as security identifiers.
> 
> When user refers "security_acl", the patched heap_getsysattr() invokes
> rowaclHeapGetSecurityAclSysattr() to translate the security identifier
> of Row-level ACLs into an array of ACLs. User will see the translated
> representation, as if there is a variable length array, not an oid.
> 
> The rowaclHeapGetSecurityAclSysattr() fetch a tuple for the given
> security identifier from pg_security. It contains a text representation
> of ACLs, so the function exchange it into Acl *.
> 
> rowaclHeapGetSecurityAclSysattr()
>   -> rowaclSidToSecurityAcl()
>       -> pgaceLookupSecurityLabel() ... It fetch a tuple from pg_security
>       -> rawAclTextToAclArray() ... It translate from TEXT -> AclItem[]
> 
> > I do see pg_security being defined:
> > 
> >     + CATALOG(pg_security,3400) BKI_SHARED_RELATION
> >     + {
> >     +   text        seclabel;       /* text representation of security label */
> >     + } FormData_pg_security;
> > 
> > and I assume both security columns reference that.
> 
> Yes, AclItem[] is internally translated into text representation to store
> it on pg_security.
> 
> Thanks,
> --
> KaiGai Kohei <kaigai@kaigai.gr.jp>

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

Tom Lane

Date:

19 December 2008, 00:19:57

KaiGai Kohei <kaigai@kaigai.gr.jp> writes:
> Bruce Momjian wrote:
>> 1034 and 25 are the oids for 'acllist' and 'text' and they are being
>> added to system tables.  Are you saying system tables don't use
>> pg_security but other tables do?

> It means users can refer the "security_acl" and "security_label",
> as if they have proper types. However, in actually, they are stroed
> as security identifiers.

> When user refers "security_acl", the patched heap_getsysattr() invokes
> rowaclHeapGetSecurityAclSysattr() to translate the security identifier
> of Row-level ACLs into an array of ACLs. User will see the translated
> representation, as if there is a variable length array, not an oid.

This seems like a pretty bad idea that will eventually bite you in an
uncomfortable place.  Lying about what datatype a field is is just not
safe.

It would probably be better to expose the actual security identifier
(as an OID or whatever it is) and provide simple translation
capabilities a la regclass and other OID-alias types.
        regards, tom lane

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

KaiGai Kohei

Date:

19 December 2008, 04:18:08

Tom Lane wrote:
> KaiGai Kohei <kaigai@kaigai.gr.jp> writes:
>> Bruce Momjian wrote:
>>> 1034 and 25 are the oids for 'acllist' and 'text' and they are being
>>> added to system tables.  Are you saying system tables don't use
>>> pg_security but other tables do?
> 
>> It means users can refer the "security_acl" and "security_label",
>> as if they have proper types. However, in actually, they are stroed
>> as security identifiers.
> 
>> When user refers "security_acl", the patched heap_getsysattr() invokes
>> rowaclHeapGetSecurityAclSysattr() to translate the security identifier
>> of Row-level ACLs into an array of ACLs. User will see the translated
>> representation, as if there is a variable length array, not an oid.
> 
> This seems like a pretty bad idea that will eventually bite you in an
> uncomfortable place.  Lying about what datatype a field is is just not
> safe.

Is it also correct for system attributes?
I don't think the format on storage has to be same as user visible one,
because it always fetched via heap_getsysattr().
In addition, all the route to import security attributes are hooked
and translated it into oid correctly.
What situation makes it unsafe?

> It would probably be better to expose the actual security identifier
> (as an OID or whatever it is) and provide simple translation
> capabilities a la regclass and other OID-alias types.

The regclass like approach will be fine, when we export security
relatede attributes. However, how does it handle it when we give
a new security context which is not on pg_security?

The prior version of SE-PostgreSQL has a similar idea.
It is a specific type to translate security context between text
and oid. But, it was opposed at CommitFest:May because its input
handler has to insert a new tuple when the given security context
is not found on pg_security.

It had a possibility that user declares a column as the type.
It was unhapply, because input handler cannot discriminate whether
the given input is toward the system column or user columns.

So, I think the current approach is more preferable.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

Tom Lane

Date:

19 December 2008, 12:29:43

KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
> Tom Lane wrote:
>> This seems like a pretty bad idea that will eventually bite you in an
>> uncomfortable place.  Lying about what datatype a field is is just not
>> safe.

> Is it also correct for system attributes?
> I don't think the format on storage has to be same as user visible one,
> because it always fetched via heap_getsysattr().

That's *exactly* the kind of thinking that will get you in trouble.
Where is it set in stone that system attributes are always fetched
via heap_getsysattr?  In any case, this amounts to putting display
formatting of the value into heap_getsysattr, which surely seems
like the wrong place for it.

> In addition, all the route to import security attributes are hooked
> and translated it into oid correctly.

More of the same.  It's only correct if you found and kluged up every
possible reference.  If you were treating these things as a genuine data
type then you'd be working with the system structure, instead of having
to hack slash and burn to coerce it to do what you need.

> The prior version of SE-PostgreSQL has a similar idea.
> It is a specific type to translate security context between text
> and oid. But, it was opposed at CommitFest:May because its input
> handler has to insert a new tuple when the given security context
> is not found on pg_security.

Yeah, I do remember that discussion ... but an input handler with
side-effects seems better than what this solution is sounding like.
        regards, tom lane

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

KaiGai Kohei

Date:

19 December 2008, 14:23:22

Tom Lane wrote:
> KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
>> Tom Lane wrote:
>>> This seems like a pretty bad idea that will eventually bite you in an
>>> uncomfortable place.  Lying about what datatype a field is is just not
>>> safe.
> 
>> Is it also correct for system attributes?
>> I don't think the format on storage has to be same as user visible one,
>> because it always fetched via heap_getsysattr().
> 
> That's *exactly* the kind of thinking that will get you in trouble.
> Where is it set in stone that system attributes are always fetched
> via heap_getsysattr?  In any case, this amounts to putting display
> formatting of the value into heap_getsysattr, which surely seems
> like the wrong place for it.

Is there any foundamental difference compared to the case when we
refers a field with external storage, like a text field?
We need it to be detoasted before it is delivered to output handler.
It is unclear for me why it is legal but my patch is illegal.

>> The prior version of SE-PostgreSQL has a similar idea.
>> It is a specific type to translate security context between text
>> and oid. But, it was opposed at CommitFest:May because its input
>> handler has to insert a new tuple when the given security context
>> is not found on pg_security.
> 
> Yeah, I do remember that discussion ... but an input handler with
> side-effects seems better than what this solution is sounding like.

I didn't replace the previous implementation blindly, I have a few
reasons that the current one is better than previous one.

For example, if an input handler has side-effects, what is happen in
the following query?
 SELECT 'valid_but_new_security_label'::regseclabel;

It looks like a read-only query, but the input handler can insert a new
tuple into pg_security. In addition, the newly inserted tuple may not
be refered any more. It is a waste of spaces.

I want to keep the number of entries in pg_security necessity minimum.
When is it actually needed to insert a new security label into pg_security?
It is a case when a inserted/updated tuple requires a new security label
which is not on pg_security at the moment. I don't think the type-casting
is a case for "necessity minimum".

In another example, how does it prevent users to apply the type to
store his data?
 CREATE TABLE t (     a    int,     b    text,     c    regseclabel );

Anyway, it seems to me the current input handler does not assume it
has a side-effect. So, I believe your suggestion at CommitFest:May
is correct, and the security label in text should be translated into
internal oid representation just before the given tuple actually
need to assign security identifier.

At least, I think using input handler is a regression.
Is there any other idea, if the current design is bad.

Thanks,
--
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

Gregory Stark

Date:

19 December 2008, 15:06:55

KaiGai Kohei <kaigai@kaigai.gr.jp> writes:

> I didn't replace the previous implementation blindly, I have a few
> reasons that the current one is better than previous one.
>
> For example, if an input handler has side-effects, what is happen in
> the following query?
>
>   SELECT 'valid_but_new_security_label'::regseclabel;
>
> It looks like a read-only query, but the input handler can insert a new
> tuple into pg_security. In addition, the newly inserted tuple may not
> be refered any more. It is a waste of spaces.

Ooh, and how would we know when to vacuum this label?

In the case of toast each external attribute is owned by precisely one record
(though possibly multiple versions of it). So when the record is deleted or
the attribute replaced we mark the toasted attribute's xmax and it can be
vacuumed later.

In this case you have pg_security attributes shared by many records. So we
have no way to know when they're no longer referenced.

--  Gregory Stark EnterpriseDB          http://www.enterprisedb.com Ask me about EnterpriseDB's 24x7 Postgres support!

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

KaiGai Kohei

Date:

19 December 2008, 15:27:17

Gregory Stark wrote:
> KaiGai Kohei <kaigai@kaigai.gr.jp> writes:
> 
>> I didn't replace the previous implementation blindly, I have a few
>> reasons that the current one is better than previous one.
>>
>> For example, if an input handler has side-effects, what is happen in
>> the following query?
>>
>>   SELECT 'valid_but_new_security_label'::regseclabel;
>>
>> It looks like a read-only query, but the input handler can insert a new
>> tuple into pg_security. In addition, the newly inserted tuple may not
>> be refered any more. It is a waste of spaces.
> 
> Ooh, and how would we know when to vacuum this label?

It is not necessary, because the number of security labels
are assumed enough small.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

Bruce Momjian

Date:

19 December 2008, 16:01:26

KaiGai Kohei wrote:
> Gregory Stark wrote:
> > KaiGai Kohei <kaigai@kaigai.gr.jp> writes:
> > 
> >> I didn't replace the previous implementation blindly, I have a few
> >> reasons that the current one is better than previous one.
> >>
> >> For example, if an input handler has side-effects, what is happen in
> >> the following query?
> >>
> >>   SELECT 'valid_but_new_security_label'::regseclabel;
> >>
> >> It looks like a read-only query, but the input handler can insert a new
> >> tuple into pg_security. In addition, the newly inserted tuple may not
> >> be refered any more. It is a waste of spaces.
> > 
> > Ooh, and how would we know when to vacuum this label?
> 
> It is not necessary, because the number of security labels
> are assumed enough small.

Yep.   I agree doing something like regproc makes sense:
test=> \d pg_operator   Table "pg_catalog.pg_operator"    Column    |  Type   |
Modifiers--------------+---------+-----------oprname      | name    | not null oprnamespace | oid     | not null
oprowner    | oid     | not null oprkind      | "char"  | not null oprcanmerge  | boolean | not null oprcanhash   |
boolean| not null oprleft      | oid     | not null oprright     | oid     | not null oprresult    | oid     | not null
oprcom      | oid     | not null oprnegate    | oid     | not null
 
-->     oprcode      | regproc | not null
-->     oprrest      | regproc | not null
-->     oprjoin      | regproc | not null

but the auto-creation of pg_security rows does make this problematic. 
regproc works because it only supports pre-created pg_proc rows.

Can we support two data types, one for read-only and another for
possible creation?  That is not going to work for a query like
SELECT * FROM x WHERE col != 'valid_but_new_security_label'::regseclabel;

because the cast would fail if the pg_security row doesn't exist, but
the query itself perhaps might succeed and return rows.

Can we require joins to use pg_security directly?  I wish I could think
of some magic that indicated the pg_security row should be created.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

KaiGai Kohei

Date:

19 December 2008, 22:53:35

> but the auto-creation of pg_security rows does make this problematic. 
> regproc works because it only supports pre-created pg_proc rows.

Yes, the security label has a different characteristic at the point.

> Can we support two data types, one for read-only and another for
> possible creation?  That is not going to work for a query like
> 
>     SELECT * FROM x WHERE col != 'valid_but_new_security_label'::regseclabel;
> 
> because the cast would fail if the pg_security row doesn't exist, but
> the query itself perhaps might succeed and return rows.

It has a same matter. If user cast a text into the type of possible
creation, it can cause an unnecessary insertion.

> Can we require joins to use pg_security directly?  I wish I could think
> of some magic that indicated the pg_security row should be created.

We should not allow users to join them directly, and should not expose
what identifier is used internally, because some of ISO/IEC15408 has
requirements to export/import user data with meaningful security labels.
The numeric security identifier is just an internal usage.

Here is one another idea.
How do you think asymmetric input/output handlers?

At first, I have to introduce the current implementation.
It can receives user input values as TEXT or ACLITEM[], ant these are
fetched at fetchWritableSystemAttribute() on execMain.c.
Please note that they have no side-effect on the prior stage.
Then, it invokes rowaclSecurityAclToSid() or pgaceSecurityLabelToSid()
to translate the values into security identifier. These functions can
insert a new tuple into pg_security, if not found.
Then, the translated security identifiers are set on the HeapTupleHeader
using HeapTupleSetRowAcl() or HeapTupleSetSecLabel().

If the input handler is asymmetric, it can handle the given user input
as a TEXT or ACLITEM[], and can deliver them fetchWritableSystemAttribute()
as is. But asymmetric output handler can translate given security identifier
into proper user visible data.

However, I don't think it is a fair enough solution. :(
The current implementation seems to me better.

Thanks,
--
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

KaiGai Kohei

Date:

19 December 2008, 23:22:26

> Here is one another idea.
> How do you think asymmetric input/output handlers?
> 
> At first, I have to introduce the current implementation.
> It can receives user input values as TEXT or ACLITEM[], ant these are
> fetched at fetchWritableSystemAttribute() on execMain.c.
> Please note that they have no side-effect on the prior stage.
> Then, it invokes rowaclSecurityAclToSid() or pgaceSecurityLabelToSid()
> to translate the values into security identifier. These functions can
> insert a new tuple into pg_security, if not found.
> Then, the translated security identifiers are set on the HeapTupleHeader
> using HeapTupleSetRowAcl() or HeapTupleSetSecLabel().
> 
> If the input handler is asymmetric, it can handle the given user input
> as a TEXT or ACLITEM[], and can deliver them fetchWritableSystemAttribute()
> as is. But asymmetric output handler can translate given security 
> identifier
> into proper user visible data.

I noticed the asymmetric input/output handler will break something
when someone declares user-columns as the type with asymmetric handlers.
Please forget it.

> However, I don't think it is a fair enough solution. :(
> The current implementation seems to me better.

Maybe, we need to consider what data type should be fetched via
system column. Previously, it returns required identifier of tuple,
like oid and tableoid, as is, because raw-data format is matched
with requirement for user interfaces, fortunatelly.
However, is it possible to consider the system column returns
a proper metadata virtually generated based on the given tuple's
attributes?

I don't intend to convert security id to security label here for
cosmetic purpose. It has to be done here, because "security_label"
is a TEXT data generated on runtime.

It is not an issue on implementation, an issue in frame of thinking.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

Alvaro Herrera

Date:

19 December 2008, 23:22:30

KaiGai Kohei wrote:

>> Can we support two data types, one for read-only and another for
>> possible creation?  That is not going to work for a query like
>>
>>     SELECT * FROM x WHERE col != 'valid_but_new_security_label'::regseclabel;
>>
>> because the cast would fail if the pg_security row doesn't exist, but
>> the query itself perhaps might succeed and return rows.
>
> It has a same matter. If user cast a text into the type of possible
> creation, it can cause an unnecessary insertion.

How's this for an idea.  Keep a cache in memory, which has a reference
counter.  This counter would only be incremented for written tuples.  At
transaction end (or when the cache is full and you need more room),
entries are evicted; if the reference count is greater than zero on
eviction, a pg_security entry is created.  Otherwise it is just
discarded.  pg_security entry creation (i.e. eviction from cache, or at
transaction end) is WAL-logged, which is enough in case of a crash,
because if a tuple is written with an unlogged pg_security OID, the
transaction did not abort and thus nobody can possibly be interested in
translating it.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

Bruce Momjian

Date:

19 December 2008, 23:45:56

KaiGai Kohei wrote:
> > but the auto-creation of pg_security rows does make this problematic. 
> > regproc works because it only supports pre-created pg_proc rows.
> 
> Yes, the security label has a different characteristic at the point.
> 
> > Can we support two data types, one for read-only and another for
> > possible creation?  That is not going to work for a query like
> > 
> >     SELECT * FROM x WHERE col != 'valid_but_new_security_label'::regseclabel;
> > 
> > because the cast would fail if the pg_security row doesn't exist, but
> > the query itself perhaps might succeed and return rows.
> 
> It has a same matter. If user cast a text into the type of possible
> creation, it can cause an unnecessary insertion.

Right.  I was trying to point out that you can't assume that every
string you cast to ::regseclabel is going to already exist in
pg_security;  in this case, we are doing a not equal so we don't assume
it is already in the table.

> > Can we require joins to use pg_security directly?  I wish I could think
> > of some magic that indicated the pg_security row should be created.
> 
> We should not allow users to join them directly, and should not expose
> what identifier is used internally, because some of ISO/IEC15408 has
> requirements to export/import user data with meaningful security labels.
> The numeric security identifier is just an internal usage.

Yes, I was going to ask if there was any API we need to follow, and you
are confirming that an insert of a text string must be possible for that
column.  We can do an auto-cast of the string to the internal data type
so the call to the function that inserts into pg_security would be
invisible, but your point that it might create lots of pg_security rows
that aren't referenced is a good one.

> Here is one another idea.
> How do you think asymmetric input/output handlers?
> 
> At first, I have to introduce the current implementation.
> It can receives user input values as TEXT or ACLITEM[], ant these are
> fetched at fetchWritableSystemAttribute() on execMain.c.
> Please note that they have no side-effect on the prior stage.
> Then, it invokes rowaclSecurityAclToSid() or pgaceSecurityLabelToSid()
> to translate the values into security identifier. These functions can
> insert a new tuple into pg_security, if not found.
> Then, the translated security identifiers are set on the HeapTupleHeader
> using HeapTupleSetRowAcl() or HeapTupleSetSecLabel().
> 
> If the input handler is asymmetric, it can handle the given user input
> as a TEXT or ACLITEM[], and can deliver them fetchWritableSystemAttribute()
> as is. But asymmetric output handler can translate given security identifier
> into proper user visible data.
> 
> However, I don't think it is a fair enough solution. :(
> The current implementation seems to me better.

Interesting.  It is really up to the people who work in the heap fetch
area of the code to tell us how disruptive the security framework can be
in that area of the code.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

KaiGai Kohei

Date:

20 December 2008, 01:36:00

Bruce Momjian wrote:
> KaiGai Kohei wrote:
>>> but the auto-creation of pg_security rows does make this problematic. 
>>> regproc works because it only supports pre-created pg_proc rows.
>> Yes, the security label has a different characteristic at the point.
>>
>>> Can we support two data types, one for read-only and another for
>>> possible creation?  That is not going to work for a query like
>>>
>>>     SELECT * FROM x WHERE col != 'valid_but_new_security_label'::regseclabel;
>>>
>>> because the cast would fail if the pg_security row doesn't exist, but
>>> the query itself perhaps might succeed and return rows.
>> It has a same matter. If user cast a text into the type of possible
>> creation, it can cause an unnecessary insertion.
> 
> Right.  I was trying to point out that you can't assume that every
> string you cast to ::regseclabel is going to already exist in
> pg_security;  in this case, we are doing a not equal so we don't assume
> it is already in the table.

Yes, I didn't describe the case explicitly, because the case when
given string is already on pg_security is not an issue.

>>> Can we require joins to use pg_security directly?  I wish I could think
>>> of some magic that indicated the pg_security row should be created.
>> We should not allow users to join them directly, and should not expose
>> what identifier is used internally, because some of ISO/IEC15408 has
>> requirements to export/import user data with meaningful security labels.
>> The numeric security identifier is just an internal usage.
> 
> Yes, I was going to ask if there was any API we need to follow, and you
> are confirming that an insert of a text string must be possible for that
> column.

Common Criteria v3.1 : Part 2 Security functional components
http://www.commoncriteriaportal.org/files/ccfiles/CCPART2V3.1R2.pdf

It describes about export/import of user data with security attributes
on FDP_ETC.2 and FDP_ITC.2 section. These items are considered as basic
facilities to be provided by MAC system, and requires it has a capability
to export/import user data with security attributes.
Since CC is designed for general purpose, it does not specify format
of security attribute, but it is quite natural it is a security label
in text representation on MAC system.
At least, previous certified MAC system (like SELinux) interprets it so.
> We can do an auto-cast of the string to the internal data type> so the call to the function that inserts into
pg_securitywould be> invisible, but your point that it might create lots of pg_security rows> that aren't referenced is
agood one.
 

Yes, it was the older design in SE-PostgreSQL, and it is worse than
the current implementation.
The current one inserts tuples necessity minimum.

>> Here is one another idea.
>> How do you think asymmetric input/output handlers?
>>
>> At first, I have to introduce the current implementation.
>> It can receives user input values as TEXT or ACLITEM[], ant these are
>> fetched at fetchWritableSystemAttribute() on execMain.c.
>> Please note that they have no side-effect on the prior stage.
>> Then, it invokes rowaclSecurityAclToSid() or pgaceSecurityLabelToSid()
>> to translate the values into security identifier. These functions can
>> insert a new tuple into pg_security, if not found.
>> Then, the translated security identifiers are set on the HeapTupleHeader
>> using HeapTupleSetRowAcl() or HeapTupleSetSecLabel().
>>
>> If the input handler is asymmetric, it can handle the given user input
>> as a TEXT or ACLITEM[], and can deliver them fetchWritableSystemAttribute()
>> as is. But asymmetric output handler can translate given security identifier
>> into proper user visible data.
>>
>> However, I don't think it is a fair enough solution. :(
>> The current implementation seems to me better.
> 
> Interesting.  It is really up to the people who work in the heap fetch
> area of the code to tell us how disruptive the security framework can be
> in that area of the code.

As I noted on the following message, I reconsidered the asymmetric
input/output handler is not a productive idea.

It is still unclear for me why heap_getsysattr() has to return
a value with same format in storage. If we can change the point
of view, it is possible to return a value generated in runtime
based on any other attributes/userdata of tuples.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

KaiGai Kohei

Date:

21 December 2008, 11:02:41

> - Todo item
>   * Documentation updates.
>     The "sepostgresql-docs-8.4devel-3-r1324.patch" is not uptodate,
>     because higher priority should be given to provide the patch
>     set for reviewers. So, I'll update the src/doc/* from now.

The documentation patch is updated, as follows:
- http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1334.patch

I revised whole of the descriptions, and it contains the recent design
changes, like two new system columns, hardwired Row-level Database ACLs
and so on.

Rest of components are unchanged, see the rev.1224 for them.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1324)

From

KaiGai Kohei

Date:

21 December 2008, 11:03:58

Tom Lane wrote:
> KaiGai Kohei <kaigai@ak.jp.nec.com> writes:
>> Tom Lane wrote:
>>> This seems like a pretty bad idea that will eventually bite you in an
>>> uncomfortable place.  Lying about what datatype a field is is just not
>>> safe.
> 
>> Is it also correct for system attributes?
>> I don't think the format on storage has to be same as user visible one,
>> because it always fetched via heap_getsysattr().
> 
> That's *exactly* the kind of thinking that will get you in trouble.
> Where is it set in stone that system attributes are always fetched
> via heap_getsysattr?  In any case, this amounts to putting display
> formatting of the value into heap_getsysattr, which surely seems
> like the wrong place for it.

I read your comments again, but I still cannot understand the reason
why the current approah is worse than the way of input/output handler
replaced before.

If your opinion is unchanged now, could you tell me the point?

At least, the security system columns need to accept new security
labels in text format which are not on pg_security.
If type input handler translate the given string into internal
security identifier, it allows to make unnecessary entries.

For example: SELECT 'valid_but_new_security_label'::regseclabel;

In the current implementation, this translation is done just before
an actual insertion or updates. So, it does not insert unrefered
entries and does not pollute pg_security under read-only transaction.

You suggested that heap_getsysattr() is wrong place for display
formatting of values. Its reason is still unclear for me.
The relationship between security identifier and security label is
similar to a data type with external storage, like TEXT.
It is not un-toasted on output handler. It is already untoasted
and its Datum hold a pointer to varlena object generated on runtime,
then textout() translate it into CString.
I think the "security_label" and "security_acl" is similar ones.
Can you consider the security identifiers are like a pointer which
indicates internal object, so these should be resolved before the
values are delivered to output handlers?

In addition, there are a few minor issues if we switch to the
input/output handler approach:

- What should it send/receive on binary type handler?The security identifiers are purely internal representation. If we
canexportthem to outside of the system, it is nonsense, because we haveno assurance same security identifier has same
labelin other hosts.In the older SE-PostgreSQL, it did not implement binary handler toprevent binary output.

- Output handler cannot know what relation is the given security identifier fetched from.When the given security
identifieris not valid, we need to preparean alternative TEXT or ACLs. The Row-level ACLs generates a defaultACLs like
'{=rwdx/kaigai}',because it allows anything on tupleswithout any ACLs. But it requires an information of who is ownerof
therelation. It is not delivered to type i/o handlers, so wehave to consider another representation.

Indeed, it is possible to implement this feature with type i/o
handler approach. In fact, I had implemented the older version
in this one.
However, there are a few demerits, but I cannot find any merits
yet. So, I still hesitate to replace the current implementation
and believe the current implementation is better.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Updates of SE-PostgreSQL 8.4devel patches (r1348)

From

KaiGai Kohei

Date:

25 December 2008, 10:35:18

I updated the patch set of SE-PostgreSQL and related stuff (r1348)

[1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1348.patch
[2/5] http://sepgsql.googlecode.com/files/sepostgresql-utils-8.4devel-3-r1348.patch
[3/5] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1348.patch
[4/5] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1348.patch
[5/5] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1348.patch
Draft of the SE-PostgreSQL documentation is here: http://wiki.postgresql.org/wiki/SEPostgreSQL (It also should be
updatedfor the recent changes...)

List of updates:
- The patches are rebased to the latest CVS HEAD. Currently, previous ones (r1324) are not suitable for this.
- It put a copied relkind value on pg_attribute.attkind. This change enables to reduce per tuple lookups for RELOID,
andimprove robustness of security model.

- bugfix: heap_getsysattr() could return NULL, when enhanced security feature is disabled. It is fixed to return an
alternativelabel/default acl.

- errcode_for_file_access() is applied on filesystem related errors, instead of ERRCODE_SELINUX_ERROR.
- Reloptions related code for Row-level ACLs feature is flattened. Now it invokes rowaclXXX() without PGACE hooks,
becausethere is an active effort to support variable kind of reloptions now.

- The "default_row_acl" got stored as text represenation due to incorrect table dump. (We should not put it as security
id.)
- bugfix: Makefile in src/test/sepgsql

Request for comments:

The current heap_reloptions() requires reloption-parser not to
raise an error when validate = false.
However, it makes a matter when we store "default_row_acl" as
a entry of reloptions. The input handler of AclItem[] can raise
an error if given input string has invalid format or users.

What solutions can be considered?
- Implement its own AclItem[] parser which does not raise an error on validate = false.
- Set dependencies on users which appears in default Row-ACLs.
- Remove default Row-level ACLs feature.
- Any other idea?

And, I have a question.
Is the new reloption framework designed to store strings?
The latest one support Bool, Int and Real, doen't it?

Thanks,
--
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1348)

From

KaiGai Kohei

Date:

26 December 2008, 08:56:57

> I updated the patch set of SE-PostgreSQL and related stuff (r1348)
> 
> [1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1348.patch
> [2/5] http://sepgsql.googlecode.com/files/sepostgresql-utils-8.4devel-3-r1348.patch
> [3/5] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1348.patch
> [4/5] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1348.patch
> [5/5] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1348.patch

I found a few bugs in this revision.
- It does not apply permission checks on trans/final functions in aggregate function.
- When a set-returning function is inlined by optimizer, trusted procedure does not work correctly.
- When we select rowtype on inherited table, it checks superfluous column's permissions.

Please wait for a while to fix them...
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Updates of SE-PostgreSQL 8.4devel patches (r1368)

From

KaiGai Kohei

Date:

29 December 2008, 11:54:06

I updated patch set of SE-PostgreSQL and related stuff (r1368)

[1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1368.patch
[2/5] http://sepgsql.googlecode.com/files/sepostgresql-utils-8.4devel-3-r1368.patch
[3/5] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1368.patch
[4/5] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1368.patch
[5/5] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1368.patch

Any comments welcome!

List of updates:
- The patches are rebased to the latest CVS HEAD.
- bugfix: proper permission check on aggregate functions. It also reworks permission checks on function invocations. In
theprevious version, it picked up appeared functions while it walks on given query trees. But it possibly give us
futuremaintenance pains, so I put a few hooks on executor phases instead.
 
- bugfix: proper permission check when a set-returning function is inlined
- bugfix: proper column permission checks on SELECT rowtype from inherited table.

-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1368)

From

KaiGai Kohei

Date:

29 December 2008, 14:57:26

Sorry, I found a unnecessary hook is not removed at ExecCallTriggerFunc().

Please see the revision r1370, instead.

[1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1370.patch
[2/5] http://sepgsql.googlecode.com/files/sepostgresql-utils-8.4devel-3-r1370.patch
[3/5] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1370.patch
[4/5] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1370.patch
[5/5] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1370.patch

KaiGai Kohei wrote:
> I updated patch set of SE-PostgreSQL and related stuff (r1368)
> 
> [1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1368.patch
> [2/5] http://sepgsql.googlecode.com/files/sepostgresql-utils-8.4devel-3-r1368.patch
> [3/5] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1368.patch
> [4/5] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1368.patch
> [5/5] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1368.patch
> 
> Any comments welcome!
> 
> List of updates:
> - The patches are rebased to the latest CVS HEAD.
> - bugfix: proper permission check on aggregate functions.
>   It also reworks permission checks on function invocations.
>   In the previous version, it picked up appeared functions while it walks on
>   given query trees. But it possibly give us future maintenance pains, so
>   I put a few hooks on executor phases instead.
> - bugfix: proper permission check when a set-returning function is inlined
> - bugfix: proper column permission checks on SELECT rowtype from inherited table.
> 


-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Updates of SE-PostgreSQL 8.4devel patches (r1386)

From

KaiGai Kohei

Date:

05 January 2009, 13:35:46

I updates patch set of SE-PostgreSQL and related stuff (r1386).

[1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1386.patch
[2/5] http://sepgsql.googlecode.com/files/sepostgresql-utils-8.4devel-3-r1386.patch
[3/5] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1386.patch
[4/5] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1386.patch
[5/5] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1386.patch

List of updates:
- The patches are rebased to the latest CVS HEAD. (Previous one (r1368) conflicts to newly added features.)
- The "default_row_acl" reloption is removed temporary, because Alvaro Herrera is now under reworking to generic
reloptionsframework.
 
- Copyrights are updated from "2008" to "2009".

Most of them are unchanged.
Please don't mind it, if you are under reviewing at r1368.

BTW, what is the currect status of my patches?
I guess you have many comments in the five patches, because they are a bit
large to be commited obviously.
Could you tell me, even if these are not comprehensive ones.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1386)

From

"Robert Haas"

Date:

05 January 2009, 15:07:09

On Mon, Jan 5, 2009 at 4:44 AM, KaiGai Kohei <kaigai@ak.jp.nec.com> wrote:
> BTW, what is the currect status of my patches?
> I guess you have many comments in the five patches, because they are a bit
> large to be commited obviously.
> Could you tell me, even if these are not comprehensive ones.

I haven't looked at the patches, but one thing I'm concerned about is
the fact that it seems we still don't have a working implementation of
non-SEPostgresql column-level privileges.  Apparently, the latest
patch set from Stephen Frost doesn't handle those permissions when
joins are involved, which presumably means that it is not committable.

...Robert

Re: Updates of SE-PostgreSQL 8.4devel patches (r1386)

From

Bruce Momjian

Date:

05 January 2009, 15:52:43

Robert Haas wrote:
> On Mon, Jan 5, 2009 at 4:44 AM, KaiGai Kohei <kaigai@ak.jp.nec.com> wrote:
> > BTW, what is the currect status of my patches?
> > I guess you have many comments in the five patches, because they are a bit
> > large to be commited obviously.
> > Could you tell me, even if these are not comprehensive ones.
> 
> I haven't looked at the patches, but one thing I'm concerned about is
> the fact that it seems we still don't have a working implementation of
> non-SEPostgresql column-level privileges.  Apparently, the latest
> patch set from Stephen Frost doesn't handle those permissions when
> joins are involved, which presumably means that it is not committable.

Yes, this certainly is going to be an issue.  The SE-PostgreSQL patch
gives us SQL and SE-level row permissions, but only SE-level column
permissions (because SQL-level is coming from another patch).

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1386)

From

Tom Lane

Date:

05 January 2009, 18:28:19

"Robert Haas" <robertmhaas@gmail.com> writes:
> I haven't looked at the patches, but one thing I'm concerned about is
> the fact that it seems we still don't have a working implementation of
> non-SEPostgresql column-level privileges.  Apparently, the latest
> patch set from Stephen Frost doesn't handle those permissions when
> joins are involved, which presumably means that it is not committable.

No, but it might be fixable.  I plan to look at that patch next after
I have my immediate to-do items cleared off ...
        regards, tom lane

Updates of SE-PostgreSQL 8.4devel patches (r1389)

From

KaiGai Kohei

Date:

06 January 2009, 07:09:28

I updated patch set of SE-PostgreSQL and related stuff (r1389).

[1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1389.patch
[2/5] http://sepgsql.googlecode.com/files/sepostgresql-utils-8.4devel-3-r1389.patch
[3/5] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1389.patch
[4/5] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1389.patch
[5/5] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1389.patch

List of updates:
- The patches are rebased to the latest CVS HEAD, which includes generic reloptions framework by Alvaro Herrera.
- Row-level ACL's reloptions ("row_level_acl" and "default_row_acl") are reworked based on the new framework.

Alvaro, could you check the patched code on reloptions.h, reloptions.c
and rel.h? It is a working example of string reloptions, and I could
found a few strange codes.

1. HANDLE_STRING_RELOPTION() always put an empty string when optstring->default_isnull is true, even if user gives a
validstring reloption.

2. HANDLE_STRING_RELOPTION() cannot handle an offset style. The patched one enables to put reloption string on the
tailof StdRdOptions structure, and adjust offset value.

3. Why the "StdRdOptions lopts;" is necessary? A string reloption need to put it on the tail of StdRdOptions and
memberof the structure indicates its offset, so it should be allocated with variable length at the begining. The
patchedone invokes palloc0() with sizeof(StdRdOptions) and length of string at first.

And, I have a request.
4. Is it possible to support a call-back to validate a given string reloption? I want to check whether the given
default Row-level ACLs has a valid format, or not.

Thanks,
--
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1386)

From

KaiGai Kohei

Date:

06 January 2009, 07:52:38

Tom Lane wrote:
> "Robert Haas" <robertmhaas@gmail.com> writes:
>> I haven't looked at the patches, but one thing I'm concerned about is
>> the fact that it seems we still don't have a working implementation of
>> non-SEPostgresql column-level privileges.  Apparently, the latest
>> patch set from Stephen Frost doesn't handle those permissions when
>> joins are involved, which presumably means that it is not committable.
> 
> No, but it might be fixable.  I plan to look at that patch next after
> I have my immediate to-do items cleared off ...

Stephen, your patch appends attribute numbers on rte->cols_sel list,
even if the rte->relkind is RTE_JOIN.
Since ExecCheckRTEPerms() skips RangeTblEntry without RTE_RELATION,
it is necessary to care special case when the given rte->relkind has
RTE_JOIN, isn't it?

I think these attribute numbers should be chained on the source
RangeTblEntry of the joins to solve the issue.

Anyway, I hope your patch getting merged as soon as possible. :-)
If I have anything to help you, please feel free to ask for.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1389)

From

Alvaro Herrera

Date:

06 January 2009, 11:13:23

KaiGai Kohei wrote:

> Alvaro, could you check the patched code on reloptions.h, reloptions.c
> and rel.h? It is a working example of string reloptions, and I could
> found a few strange codes.

I'm intending to revisit the string code ... I was thinking yesterday
night that I shouldn't have committed it at all, and left it for a
subsequent patch that I had more chance to test properly :-(

> 1. HANDLE_STRING_RELOPTION() always put an empty string when
>    optstring->default_isnull is true, even if user gives a
>    valid string reloption.

This is a plain bug, sorry.

> 2. HANDLE_STRING_RELOPTION() cannot handle an offset style.
>    The patched one enables to put reloption string on the
>    tail of StdRdOptions structure, and adjust offset value.

I'll look at it, thanks.

> 3. Why the "StdRdOptions lopts;" is necessary?

It is like this because the autovacuum patch adds a few more options and
I want to have the chance to not allocate the part belonging to
autovacuum when none of the options are present.

> And, I have a request.
> 4. Is it possible to support a call-back to validate a given
>    string reloption? I want to check whether the given default
>    Row-level ACLs has a valid format, or not.

Hmm, why a callback and not just call the validation function in
heap_reloptions?

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Re: Updates of SE-PostgreSQL 8.4devel patches (r1389)

From

KaiGai Kohei

Date:

06 January 2009, 12:28:41

Alvaro Herrera wrote:
> KaiGai Kohei wrote:
> 
>> Alvaro, could you check the patched code on reloptions.h, reloptions.c
>> and rel.h? It is a working example of string reloptions, and I could
>> found a few strange codes.
> 
> I'm intending to revisit the string code ... I was thinking yesterday
> night that I shouldn't have committed it at all, and left it for a
> subsequent patch that I had more chance to test properly :-(
> 
>> 1. HANDLE_STRING_RELOPTION() always put an empty string when
>>    optstring->default_isnull is true, even if user gives a
>>    valid string reloption.
> 
> This is a plain bug, sorry.
> 
>> 2. HANDLE_STRING_RELOPTION() cannot handle an offset style.
>>    The patched one enables to put reloption string on the
>>    tail of StdRdOptions structure, and adjust offset value.
> 
> I'll look at it, thanks.
> 
>> 3. Why the "StdRdOptions lopts;" is necessary?
> 
> It is like this because the autovacuum patch adds a few more options and
> I want to have the chance to not allocate the part belonging to
> autovacuum when none of the options are present.

We can return NULL immediately without any allocation, when numoptions=0.
Does it give us any pains?
http://code.google.com/p/sepgsql/source/browse/trunk/sepgsql/src/backend/access/common/reloptions.c#765

>> And, I have a request.
>> 4. Is it possible to support a call-back to validate a given
>>    string reloption? I want to check whether the given default
>>    Row-level ACLs has a valid format, or not.
> 
> Hmm, why a callback and not just call the validation function in
> heap_reloptions?

I thought you intend to apply validation checks in parse_one_reloption()
invoked from parseRelOptions(), but now we have no checks in string
reloptions.
In my personal preference, it is more simple design parse_one_reloption()
invoke a function pointer for validation checks.

Please decide a guideline to be followed when we add a new string reloption.
If it requires to invoke the function from heap_reloptions(), I'll follow it.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1389)

From

Alvaro Herrera

Date:

07 January 2009, 12:46:32

KaiGai Kohei wrote:
> Alvaro Herrera wrote:

>>> 3. Why the "StdRdOptions lopts;" is necessary?
>>
>> It is like this because the autovacuum patch adds a few more options and
>> I want to have the chance to not allocate the part belonging to
>> autovacuum when none of the options are present.
>
> We can return NULL immediately without any allocation, when numoptions=0.
> Does it give us any pains?
> http://code.google.com/p/sepgsql/source/browse/trunk/sepgsql/src/backend/access/common/reloptions.c#765

That's not what I mean -- the problem is that some tables can have only
fillfactor allocated, and I didn't want to allocate the whole struct
just for fillfactor.  The technique I was using (which was to check the
length of the struct) is not going to work now with string reloptions
though, so it's not much of an issue.

> I thought you intend to apply validation checks in parse_one_reloption()
> invoked from parseRelOptions(), but now we have no checks in string
> reloptions.
> In my personal preference, it is more simple design parse_one_reloption()
> invoke a function pointer for validation checks.

Agreed, it seems better.  The attached patch adds that, a macro you
originally requested and another one, and it also fixes an off-by-one
bug I discovered while testing all of this.  I also attach the testing
patch I play with to check that this all works nicely.

Oh, the patch also removes a bunch of "continue" statements that, as far
as I can tell, no longer work after the macros were wrapped in
do { ... } while (0) :-(  I don't see any nice way to put the facility
back.

Thanks for all the input.

--
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Tom Lane wrote:

> I guess I'm still wondering which part of this actually needs to be
> hand-coded so that it can be flexible.  I'm envisioning the whole
> loop replaced by something like
>
>     FillRelOptions((void *) rdopts, options, &constanttable);
>
> where the constant table contains entries like
>
>     { "fillfactor", RELOPT_TYPE_INT, offsetof(StdRdOptions, fillfactor) }

I attach a patch that does things this way (it includes the btree test
code because I'm too lazy right now to strip it out).

I'm not really sure about removing the other macros completely, because
they would be useful whenever one wanted to create something
nonstandard.

> BTW, are we just assuming that there's never a possibility of no match?
> It seems like there ought to be an elog complaint if you get to the
> bottom of the loop; which again is something I don't see the point of
> writing out each time.

We need to be quiet about it when not validating, I think.

--
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Attachment

reloptions-9.patch

Re: Updates of SE-PostgreSQL 8.4devel patches (r1389)

From

Alvaro Herrera

Date:

09 January 2009, 23:42:45

Alvaro Herrera wrote:
> Tom Lane wrote:
> 
> > I guess I'm still wondering which part of this actually needs to be
> > hand-coded so that it can be flexible.  I'm envisioning the whole
> > loop replaced by something like
> > 
> >     FillRelOptions((void *) rdopts, options, &constanttable);
> > 
> > where the constant table contains entries like
> > 
> >     { "fillfactor", RELOPT_TYPE_INT, offsetof(StdRdOptions, fillfactor) }
> 
> I attach a patch that does things this way (it includes the btree test
> code because I'm too lazy right now to strip it out).

The irony of doing things this way is that we've come full-circle from
the original coding of these routines (the main difference being that
the default values and checks no longer need to be written as code, but
rather as table entries).

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Updates of SE-PostgreSQL 8.4devel patches (r1403)

From

KaiGai Kohei

Date:

13 January 2009, 05:22:18

I updated patch set of SE-PostgreSQL and related stuff (r1403).

[1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1403.patch
[2/5] http://sepgsql.googlecode.com/files/sepostgresql-utils-8.4devel-3-r1403.patch
[3/5] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1403.patch
[4/5] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1403.patch
[5/5] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1403.patch

List of updates:
- These are rebased to the latest CVS HEAD.
- I removed the argument of "bool validate" from string reloption callbacks due to its improvement on the latest CVS
HEAD.

There are no changes in rest of the parts.
If you saw the previous version partway, here is no matter to switch them.

The column-level privileges feature is coming to get merged increasingly.
It is not too early to *begin* reviewing the SE-PostgreSQL features now.
Please make it progress!

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1403)

From

Alvaro Herrera

Date:

13 January 2009, 12:08:51

KaiGai Kohei wrote:
> I updated patch set of SE-PostgreSQL and related stuff (r1403).
> 
> [1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1403.patch

Random observations:

heapam.c: you've got a bunch of elog(ERROR) calls in there that should
be ereport(ERROR), and should probably have a errcode() on them too.
Also the message should be worded like this:
"could not insert tuple on \"%s\" due to security reasons"
or something like that.  I mean: do not use C function names in error
messages; quote the table name.

tuptoaster.c: same problem

heap.c: typo on line 1062, says "el", should say "rel"

pgace.h: you have a bunch of "static inline" functions in here.  As far
as I know this doesn't work in compilers other than GCC :-(  See
pg_list.h (list_head) for an example.  I think we can tolerate this for
the three functions in pg_list.h because they are so few and so tiny,
but I'm not sure about PGACE because they are a large lot.  On the other
hand, turning them to real functions would be a performance hit.

The pgace worker process ... do your postmaster.c changes work when
pgace is disabled?  I think it tries to start the worker on every
iteration.  Maybe it needs more smarts in postmaster.c so that when
!sepgsqlIsEnabled() it just doesn't try to start it up.  Also, I think
there should be a separate function to tell whether a particular
PGACE_FEATURE actually needs a worker process; right now the only
feature (SELinux) does need it, but is this the case for them all?

I didn't delve into many details in the avc, the worker, or anything
much here actually -- I just skimmed randomly.  This is a really huge
patch; sorry I do not have time right now to review it.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1403)

From

KaiGai Kohei

Date:

14 January 2009, 01:36:29

Alvaro Herrera wrote:
> KaiGai Kohei wrote:
>> I updated patch set of SE-PostgreSQL and related stuff (r1403).
>>
>> [1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1403.patch
> 
> Random observations:

Thanks for your comment!

> heapam.c: you've got a bunch of elog(ERROR) calls in there that should
> be ereport(ERROR), and should probably have a errcode() on them too.
> Also the message should be worded like this:
> "could not insert tuple on \"%s\" due to security reasons"
> or something like that.  I mean: do not use C function names in error
> messages; quote the table name.
> 
> tuptoaster.c: same problem
>> heap.c: typo on line 1062, says "el", should say "rel"

Fixed,  http://code.google.com/p/sepgsql/source/detail?r=1404

IIRC, Tom pointed out we should apply ereport() for user visible error
messages, not elog(), on CommitFest:May, but I missed to fix them.

> pgace.h: you have a bunch of "static inline" functions in here.  As far
> as I know this doesn't work in compilers other than GCC :-(  See
> pg_list.h (list_head) for an example.  I think we can tolerate this for
> the three functions in pg_list.h because they are so few and so tiny,
> but I'm not sure about PGACE because they are a large lot.  On the other
> hand, turning them to real functions would be a performance hit.

It was the original implementation of PGACE hooks about 700 revisions ago. :-)
http://code.google.com/p/sepgsql/source/browse/trunk/sepgsql/src/backend/security/pgaceHooks.c?r=739

On CommitFest:May, I got a comment to replace them as inline functions,
so they are moved to src/inclide/security/pgace.h.
At that time, it turned on/off a security feature using compile-time
option, so I guess reviewer concerned about cost to invoke empty function
when no enhanced security feature is available.

However, I also think what you pointed out is fair enough.
Now pgace hooks allows 'pgace_feature' option to choose an enhanced security
feature on startup time, not a compile time, so the assumption was changed.
I think they should be reverted to real functions again.

Is there any other opinion?

> The pgace worker process ... do your postmaster.c changes work when
> pgace is disabled?  I think it tries to start the worker on every
> iteration.

It is incorrect. When enhanced security is disabled, pgaceStartupWorkerProcess()
does nothing and always returns (pid_t) 0. It means there is no worker process.

> Maybe it needs more smarts in postmaster.c so that when
> !sepgsqlIsEnabled() it just doesn't try to start it up.

When user chooses selinux as 'pgace_feature', and sepgsqlIsEnabled()
returns true, it tries to start it up.
Yes, it is already done.

> Also, I think
> there should be a separate function to tell whether a particular
> PGACE_FEATURE actually needs a worker process; right now the only
> feature (SELinux) does need it, but is this the case for them all?

Basically, I think the core code should not invoke functions of
enhanced security features directly, as far as possible.
(GUC options and reloptions are exceptions.)

It is worthwhile that pgaceXXXX() wraps and turns on/off all the
security features, even if SELinux is currently the only guest,
because it enables to separate security related code so well.

> I didn't delve into many details in the avc, the worker, or anything
> much here actually -- I just skimmed randomly.  This is a really huge
> patch; sorry I do not have time right now to review it.

Never mind it.
I think your comments help us to improve SE-PostgreSQL more.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Updates of SE-PostgreSQL 8.4devel patches (r1408)

From

KaiGai Kohei

Date:

14 January 2009, 06:00:35

I updated patch set of SE-PostgreSQL and related stuff (r1408).

[1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1408.patch
[2/5] http://sepgsql.googlecode.com/files/sepostgresql-utils-8.4devel-3-r1408.patch
[3/5] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1408.patch
[4/5] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1408.patch
[5/5] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1408.patch

List of updates:
- It replaces some of elog() to raise user visible error messages,  by ereport() with proper error code.
- It replaces "static inline" functions by real ones, and moves them  to src/backend/security/pgaceHooks.c, to
eliminateGCC dependency.
 

These are based on Alvaro's comments.

Thanks for your help!
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1403)

From

Martijn van Oosterhout

Date:

14 January 2009, 06:35:22

On Tue, Jan 13, 2009 at 10:05:45AM -0300, Alvaro Herrera wrote:
> pgace.h: you have a bunch of "static inline" functions in here.  As far
> as I know this doesn't work in compilers other than GCC :-(  See
> pg_list.h (list_head) for an example.  I think we can tolerate this for
> the three functions in pg_list.h because they are so few and so tiny,
> but I'm not sure about PGACE because they are a large lot.  On the other
> hand, turning them to real functions would be a performance hit.

Really? C99 requires it and MSVC does support it. At least the other
compilers whose name I remembered (HP, Sun) support it also. I'd be
surprised if a compiler didn't since it's the form of inline that most
matches what people expect to happen.

Do you have an example?

http://www.greenend.org.uk/rjk/2003/03/inline.html
http://hi.baidu.com/junru/blog/item/4d8db11339050c856438db7a.html

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> Please line up in a tree and maintain the heap invariant while
> boarding. Thank you for flying nlogn airlines.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1403)

From

KaiGai Kohei

Date:

14 January 2009, 07:04:55

Martijn van Oosterhout wrote:
> On Tue, Jan 13, 2009 at 10:05:45AM -0300, Alvaro Herrera wrote:
>> pgace.h: you have a bunch of "static inline" functions in here.  As far
>> as I know this doesn't work in compilers other than GCC :-(  See
>> pg_list.h (list_head) for an example.  I think we can tolerate this for
>> the three functions in pg_list.h because they are so few and so tiny,
>> but I'm not sure about PGACE because they are a large lot.  On the other
>> hand, turning them to real functions would be a performance hit.
> 
> Really? C99 requires it and MSVC does support it. At least the other
> compilers whose name I remembered (HP, Sun) support it also. I'd be
> surprised if a compiler didn't since it's the form of inline that most
> matches what people expect to happen.
> 
> Do you have an example?

I have no preference either of them, because it is not an essence of
my patches whether its security hooks are implemented as inline, or not.

IIRC, indeed, some of compiler also supported "static inline".
However, it also seems to me that PostgreSQL implementation tend to
avoid to use inline functions actively.
For example, heap_getattr() and fastgetattr() are implemented as
macros, even if they have a bit complex conditional branches, which
can be rewritten more simple with inline functions.

If we have any policy to use inline functions, I'll follow this.

Thanks,

> http://www.greenend.org.uk/rjk/2003/03/inline.html
> http://hi.baidu.com/junru/blog/item/4d8db11339050c856438db7a.html
> 
> Have a nice day,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1403)

From

Alvaro Herrera

Date:

14 January 2009, 10:45:23

Martijn van Oosterhout wrote:
> On Tue, Jan 13, 2009 at 10:05:45AM -0300, Alvaro Herrera wrote:
> > pgace.h: you have a bunch of "static inline" functions in here.  As far
> > as I know this doesn't work in compilers other than GCC :-(  See
> > pg_list.h (list_head) for an example.  I think we can tolerate this for
> > the three functions in pg_list.h because they are so few and so tiny,
> > but I'm not sure about PGACE because they are a large lot.  On the other
> > hand, turning them to real functions would be a performance hit.
> 
> Really? C99 requires it and MSVC does support it.

Apparently we're stuck with C89 for some time yet; maybe an upgrade can
be argued, if a large gain by the jump to C99 can be demonstrated:

http://archives.postgresql.org/pgsql-hackers/2008-11/msg01494.php

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1403)

From

Bruce Momjian

Date:

14 January 2009, 13:46:45

KaiGai Kohei wrote:
> Martijn van Oosterhout wrote:
> > On Tue, Jan 13, 2009 at 10:05:45AM -0300, Alvaro Herrera wrote:
> >> pgace.h: you have a bunch of "static inline" functions in here.  As far
> >> as I know this doesn't work in compilers other than GCC :-(  See
> >> pg_list.h (list_head) for an example.  I think we can tolerate this for
> >> the three functions in pg_list.h because they are so few and so tiny,
> >> but I'm not sure about PGACE because they are a large lot.  On the other
> >> hand, turning them to real functions would be a performance hit.
> > 
> > Really? C99 requires it and MSVC does support it. At least the other
> > compilers whose name I remembered (HP, Sun) support it also. I'd be
> > surprised if a compiler didn't since it's the form of inline that most
> > matches what people expect to happen.
> > 
> > Do you have an example?
> 
> I have no preference either of them, because it is not an essence of
> my patches whether its security hooks are implemented as inline, or not.
> 
> IIRC, indeed, some of compiler also supported "static inline".
> However, it also seems to me that PostgreSQL implementation tend to
> avoid to use inline functions actively.
> For example, heap_getattr() and fastgetattr() are implemented as
> macros, even if they have a bit complex conditional branches, which
> can be rewritten more simple with inline functions.

I thought one advantage of using macros is that we force the inlining,
while I think inline compiler directives are more of a hint, but maybe
the compiler knows better than we do in some cases.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: Updates of SE-PostgreSQL 8.4devel patches (r1403)

From

Tom Lane

Date:

14 January 2009, 13:52:58

Martijn van Oosterhout <kleptog@svana.org> writes:
> On Tue, Jan 13, 2009 at 10:05:45AM -0300, Alvaro Herrera wrote:
>> pgace.h: you have a bunch of "static inline" functions in here.  As far
>> as I know this doesn't work in compilers other than GCC :-(

> Really? C99 requires it and MSVC does support it.

Wrong.  What C99 requires is a uselessly cumbersome form of "inline"
that is not compatible with the GCC feature.  We did actually implement
C99-compatible inlines in one or two places (in the sorting code IIRC),
but it's not something that I want to put up with on a large scale.

If you ran the current sepostgres patch through an actual C99 compiler,
it would fail.
        regards, tom lane

Re: Updates of SE-PostgreSQL 8.4devel patches (r1403)

From

Tom Lane

Date:

14 January 2009, 14:01:32

Bruce Momjian <bruce@momjian.us> writes:
> KaiGai Kohei wrote:
>> However, it also seems to me that PostgreSQL implementation tend to
>> avoid to use inline functions actively.

> I thought one advantage of using macros is that we force the inlining,

The (only) good thing about macros is they're portable: they work,
and work the same, on every C compiler.  This cannot be said of "inline".
        regards, tom lane

Re: Updates of SE-PostgreSQL 8.4devel patches (r1403)

From

KaiGai Kohei

Date:

14 January 2009, 14:12:03

Tom Lane wrote:
> If you ran the current sepostgres patch through an actual C99 compiler,
> it would fail.

The current one (r1408) is reworked to use normal functions, and inlines
are eliminated. :-)
 http://code.google.com/p/sepgsql/source/browse/trunk/sepgsql/src/include/security/sepgsql.h
http://code.google.com/p/sepgsql/source/browse/trunk/sepgsql/src/backend/security/pgaceHooks.c

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>

Re: Updates of SE-PostgreSQL 8.4devel patches (r1403)

From

"Robert Haas"

Date:

14 January 2009, 14:15:55

On Wed, Jan 14, 2009 at 10:00 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Bruce Momjian <bruce@momjian.us> writes:
>> KaiGai Kohei wrote:
>>> However, it also seems to me that PostgreSQL implementation tend to
>>> avoid to use inline functions actively.
>
>> I thought one advantage of using macros is that we force the inlining,
>
> The (only) good thing about macros is they're portable: they work,
> and work the same, on every C compiler.  This cannot be said of "inline".

Just out of curiosity, does C89, or whatever standard we follow, allow this?

int
somefunc(int x)
{   int foo[x];   /* use foo[] for scratch space */
}

Obviously this is a bad plan if x can be a big number because you
might crash your stack, but suppose we know that's not an issue?  It
seems a shame to have to do palloc/pfree in a situation like this.

...Robert

Re: Updates of SE-PostgreSQL 8.4devel patches (r1403)

From

Gregory Stark

Date:

14 January 2009, 14:34:45

"Robert Haas" <robertmhaas@gmail.com> writes:

> Just out of curiosity, does C89, or whatever standard we follow, allow this?
>
> int
> somefunc(int x)
> {
>     int foo[x];
>     /* use foo[] for scratch space */
> }

It's not in C89 but look up "alloca". 

We don't use it anywhere in postgres currently so it's kind of unlikely we
would start now.

I think C99 does allow what you typed, and I think gcc has an extension to
allow it too.

> Obviously this is a bad plan if x can be a big number because you
> might crash your stack, but suppose we know that's not an issue?  It
> seems a shame to have to do palloc/pfree in a situation like this.

palloc really isn't that expensive, unless you're allocating tons of tiny
objects or you're in a tight loop it's not worth worrying about.

--  Gregory Stark EnterpriseDB          http://www.enterprisedb.com Ask me about EnterpriseDB's Slony Replication
support!

Re: Updates of SE-PostgreSQL 8.4devel patches (r1403)

From

"Robert Haas"

Date:

14 January 2009, 17:45:19

> It's not in C89 but look up "alloca".

I know about alloca...

> We don't use it anywhere in postgres currently so it's kind of unlikely we
> would start now.

:-(

>> Obviously this is a bad plan if x can be a big number because you
>> might crash your stack, but suppose we know that's not an issue?  It
>> seems a shame to have to do palloc/pfree in a situation like this.
>
> palloc really isn't that expensive, unless you're allocating tons of tiny
> objects or you're in a tight loop it's not worth worrying about.

Yeah... but...

It really depends on what you compare it to.  It's cheap compared to
99% of the functions in the code base - perhaps so.  But it's darn
expensive compared to moving the stack pointer.  I have seen profiles
for PostgreSQL and other systems where memory management is a sizable
percentage of the CPU time, so it is not silly to worry about
economizing.

...Robert

Re: Updates of SE-PostgreSQL 8.4devel patches (r1403)

From

Martijn van Oosterhout

Date:

14 January 2009, 18:15:57

On Wed, Jan 14, 2009 at 09:52:20AM -0500, Tom Lane wrote:
> Martijn van Oosterhout <kleptog@svana.org> writes:
> > On Tue, Jan 13, 2009 at 10:05:45AM -0300, Alvaro Herrera wrote:
> >> pgace.h: you have a bunch of "static inline" functions in here.  As far
> >> as I know this doesn't work in compilers other than GCC :-(
>
> > Really? C99 requires it and MSVC does support it.
>
> Wrong.  What C99 requires is a uselessly cumbersome form of "inline"
> that is not compatible with the GCC feature.  We did actually implement
> C99-compatible inlines in one or two places (in the sorting code IIRC),
> but it's not something that I want to put up with on a large scale.

I was talking about "static inline", where C99 agrees completely with
GCC and is significantly more portable.

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> Please line up in a tree and maintain the heap invariant while
> boarding. Thank you for flying nlogn airlines.

Re: Updates of SE-PostgreSQL 8.4devel patches (r1403)

From

KaiGai Kohei

Date:

15 January 2009, 00:23:08

Martijn van Oosterhout wrote:
> On Wed, Jan 14, 2009 at 09:52:20AM -0500, Tom Lane wrote:
>> Martijn van Oosterhout <kleptog@svana.org> writes:
>>> On Tue, Jan 13, 2009 at 10:05:45AM -0300, Alvaro Herrera wrote:
>>>> pgace.h: you have a bunch of "static inline" functions in here.  As far
>>>> as I know this doesn't work in compilers other than GCC :-(
>>> Really? C99 requires it and MSVC does support it.
>> Wrong.  What C99 requires is a uselessly cumbersome form of "inline"
>> that is not compatible with the GCC feature.  We did actually implement
>> C99-compatible inlines in one or two places (in the sorting code IIRC),
>> but it's not something that I want to put up with on a large scale.
> 
> I was talking about "static inline", where C99 agrees completely with
> GCC and is significantly more portable.

As I noted yesterday, I have no preference between inline and real one.
However, I don't think it is a good idea to apply such an arguable manner
because of current v8.4 development schedule.

All patches are available here for a long time:
[1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1408.patch
[2/5] http://sepgsql.googlecode.com/files/sepostgresql-utils-8.4devel-3-r1408.patch
[3/5] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1408.patch
[4/5] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1408.patch
[5/5] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1408.patch

I would like committer to begin their reviews.
If necessary, I can rework/update them with my highest priority.

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>

Updates of SE-PostgreSQL 8.4devel patches (r1425)

From

KaiGai Kohei

Date:

16 January 2009, 08:39:40

I updated my patch set of SE-PostgreSQL and related stuff (r1425).

[1/5] http://sepgsql.googlecode.com/files/sepostgresql-sepgsql-8.4devel-3-r1425.patch
[2/5] http://sepgsql.googlecode.com/files/sepostgresql-utils-8.4devel-3-r1425.patch
[3/5] http://sepgsql.googlecode.com/files/sepostgresql-policy-8.4devel-3-r1425.patch
[4/5] http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1425.patch
[5/5] http://sepgsql.googlecode.com/files/sepostgresql-tests-8.4devel-3-r1425.patch

I tried to check my patches again, as if I am a reviewer in my mind.
Then, I could find some points to be fixed.
Please change reviewing base, if you saw the previous version partway.
(No changes here except for the following parts.)

List of updates:
- It rebased to the latest CVS HEAD.
- pgaceProxyQuery() is renamed to pgacePostRewriteQuery().  In the legacy version, SE-PostgreSQL modified WHERE clause
here, so it has name of "Proxy", but it become a nonsense name now.
 
- triggerIsForeignKeyConstraint() is replaced by  RI_FKey_trigger_type() due to code duplication.
- bugfix: avc_datum_count was not incremented on avc_make_entry()  correctly.
- Hook is reverted from fmgr_info_cxt() because it applies access  control on purely internal function usage. Now we
followthe  manner of pg_proc_aclcheck(), and add checks on system catalog  updates related to function usage.
 
- bugfix: When we update security_label, sepgsqlHeapTupleUpdate()  checked *:{relabelfrom} permission twice. It was
redundant.
- Security policy was updated to fit latest selinux-policy package.


Folks in pgsql-hackers,

My patch set has grew up a bit large for this two and half years, indeed,
but most of them are deployments of security hooks, and well modulared.
Don't hesitate code reviewing and comment anything, please.
We need any volunteers so much, even if you cannot provide comprehensive
reviewing. If necessary, I shall pay my efforts to update them with highest
priority, to get it merged at v8.4.


$ diffstat sepostgresql-sepgsql-8.4devel-3-r1425.patch configure                                     |  113 +
configure.in                                 |   13 src/Makefile.global.in                        |    1
src/backend/Makefile                         |    7 src/backend/access/common/heaptuple.c         |   35
src/backend/access/common/reloptions.c       |   22 src/backend/access/common/tupdesc.c           |   12
src/backend/access/heap/heapam.c             |   19 src/backend/access/heap/tuptoaster.c          |   19
src/backend/bootstrap/bootparse.y            |   13 src/backend/bootstrap/bootstrap.c             |    8
src/backend/catalog/Makefile                 |    1 src/backend/catalog/aclchk.c                  |    2
src/backend/catalog/catalog.c                |    4 src/backend/catalog/heap.c                    |   91 !
src/backend/catalog/index.c                  |   16 src/backend/catalog/pg_aggregate.c            |    3
src/backend/catalog/pg_largeobject.c         |    5 src/backend/catalog/pg_proc.c                 |    6
src/backend/catalog/toasting.c               |    3 src/backend/commands/cluster.c                |   11
src/backend/commands/copy.c                  |  293 +++! src/backend/commands/dbcommands.c             |   20
src/backend/commands/functioncmds.c          |   29 src/backend/commands/lockcmds.c               |    3
src/backend/commands/proclang.c              |    6 src/backend/commands/tablecmds.c              |   23
src/backend/commands/trigger.c               |   25 src/backend/executor/execJunk.c               |    6
src/backend/executor/execMain.c              |  210 +++ src/backend/executor/execQual.c               |    4
src/backend/executor/execScan.c              |   40 src/backend/executor/execTuples.c             |   19
src/backend/executor/execUtils.c             |   10 src/backend/executor/functions.c              |    6
src/backend/executor/nodeAgg.c               |    5 src/backend/executor/nodeMergejoin.c          |    2
src/backend/executor/nodeSubplan.c           |    4 src/backend/executor/nodeWindowAgg.c          |    4
src/backend/executor/spi.c                   |    4 src/backend/libpq/be-fsstubs.c                |   16
src/backend/nodes/copyfuncs.c                |   44 src/backend/nodes/equalfuncs.c                |   34
src/backend/nodes/outfuncs.c                 |   41 src/backend/nodes/readfuncs.c                 |   36
src/backend/optimizer/plan/createplan.c      |    6 src/backend/optimizer/plan/planner.c          |    1
src/backend/optimizer/util/clauses.c         |    5 src/backend/optimizer/util/relnode.c          |    1
src/backend/parser/analyze.c                 |   49 src/backend/parser/gram.y                     |   64 !
src/backend/parser/parse_target.c            |   64 ! src/backend/postmaster/postmaster.c           |   43
src/backend/rewrite/rewriteHandler.c         |    3 src/backend/security/Makefile                 |   23
src/backend/security/pgaceCommon.c           |  729 ++++++++++++ src/backend/security/pgaceHooks.c             | 1524
++++++++++++++++++++++++++src/backend/security/rowacl/rowacl.c          |  721 ++++++++++++
src/backend/security/sepgsql/avc.c           | 1118 +++++++++++++++++++ src/backend/security/sepgsql/core.c           |
623 ++++++++++ src/backend/security/sepgsql/hooks.c          |  952 ++++++++++++++++
src/backend/security/sepgsql/permissions.c   |  785 +++++++++++++ src/backend/security/sepgsql/proxy.c          | 1134
+++++++++++++++++++src/backend/storage/file/fd.c                 |    7 src/backend/storage/ipc/ipci.c                |
  2 src/backend/tcop/fastpath.c                   |    2 src/backend/tcop/pquery.c                     |    2
src/backend/tcop/utility.c                   |    3 src/backend/utils/adt/acl.c                   |    6
src/backend/utils/adt/ri_triggers.c          |   25 src/backend/utils/adt/trigfuncs.c             |   11
src/backend/utils/cache/catcache.c           |   32 src/backend/utils/cache/plancache.c           |   12
src/backend/utils/cache/relcache.c           |   38 src/backend/utils/cache/syscache.c            |   40
src/backend/utils/fmgr/dfmgr.c               |   10 src/backend/utils/init/postinit.c             |    4
src/backend/utils/misc/guc.c                 |   58 src/backend/utils/misc/postgresql.conf.sample |    6
src/include/access/htup.h                    |   68 + src/include/access/sysattr.h                  |    9
src/include/access/tupdesc.h                 |    2 src/include/catalog/heap.h                    |   11
src/include/catalog/indexing.h               |    5 src/include/catalog/pg_attribute.h            |  495 !!!!!!!!
src/include/catalog/pg_class.h               |    2 src/include/catalog/pg_proc.h                 |   21
src/include/catalog/pg_proc_fn.h             |    3 src/include/catalog/pg_security.h             |   31
src/include/catalog/pg_type.h                |    1 src/include/executor/executor.h               |   11
src/include/executor/tuptable.h              |    4 src/include/fmgr.h                            |    3
src/include/libpq/be-fsstubs.h               |    3 src/include/nodes/nodes.h                     |    4
src/include/nodes/parsenodes.h               |   17 src/include/nodes/plannodes.h                 |   10
src/include/nodes/relation.h                 |    2 src/include/nodes/security.h                  |   45
src/include/pg_config.h.in                   |    3 src/include/security/pgace.h                  |  180 +++
src/include/security/rowacl.h                |   41 src/include/security/sepgsql.h                |  230 +++
src/include/storage/fd.h                     |    1 src/include/storage/lwlock.h                  |    1
src/include/utils/acl.h                      |    7 src/include/utils/catcache.h                  |    1
src/include/utils/errcodes.h                 |    7 src/include/utils/rel.h                       |   18
src/include/utils/syscache.h                 |    4 110 files changed, 9697 insertions(+), 16 deletions(-), 918
modifications(!)

Thanks,
-- 
OSS Platform Development Division, NEC
KaiGai Kohei <kaigai@ak.jp.nec.com>