Thread: Re: [HACKERS] Arrays of Complex Types

Re: [HACKERS] Arrays of Complex Types

From
David Fetter
Date:
On Fri, Mar 30, 2007 at 05:08:42PM -0400, Tom Lane wrote:
> David Fetter <david@fetter.org> writes:
> > After several rounds of patches, it appears that it might be easier to
> > create a new typtype entry, which I'll tentatively call 'a' because it
> > seems a little fragile and a lot inelegant and hard to maintain to
> > have typtype='c' and typrelid=InvalidOid mean, "this is an array of
> > complex types."
>
> Uh, wouldn't it be typtype = 'c' and typelem != 0 ?

Right.  The attached patch passes the current regression tests and at
least to a "smoke test" level does what it's supposed to do.  I'd
really like to help refactor the whole array system to use 'a', tho.

Cheers,
D
--
David Fetter <david@fetter.org> http://fetter.org/
phone: +1 415 235 3778        AIM: dfetter666
                              Skype: davidfetter

Remember to vote!
Consider donating to PostgreSQL: http://www.postgresql.org/about/donate

Attachment

Re: [HACKERS] Arrays of Complex Types

From
Bruce Momjian
Date:
Your patch has been added to the PostgreSQL unapplied patches list at:

    http://momjian.postgresql.org/cgi-bin/pgpatches

It will be applied as soon as one of the PostgreSQL committers reviews
and approves it.

---------------------------------------------------------------------------


David Fetter wrote:
> On Fri, Mar 30, 2007 at 05:08:42PM -0400, Tom Lane wrote:
> > David Fetter <david@fetter.org> writes:
> > > After several rounds of patches, it appears that it might be easier to
> > > create a new typtype entry, which I'll tentatively call 'a' because it
> > > seems a little fragile and a lot inelegant and hard to maintain to
> > > have typtype='c' and typrelid=InvalidOid mean, "this is an array of
> > > complex types."
> >
> > Uh, wouldn't it be typtype = 'c' and typelem != 0 ?
>
> Right.  The attached patch passes the current regression tests and at
> least to a "smoke test" level does what it's supposed to do.  I'd
> really like to help refactor the whole array system to use 'a', tho.
>
> Cheers,
> D
> --
> David Fetter <david@fetter.org> http://fetter.org/
> phone: +1 415 235 3778        AIM: dfetter666
>                               Skype: davidfetter
>
> Remember to vote!
> Consider donating to PostgreSQL: http://www.postgresql.org/about/donate

[ Attachment, skipping... ]

>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
>        choose an index scan if your joining column's datatypes do not
>        match

--
  Bruce Momjian  <bruce@momjian.us>          http://momjian.us
  EnterpriseDB                               http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

Re: [HACKERS] Arrays of Complex Types

From
Alvaro Herrera
Date:
Bruce Momjian wrote:
>
> Your patch has been added to the PostgreSQL unapplied patches list at:
>
>     http://momjian.postgresql.org/cgi-bin/pgpatches
>
> It will be applied as soon as one of the PostgreSQL committers reviews
> and approves it.
>

So, hum, what happened to the idea of creating the array types only on
demand?

>
>
> David Fetter wrote:
> > On Fri, Mar 30, 2007 at 05:08:42PM -0400, Tom Lane wrote:
> > > David Fetter <david@fetter.org> writes:
> > > > After several rounds of patches, it appears that it might be easier to
> > > > create a new typtype entry, which I'll tentatively call 'a' because it
> > > > seems a little fragile and a lot inelegant and hard to maintain to
> > > > have typtype='c' and typrelid=InvalidOid mean, "this is an array of
> > > > complex types."
> > >
> > > Uh, wouldn't it be typtype = 'c' and typelem != 0 ?
> >
> > Right.  The attached patch passes the current regression tests and at
> > least to a "smoke test" level does what it's supposed to do.  I'd
> > really like to help refactor the whole array system to use 'a', tho.
> >
> > Cheers,


--
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Re: [HACKERS] Arrays of Complex Types

From
David Fetter
Date:
On Mon, Apr 02, 2007 at 10:01:44PM -0400, Alvaro Herrera wrote:
> Bruce Momjian wrote:
> >
> > Your patch has been added to the PostgreSQL unapplied patches list
> > at:
> >
> >     http://momjian.postgresql.org/cgi-bin/pgpatches
> >
> > It will be applied as soon as one of the PostgreSQL committers
> > reviews and approves it.
> >
>
> So, hum, what happened to the idea of creating the array types only
> on demand?

Scotched, as far as I could tell, partly due to nobody's having
actually done work toward such a thing, and partly because the closest
thing I've heard to an objection is pretty nebulous. :)

It's a lot simpler to have them always, and it fits in with the larger
picture of making arrays fully composable with other operations like
DOMAIN, ENUM and TYPE.

Cheers,
D
--
David Fetter <david@fetter.org> http://fetter.org/
phone: +1 415 235 3778        AIM: dfetter666
                              Skype: davidfetter

Remember to vote!
Consider donating to PostgreSQL: http://www.postgresql.org/about/donate

Re: [HACKERS] Arrays of Complex Types

From
Tom Lane
Date:
David Fetter <david@fetter.org> writes:
> On Mon, Apr 02, 2007 at 10:01:44PM -0400, Alvaro Herrera wrote:
>> So, hum, what happened to the idea of creating the array types only
>> on demand?

> Scotched, as far as I could tell,

More like "you submitted a patch that entirely ignores multiple people's
opinion on what is needed".

Bruce may have put this into the patch queue, but do not labor under
the delusion that that means it'll get applied as-is.  The queue is
currently operating as a list of open issues.

            regards, tom lane

Re: [HACKERS] Arrays of Complex Types

From
David Fetter
Date:
On Tue, Apr 03, 2007 at 02:30:07AM -0400, Tom Lane wrote:
> David Fetter <david@fetter.org> writes:
> > On Mon, Apr 02, 2007 at 10:01:44PM -0400, Alvaro Herrera wrote:
> >> So, hum, what happened to the idea of creating the array types
> >> only on demand?
>
> > Scotched, as far as I could tell,
>
> More like "you submitted a patch that entirely ignores multiple
> people's opinion on what is needed".
>
> Bruce may have put this into the patch queue, but do not labor under
> the delusion that that means it'll get applied as-is.

I assure you I'm not.  Two glaring things it's missing are regression
tests and documentation.  I should have those in this week.

Cheers,
D
--
David Fetter <david@fetter.org> http://fetter.org/
phone: +1 415 235 3778        AIM: dfetter666
                              Skype: davidfetter

Remember to vote!
Consider donating to PostgreSQL: http://www.postgresql.org/about/donate

Re: [HACKERS] Arrays of Complex Types

From
Bruce Momjian
Date:
Tom Lane wrote:
> David Fetter <david@fetter.org> writes:
> > On Mon, Apr 02, 2007 at 10:01:44PM -0400, Alvaro Herrera wrote:
> >> So, hum, what happened to the idea of creating the array types only
> >> on demand?
>
> > Scotched, as far as I could tell,
>
> More like "you submitted a patch that entirely ignores multiple people's
> opinion on what is needed".
>
> Bruce may have put this into the patch queue, but do not labor under
> the delusion that that means it'll get applied as-is.  The queue is
> currently operating as a list of open issues.

Correct.

--
  Bruce Momjian  <bruce@momjian.us>          http://momjian.us
  EnterpriseDB                               http://www.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

Re: [HACKERS] Arrays of Complex Types

From
Andrew Dunstan
Date:
Tom Lane wrote:
> David Fetter <david@fetter.org> writes:
>
>> On Mon, Apr 02, 2007 at 10:01:44PM -0400, Alvaro Herrera wrote:
>>
>>> So, hum, what happened to the idea of creating the array types only
>>> on demand?
>>>
>
>
>> Scotched, as far as I could tell,
>>
>
> More like "you submitted a patch that entirely ignores multiple people's
> opinion on what is needed".
>
> Bruce may have put this into the patch queue, but do not labor under
> the delusion that that means it'll get applied as-is.  The queue is
> currently operating as a list of open issues.
>
>

One of the things that's been bothering me about this proposal is that
it leaves untouched and indeed greatly expands the scope of the typename
mangling we do. (i.e. adding an entry to pg_type with _ prepended). Up
to now we've only used this gadget in a way that might matter a lot on
user defined non-composite types, I think, and now we have expanded that
to include enums, which are really a special case of user defined
non-composites which don't require an extra C module. That's a
comparatively small window, but this proposal will extend it to all
composites, which is quite a large expansion in scope. And since _ is a
perfectly legal initial char for an identifier, if type _foo exists then
any attempt to create a table or view or composite called foo will fail.

Is it possible to fix this, or am I trying to shut the stable door after
the horse has well and truly bolted? If it can be fixed, I'd like to see
it fixed before we fix the problem David is trying to address here.

It's been suggested to me that this is an insignificant corner case. But
I have often seen coding standards that actually require certain classes
of identifier to being with _, so it's very far from a merely
theoretical point.

I'm slightly inclined to agree with David that the danger of catalog
bloat isn't that great, and might not justify the extra work that some
sort of explicit array creation would involve (e.g. changes in grammar,
pg_dump), as long as we are agreed that we don't want array types ever
to have their own user definable names or settable namespace.

cheers

andrew


Re: [HACKERS] Arrays of Complex Types

From
Tom Lane
Date:
Andrew Dunstan <andrew@dunslane.net> writes:
> One of the things that's been bothering me about this proposal is that
> it leaves untouched and indeed greatly expands the scope of the typename
> mangling we do. (i.e. adding an entry to pg_type with _ prepended).

Yeah, that's been bothering me too.  One of the problems with the patch
as-is is that it extends the 62-instead-of-63-char limit to table names
as well as type names.

I've been thinking of proposing that we add a column to pg_type that
points from a type to its array type (if any), ie the reverse link
from typelem.  If we had that then the parser could follow that to
determine which type is foo[], instead of relying on the _foo naming
convention.  I don't suggest that we stop using the naming convention,
but it would no longer be a hard-and-fast rule, just a convention.
In particular we could rejigger things around the edges to reduce
the name conflict problem.  For instance the rule for forming array type
names could be "prepend _, truncate to less than 64 bytes if necessary,
then substitute numbers at the end if needed to get something unique".
This is not all that different from what we do now to get unique
serial sequence names, for example.

This would also open the door to supporting

CREATE TYPE foo AS ARRAY OF bar

without having to have any restrictions about the name of foo.
I'd still much rather do things that way for arrays of composites
than invent a ton of pg_type entries that are mostly going to go
unused.

            regards, tom lane

PS: Has anyone looked at what it will take to make the entries
in an array-of-composite be something smaller than full tuples?
It's not going to be anything but a toy unless you can get the
per-entry overhead down to something sane.  Perhaps the
MinimalTuple representation would work.

Re: [HACKERS] Arrays of Complex Types

From
David Fetter
Date:
On Sun, Apr 08, 2007 at 07:08:38PM -0400, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
> > One of the things that's been bothering me about this proposal is that
> > it leaves untouched and indeed greatly expands the scope of the typename
> > mangling we do. (i.e. adding an entry to pg_type with _ prepended).
>
> Yeah, that's been bothering me too.  One of the problems with the patch
> as-is is that it extends the 62-instead-of-63-char limit to table names
> as well as type names.

I did this by copying some code which already creates array names, so
should that code change to do something different, the
62-instead-of-63-char thing would go away along with it.  I agree that
the prepended _s are far from optimal.

> I've been thinking of proposing that we add a column to pg_type that
> points from a type to its array type (if any), ie the reverse link
> from typelem.  If we had that then the parser could follow that to
> determine which type is foo[], instead of relying on the _foo naming
> convention.  I don't suggest that we stop using the naming convention,
> but it would no longer be a hard-and-fast rule, just a convention.

That'd be neat :)

> In particular we could rejigger things around the edges to reduce
> the name conflict problem.  For instance the rule for forming array type
> names could be "prepend _, truncate to less than 64 bytes if necessary,
> then substitute numbers at the end if needed to get something unique".
> This is not all that different from what we do now to get unique
> serial sequence names, for example.
>
> This would also open the door to supporting
>
> CREATE TYPE foo AS ARRAY OF bar

I'm sorry to keep harping on this, but I really don't see a use case
and do see foot guns both with making the array types optional and
with decoupling their names from those of their respective compound
types.  When they're optional, we get all kinds of "stepping on a step
that isn't there" issues, and when they're decoupled, operations like,
"ALTER TABLE foo RENAME TO bar" have either surprising or undefined
behavior, or both.

> without having to have any restrictions about the name of foo.
> I'd still much rather do things that way for arrays of composites
> than invent a ton of pg_type entries that are mostly going to go
> unused.

I'm sure there's a better way than my first attempt.

> PS: Has anyone looked at what it will take to make the entries in an
> array-of-composite be something smaller than full tuples?  It's not
> going to be anything but a toy unless you can get the per-entry
> overhead down to something sane.  Perhaps the MinimalTuple
> representation would work.

Sounds neat, too :)

Cheers,
D
--
David Fetter <david@fetter.org> http://fetter.org/
phone: +1 415 235 3778        AIM: dfetter666
                              Skype: davidfetter

Remember to vote!
Consider donating to PostgreSQL: http://www.postgresql.org/about/donate

Re: [HACKERS] Arrays of Complex Types

From
Andrew Dunstan
Date:
Tom Lane wrote:
>
> I've been thinking of proposing that we add a column to pg_type that
> points from a type to its array type (if any), ie the reverse link
> from typelem.  If we had that then the parser could follow that to
> determine which type is foo[], instead of relying on the _foo naming
> convention.

good.


> I don't suggest that we stop using the naming convention,
> but it would no longer be a hard-and-fast rule, just a convention.
> In particular we could rejigger things around the edges to reduce
> the name conflict problem.  For instance the rule for forming array type
> names could be "prepend _, truncate to less than 64 bytes if necessary,
> then substitute numbers at the end if needed to get something unique".
> This is not all that different from what we do now to get unique
> serial sequence names, for example.
>

Sounds OK but I'd add something that might make it even more unlikely to
generate a name clash.

> This would also open the door to supporting
>
> CREATE TYPE foo AS ARRAY OF bar
>
> without having to have any restrictions about the name of foo.
> I'd still much rather do things that way for arrays of composites
> than invent a ton of pg_type entries that are mostly going to go
> unused.
>
>
>

ISTM we should either do it all automatically or all manually. If you
want user defined names for array types then we can forget name mangling
for user defined types and do everything manually.

cheers

andrew



Re: [HACKERS] Arrays of Complex Types

From
Tom Lane
Date:
Andrew Dunstan <andrew@dunslane.net> writes:
> Tom Lane wrote:
>> I don't suggest that we stop using the naming convention,
>> but it would no longer be a hard-and-fast rule, just a convention.
>> In particular we could rejigger things around the edges to reduce
>> the name conflict problem.  For instance the rule for forming array type
>> names could be "prepend _, truncate to less than 64 bytes if necessary,
>> then substitute numbers at the end if needed to get something unique".
>> This is not all that different from what we do now to get unique
>> serial sequence names, for example.

> Sounds OK but I'd add something that might make it even more unlikely to
> generate a name clash.

Like what?  I don't want to stray far from _foo when we don't have to,
because I'm sure there is user code out there that'll still rely on
that naming convention; we shouldn't break it if we don't have to.

            regards, tom lane

Re: [HACKERS] Arrays of Complex Types

From
Andrew Dunstan
Date:
Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>
>> Tom Lane wrote:
>>
>>> I don't suggest that we stop using the naming convention,
>>> but it would no longer be a hard-and-fast rule, just a convention.
>>> In particular we could rejigger things around the edges to reduce
>>> the name conflict problem.  For instance the rule for forming array type
>>> names could be "prepend _, truncate to less than 64 bytes if necessary,
>>> then substitute numbers at the end if needed to get something unique".
>>> This is not all that different from what we do now to get unique
>>> serial sequence names, for example.
>>>
>
>
>> Sounds OK but I'd add something that might make it even more unlikely to
>> generate a name clash.
>>
>
> Like what?  I don't want to stray far from _foo when we don't have to,
> because I'm sure there is user code out there that'll still rely on
> that naming convention; we shouldn't break it if we don't have to.
>
>
>
Oh, in that case maybe we'd better live with it :-(

I certainly think we should deprecate relying on it.

cheers

andrew


Re: [HACKERS] Arrays of Complex Types

From
Tom Lane
Date:
Andrew Dunstan <andrew@dunslane.net> writes:
> I'm slightly inclined to agree with David that the danger of catalog
> bloat isn't that great, and might not justify the extra work that some
> sort of explicit array creation would involve (e.g. changes in grammar,
> pg_dump), as long as we are agreed that we don't want array types ever
> to have their own user definable names or settable namespace.

I did some tests just now to determine the total number of catalog
entries associated with a simple table definition.  Assuming it has
N user columns of built-in types (hence not requiring pg_depend entries
for the datatypes), I count

1 pg_class entry for the table itself
1 pg_type entry for the rowtype
N + 6 pg_attribute entries for the user and system columns
2 pg_depend entries (type -> table and table -> namespace)
2 pg_shdepend entries (ownership of table and type)

Of course this goes up *fast* if you need a toast table, indexes,
constraints, etc, but that's the irreducible minimum.

Generating an array rowtype would add three more catalog entries to this
(the array pg_type entry, a pg_depend arraytype->rowtype link, and
another pg_shdepend entry), which isn't a huge percentage overhead.
Obviously if we wanted to trim some fat here, getting rid of the
redundant pg_attribute entries for system columns would be the first
place to look.

Based on this, I withdraw my efficiency concern about generating
rowtypes for all user tables.  I do, however, still object to generating
them for system tables.  In particular an array type for pg_statistic
will actively Not Work and probably constitute a security hole, because
of the "anyarray" hack we use there.

BTW, I just noticed that we currently create array types with AUTO
dependencies on their element type, meaning that you can drop them
separately:

regression=# create type fooey as enum ('a','b');
CREATE TYPE
regression=# drop type _fooey;
DROP TYPE

Is this a bad idea?  If we made the dependency INTERNAL then the
system would refuse the drop above.  I think we would have to do
that if we wanted to add the base->array link I suggested, because
otherwise this drop would leave a dangling pointer in pg_type.

            regards, tom lane

Re: [HACKERS] Arrays of Complex Types

From
Andrew Dunstan
Date:
Tom Lane wrote:
>
> Based on this, I withdraw my efficiency concern about generating
> rowtypes for all user tables.  I do, however, still object to generating
> them for system tables.  In particular an array type for pg_statistic
> will actively Not Work and probably constitute a security hole, because
> of the "anyarray" hack we use there.
>
>

How would we do that? Not create the array types in bootstrap mode? Or
just special-case pg_statistic?

cheers

andrew

Re: [HACKERS] Arrays of Complex Types

From
Tom Lane
Date:
Andrew Dunstan <andrew@dunslane.net> writes:
> How would we do that? Not create the array types in bootstrap mode? Or
> just special-case pg_statistic?

Not generate them in bootstrap mode works for me.  IIRC, there's code
somewhere in there that allows anyarray to pass as a column type in
bootstrap mode, so that seems to fit ...

            regards, tom lane

Re: [HACKERS] Arrays of Complex Types

From
Andrew Dunstan
Date:
Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>
>> How would we do that? Not create the array types in bootstrap mode? Or
>> just special-case pg_statistic?
>>
>
> Not generate them in bootstrap mode works for me.  IIRC, there's code
> somewhere in there that allows anyarray to pass as a column type in
> bootstrap mode, so that seems to fit ...
>
>
>

OK, summarising what looks to me like a consensus position, ISTM the
plan could be:

. fix makeArrayTypeName() or similar to make it try harder to generate a
unique non-clashing name
. remove the existing "62 instead of 63" name length restrictions
. autogenerate array types for all explicitly or implicitly created
composite types other than for system catalog objects.
. defer for the present any consideration of a "CREATE TYPE foo AS ARRAY
..." command.

Regarding catalog objects, we might have to try a little harder than
just not generating in bootstrap mode - IIRC we generate system views
(including pg_stats) in non-bootstrap mode. Maybe we just need to exempt
anything in the pg_catalog namespace. What would happen if a user
created a view over pg_statistic? Should the test be to avoid arrays for
things that depend on the catalogs? Or maybe we should go to the heart
of the problem and simply check for pseudo-types directly.

cheers

andrew


Re: [HACKERS] Arrays of Complex Types

From
Tom Lane
Date:
Andrew Dunstan <andrew@dunslane.net> writes:
> Regarding catalog objects, we might have to try a little harder than
> just not generating in bootstrap mode - IIRC we generate system views
> (including pg_stats) in non-bootstrap mode. Maybe we just need to exempt
> anything in the pg_catalog namespace. What would happen if a user
> created a view over pg_statistic?

Nothing:

regression=# create view vvv as select * from pg_statistic;
ERROR:  column "stavalues1" has pseudo-type anyarray

which means we do have an issue for the pg_stats view.  Now that I look
instead of guessing, the existing test in CheckAttributeType is not on
bootstrap mode but standalone mode:

        /* Special hack for pg_statistic: allow ANYARRAY during initdb */
        if (atttypid != ANYARRAYOID || IsUnderPostmaster)
            ereport(ERROR,
                    (errcode(ERRCODE_INVALID_TABLE_DEFINITION),
                     errmsg("column \"%s\" has pseudo-type %s",
                            attname, format_type_be(atttypid))));

so for consistency we should use the same condition to suppress types
for system catalogs.

> Or maybe we should go to the heart
> of the problem and simply check for pseudo-types directly.

Actually we may have an issue already:

regression=# create table zzz (f1 pg_statistic);
CREATE TABLE

I couldn't make it misbehave in a short amount of trying:

regression=# insert into zzz
values(row(0,0,0,0,0,0,0,0,0,0,0,0,0,array[1,2],null,null,null,array[12,13],null,null,null));
ERROR:  ROW() column has type integer[] instead of type anyarray

but I don't feel comfortable about this at all.  Maybe
CheckAttributeType should be made to recurse into composite columns.

            regards, tom lane

Re: [HACKERS] Arrays of Complex Types

From
David Fetter
Date:
On Mon, Apr 09, 2007 at 10:40:49AM -0400, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
> > Regarding catalog objects, we might have to try a little harder than
> > just not generating in bootstrap mode - IIRC we generate system views
> > (including pg_stats) in non-bootstrap mode. Maybe we just need to exempt
> > anything in the pg_catalog namespace. What would happen if a user
> > created a view over pg_statistic?
>
> Nothing:
>
> regression=# create view vvv as select * from pg_statistic;
> ERROR:  column "stavalues1" has pseudo-type anyarray
>
> which means we do have an issue for the pg_stats view.  Now that I look
> instead of guessing, the existing test in CheckAttributeType is not on
> bootstrap mode but standalone mode:
>
>         /* Special hack for pg_statistic: allow ANYARRAY during initdb */
>         if (atttypid != ANYARRAYOID || IsUnderPostmaster)
>             ereport(ERROR,
>                     (errcode(ERRCODE_INVALID_TABLE_DEFINITION),
>                      errmsg("column \"%s\" has pseudo-type %s",
>                             attname, format_type_be(atttypid))));
>
> so for consistency we should use the same condition to suppress types
> for system catalogs.

Groovy :)

> > Or maybe we should go to the heart
> > of the problem and simply check for pseudo-types directly.
>
> Actually we may have an issue already:
>
> regression=# create table zzz (f1 pg_statistic);
> CREATE TABLE
>
> I couldn't make it misbehave in a short amount of trying:
>
> regression=# insert into zzz
values(row(0,0,0,0,0,0,0,0,0,0,0,0,0,array[1,2],null,null,null,array[12,13],null,null,null));
> ERROR:  ROW() column has type integer[] instead of type anyarray
>
> but I don't feel comfortable about this at all.  Maybe
> CheckAttributeType should be made to recurse into composite columns.

That'd be great :)

Cheers,
D
--
David Fetter <david@fetter.org> http://fetter.org/
phone: +1 415 235 3778        AIM: dfetter666
                              Skype: davidfetter

Remember to vote!
Consider donating to PostgreSQL: http://www.postgresql.org/about/donate

Re: [HACKERS] Arrays of Complex Types

From
Martijn van Oosterhout
Date:
On Mon, Apr 09, 2007 at 10:14:41AM -0400, Andrew Dunstan wrote:
> . defer for the present any consideration of a "CREATE TYPE foo AS ARRAY
> ..." command.

What is the rationale for allowing people to name the array type. When
I originally proposed the syntax I presumed that the array name would
be kept internal and hidden from the user, just that it would exist
after that command.

What possible reason is there for allowing the user to give the array
type a name?

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Attachment

Re: [HACKERS] Arrays of Complex Types

From
Andrew Dunstan
Date:
Martijn van Oosterhout wrote:
> On Mon, Apr 09, 2007 at 10:14:41AM -0400, Andrew Dunstan wrote:
>
>> . defer for the present any consideration of a "CREATE TYPE foo AS ARRAY
>> ..." command.
>>
>
> What is the rationale for allowing people to name the array type. When
> I originally proposed the syntax I presumed that the array name would
> be kept internal and hidden from the user, just that it would exist
> after that command.
>
> What possible reason is there for allowing the user to give the array
> type a name?
>
> Have a nice day,
>

Some type systems have named array types, some don't. I can live happily
with either. Are array types anonymous in the standard?

At any rate, the point of the remark was to take this off the table for now.

cheers

andrew

Re: [HACKERS] Arrays of Complex Types

From
Martijn van Oosterhout
Date:
On Mon, Apr 09, 2007 at 04:07:16PM -0400, Andrew Dunstan wrote:
> Some type systems have named array types, some don't. I can live happily
> with either. Are array types anonymous in the standard?

Yes, they're anonymous in the standard. That doesn't mean we can't give
them names if we wanted...

> At any rate, the point of the remark was to take this off the table for now.

Sure, once the array types are created automatically the command
becomes completely redundant.

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Attachment

Re: [HACKERS] Arrays of Complex Types

From
Andrew Dunstan
Date:

I wrote:
>
> OK, summarising what looks to me like a consensus position, ISTM the
> plan could be:
>
> . fix makeArrayTypeName() or similar to make it try harder to generate
> a unique non-clashing name
> . remove the existing "62 instead of 63" name length restrictions
> . autogenerate array types for all explicitly or implicitly created
> composite types other than for system catalog objects.
> . defer for the present any consideration of a "CREATE TYPE foo AS
> ARRAY ..." command.
>
> Regarding catalog objects, we might have to try a little harder than
> just not generating in bootstrap mode - IIRC we generate system views
> (including pg_stats) in non-bootstrap mode. Maybe we just need to
> exempt anything in the pg_catalog namespace. What would happen if a
> user created a view over pg_statistic? Should the test be to avoid
> arrays for things that depend on the catalogs? Or maybe we should go
> to the heart of the problem and simply check for pseudo-types directly.
>
>

I've been working on David's patch and done the following:

. inhibit creation of array types for composites during initdb
. some bug fixes
. have CheckAttributeType recurse into composite types, so you can no
longer create a table/type with a composite field which contains a
pseudo-type column (like pg_statistic)

However, there are still some oddities. For example, a change to or
removal of the base type affects the array type, but the array type can
be directly operated on (e.g. alter type _aa set schema foo ). I'm
inclined to say we should prevent direct operations on array types, and
they should live or die by their parent types.

Thoughts?

cheers

andrew



Re: [HACKERS] Arrays of Complex Types

From
David Fetter
Date:
On Sun, May 06, 2007 at 01:33:47PM -0400, Andrew Dunstan wrote:

> However, there are still some oddities. For example, a change to or
> removal of the base type affects the array type, but the array type
> can be directly operated on (e.g. alter type _aa set schema foo ).
> I'm inclined to say we should prevent direct operations on array
> types, and they should live or die by their parent types.
>
> Thoughts?

+1 on binding the array types tightly to the parent types.

Cheers,
D
--
David Fetter <david@fetter.org> http://fetter.org/
phone: +1 415 235 3778        AIM: dfetter666
                              Skype: davidfetter

Remember to vote!
Consider donating to PostgreSQL: http://www.postgresql.org/about/donate

Re: [HACKERS] Arrays of Complex Types

From
Tom Lane
Date:
Awhile back I wrote:
> I did some tests just now to determine the total number of catalog
> entries associated with a simple table definition.  Assuming it has
> N user columns of built-in types (hence not requiring pg_depend entries
> for the datatypes), I count

> 1 pg_class entry for the table itself
> 1 pg_type entry for the rowtype
> N + 6 pg_attribute entries for the user and system columns
> 2 pg_depend entries (type -> table and table -> namespace)
> 2 pg_shdepend entries (ownership of table and type)

> Of course this goes up *fast* if you need a toast table, indexes,
> constraints, etc, but that's the irreducible minimum.

> Generating an array rowtype would add three more catalog entries to this
> (the array pg_type entry, a pg_depend arraytype->rowtype link, and
> another pg_shdepend entry), which isn't a huge percentage overhead.
> Obviously if we wanted to trim some fat here, getting rid of the
> redundant pg_attribute entries for system columns would be the first
> place to look.

BTW, in the array patch as just committed, I was able to get rid of the
pg_shdepend entries for a table rowtype (when it's not a free-standing
composite type) and for an array type; instead they indirectly depend
on the owner of the parent table or element type respectively.  So the
net increase from 8.2 is only one catalog entry (we save one existing
pg_shdepend entry for the rowtype, and then add a pg_type entry for the
array type and a pg_depend entry to link it to the rowtype).

            regards, tom lane