Thread: Problems with casting

I've created a variant data type [1]. It seems to work pretty well,
except for some issues with casting.

Since the idea of the type is to allow storing any other data type, it
creates casts to and from all other types. At first these were all
marked as ASSIGNMENT, but that made using variant with functions quite
cumbersome. With functions that accepted a variant, you still had to
explicitly cast it:

SELECT variant_function( some_field::variant.variant ) FROM some_table;

I was reluctant to make the casts to variant IMPLICIT, but it seems like
it actually works rather well... except for arrays:

ERROR:  operator is not unique: regtype[] = regtype[]

This was true for all operators, not something unique to regtype[],
presumably because array_cmp() does something slightly different than
the rest of the system. I do have a = operator, but I do not have an
operator class.

For now, I work around this by leaving casts from arrays to variant as
ASSIGNMENT, but I'm wondering if there's a better solution to be had.

I could change my = operator to something else, but I believe that will
break things like IS DISTINCT.

I've wondered if creating an operator class would just fix this, but I'm
not sure. I'd also need a somewhat different comparison function because
right now I don't enforce that there's an operator class to do comparison.

I tried putting the operators into a different schema, but operator
lookup appears to ignore schema.

It's worth noting that the only problem I've seen so far has been
dealing with function calls. It reminds me of the surprise people run
into when they define a function that accepts smallint and then they
can't call it directly. I find myself wondering if there's some way to
handle this at the function call level.
--
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com

Jim Nasby <Jim.Nasby@BlueTreble.com> writes:
> I've created a variant data type [1]. It seems to work pretty well,
> except for some issues with casting.

> Since the idea of the type is to allow storing any other data type, it
> creates casts to and from all other types. At first these were all
> marked as ASSIGNMENT, but that made using variant with functions quite
> cumbersome. With functions that accepted a variant, you still had to
> explicitly cast it:

> SELECT variant_function( some_field::variant.variant ) FROM some_table;

> I was reluctant to make the casts to variant IMPLICIT, but it seems like
> it actually works rather well... except for arrays:

I suspect that that's only the tip of the iceberg.  Remember the mess
we had with implicit casts to text?  And those only existed for a dozen
or so types, not for everything.  Every function or operator you define
for "variant" is going to be a loaded gun just waiting to shoot your foot
off, if you make all those casts implicit.

            regards, tom lane

On 4/7/15 4:17 PM, Tom Lane wrote:
> Jim Nasby <Jim.Nasby@BlueTreble.com> writes:
>> I've created a variant data type [1]. It seems to work pretty well,
>> except for some issues with casting.
>
>> Since the idea of the type is to allow storing any other data type, it
>> creates casts to and from all other types. At first these were all
>> marked as ASSIGNMENT, but that made using variant with functions quite
>> cumbersome. With functions that accepted a variant, you still had to
>> explicitly cast it:
>
>> SELECT variant_function( some_field::variant.variant ) FROM some_table;
>
>> I was reluctant to make the casts to variant IMPLICIT, but it seems like
>> it actually works rather well... except for arrays:
>
> I suspect that that's only the tip of the iceberg.  Remember the mess
> we had with implicit casts to text?  And those only existed for a dozen
> or so types, not for everything.  Every function or operator you define
> for "variant" is going to be a loaded gun just waiting to shoot your foot
> off, if you make all those casts implicit.

Yeah, that's why I avoided it. But that makes using it in a function a
real pain. :( I think this is a bit of a different scenario though,
because I don't see why you'd want to overload a function to accept both
variant and some other type.

Really what I want is for casting to variant to be a last-choice option,
and even then only for function calls, not operators. I believe that
would be safe, because then you'd have to explicitly be calling a
function, or explicitly doing something::variant = variant.

The other option I thought of was controlling this better by putting the
variant operators in their own schema, but that didn't work.
--
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com

Jim Nasby <Jim.Nasby@BlueTreble.com> writes:
> On 4/7/15 4:17 PM, Tom Lane wrote:
>> I suspect that that's only the tip of the iceberg.  Remember the mess
>> we had with implicit casts to text?  And those only existed for a dozen
>> or so types, not for everything.  Every function or operator you define
>> for "variant" is going to be a loaded gun just waiting to shoot your foot
>> off, if you make all those casts implicit.

> Yeah, that's why I avoided it. But that makes using it in a function a
> real pain. :( I think this is a bit of a different scenario though,
> because I don't see why you'd want to overload a function to accept both
> variant and some other type.

> Really what I want is for casting to variant to be a last-choice option,
> and even then only for function calls, not operators. I believe that
> would be safe, because then you'd have to explicitly be calling a
> function, or explicitly doing something::variant = variant.

Just out of curiosity, what's the point of this type at all, compared
to "anyelement" and friends?

            regards, tom lane

On 4/7/15 4:35 PM, Tom Lane wrote:
> Jim Nasby <Jim.Nasby@BlueTreble.com> writes:
>> On 4/7/15 4:17 PM, Tom Lane wrote:
>>> I suspect that that's only the tip of the iceberg.  Remember the mess
>>> we had with implicit casts to text?  And those only existed for a dozen
>>> or so types, not for everything.  Every function or operator you define
>>> for "variant" is going to be a loaded gun just waiting to shoot your foot
>>> off, if you make all those casts implicit.
>
>> Yeah, that's why I avoided it. But that makes using it in a function a
>> real pain. :( I think this is a bit of a different scenario though,
>> because I don't see why you'd want to overload a function to accept both
>> variant and some other type.
>
>> Really what I want is for casting to variant to be a last-choice option,
>> and even then only for function calls, not operators. I believe that
>> would be safe, because then you'd have to explicitly be calling a
>> function, or explicitly doing something::variant = variant.
>
> Just out of curiosity, what's the point of this type at all, compared
> to "anyelement" and friends?

The two big differences are that you can store a variant in a table
(with reasonable protection against things like dropping the underlying
type out from under it), and you can readily determine what the original
type was. Well, and you're not limited to a single type in a function as
you are with polymorphic.

One place I've wanted this in the past is to allow storing settings or
other configuration in the database. Currently you're stuck either
casting everything to and from text or having a bunch of fields. With
variant you just store what you're handed.

The other thing I'm currently working on is a template system that would
allow you to use whatever type you wanted to pass data to a template
(and for the template itself), as well as allowing you to store
templates for later re-use. The nice thing about variant in this context
is that the framework itself doesn't really need to care about what's
being passed through it. If it didn't support storing templates I could
probably get away with anyelement for this; but that kinda defeats the
purpose.

I think there's a chicken and egg problem here. I've pondered variant
for several years and never thought of anything better than the case of
storing settings, which was hardly compelling enough to invest the work.
I finally decided to do it anyway just to see what would be required.
Only after I had something working did it occur to me that I could use
this to build a template system. It's certainly possible that there
isn't all that compelling of a case for variants afterall, but I don't
think they'll get a fair shake unless there's something available that's
pretty workable. I suspect there's actually some rather interesting
things it could be used for if people start thinking about it.

Your question does raise an interesting thought though... is there some
way I could leverage the polymorphic system here? I did experiment with
having functions accept anyelement instead of a variant and had some
success with that (though IIRC plpgsql tended to revolt when trying to
assign that to a variant in older versions).
--
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com

Jim Nasby <Jim.Nasby@BlueTreble.com> writes:
> On 4/7/15 4:35 PM, Tom Lane wrote:
>> Just out of curiosity, what's the point of this type at all, compared
>> to "anyelement" and friends?

> The two big differences are that you can store a variant in a table
> (with reasonable protection against things like dropping the underlying
> type out from under it), and you can readily determine what the original
> type was. Well, and you're not limited to a single type in a function as
> you are with polymorphic.

I'm fairly skeptical of the idea that you should want to store a variant
in a table --- smells of EAV schema design to me.  What would a unique
index mean on such a column, for instance?  As for the other two, the only
reason you can't do them with polymorphic arguments is nobody has wanted
them bad enough to do something about it.

            regards, tom lane

On 4/7/15 5:56 PM, David G. Johnston wrote:
> On Tue, Apr 7, 2015 at 3:09 PM, Jim Nasby <Jim.Nasby@bluetreble.com
> <mailto:Jim.Nasby@bluetreble.com>>wrote:
>
>     On 4/7/15 4:35 PM, Tom Lane wrote:
>
>         Jim Nasby <Jim.Nasby@BlueTreble.com> writes:
>
>             On 4/7/15 4:17 PM, Tom Lane wrote:
>
>                 I suspect that that's only the tip of the iceberg.
>                 Remember the mess
>                 we had with implicit casts to text?  And those only
>                 existed for a dozen
>                 or so types, not for everything.  Every function or
>                 operator you define
>                 for "variant" is going to be a loaded gun just waiting
>                 to shoot your foot
>                 off, if you make all those casts implicit.
>
>
>             Yeah, that's why I avoided it. But that makes using it in a
>             function a
>             real pain. :( I think this is a bit of a different scenario
>             though,
>             because I don't see why you'd want to overload a function to
>             accept both
>             variant and some other type.
>
>
>             Really what I want is for casting to variant to be a
>             last-choice option,
>             and even then only for function calls, not operators. I
>             believe that
>             would be safe, because then you'd have to explicitly be
>             calling a
>             function, or explicitly doing something::variant = variant.
>
>
>         Just out of curiosity, what's the point of this type at all,
>         compared
>         to "anyelement" and friends?
>
>
>     The two big differences are that you can store a variant in a table
>     (with reasonable protection against things like dropping the
>     underlying type out from under it), and you can readily determine
>     what the original type was.
>     ​ ​
>     Well, and you're not limited to a single type in a function as you
>     are with polymorphic.
>
>
>     One place I've wanted this in the past is to allow storing settings
>     or other configuration in the database. Currently you're stuck
>     either casting everything to and from text or having a bunch of
>     fields. With variant you just store what you're handed.
>
>     The other thing I'm currently working on is a template system that
>     would allow you to use whatever type you wanted to pass data to a
>     template (and for the template itself), as well as allowing you to
>     store templates for later re-use. The nice thing about variant in
>     this context is that the framework itself doesn't really need to
>     care about what's being passed through it. If it didn't support
>     storing templates I could probably get away with anyelement for
>     this; but that kinda defeats the purpose.
>
>     I think there's a chicken and egg problem here. I've pondered
>     variant for several years and never thought of anything better than
>     the case of storing settings, which was hardly compelling enough to
>     invest the work. I finally decided to do it anyway just to see what
>     would be required. Only after I had something working did it occur
>     to me that I could use this to build a template system. It's
>     certainly possible that there isn't all that compelling of a case
>     for variants afterall, but I don't think they'll get a fair shake
>     unless there's something available that's pretty workable. I suspect
>     there's actually some rather interesting things it could be used for
>     if people start thinking about it.
>
>     Your question does raise an interesting thought though... is there
>     some way I could leverage the polymorphic system here? I did
>     experiment with having functions accept anyelement instead of a
>     variant and had some success with that (though IIRC plpgsql tended
>     to revolt when trying to assign that to a variant in older versions).
>
>
> ​I recently posited a use for an "anyelement"-like pseudo type that
> didn't have all the function restrictions of existing pseudo-types.
>
> http://www.postgresql.org/message-id/CAKFQuwazck37J7fA4pZOz8m9JsKMQQoPAftxRY0cA_n4R2xfbQ@mail.gmail.com
>
> The idea was to define a function with one pseudo-type and one generic
> (any) type that the caller is responsible for supplying a meaningful
> specific type that the function can act upon.  But this specific use
> would not need an actual type but only another pseudo-type.
> ​
>
> Given the nature of SQL, and PostgreSQL's implementation thereof, a
> storage "variant" type seems non-idiomatic and problematic in usage.
> Hell, my recollection is that our implementation of Domains has some
> meaningful hiccups when dealing with type promotion and base-type
> comparisons; and domains are considerably less complicated than "Variant"...

BTW, to answer Tom's question... I'm definitely NOT trying to use
variant to do EAV. I'm sure someone that thinks EAV is a good idea (NOT
me!) might get excited at being able to natively remember what the
original type was, but they're likely to have much bigger problems than
variant in the long run... ;)

My recollection on domains is that all the problems stem not from
storage but because parts of the system just ignore them. IE: not
enforcing the constraints, or not allowing casting. Basically, they're
not really fully implemented.

> Neither "settings" nor "templates" screams for a non-text solution; but
> I also haven't given topic much consideration.
>
> The typed text capability would allow for a simpler UI but for the
> limited cases where it is a valid model (e.g., a settings table) writing
> a function-based UI would provide a place to hook in the desired input
> validation without introducing a entirely new global concept.

For really simple cases (such as settings), I agree, it's not terribly
worth it.

For more interesting usage though, I think just casting everything to
text and crossing your fingers is pretty crappy. You no longer know what
the original type was, nor do you have any way to ensure you're getting
what you expect. If someone originally handed you 1::numeric and you
blindly cast that to int, it'll work. If they handed you 1.1::numeric
now you're in trouble. A variant can actually handle this correctly. Oh,
there's also typmods to consider. Variant does it's best to determine
the original typmod and remember it (though PG plays a bit fast and
loose with typmods in some places).

It's certainly possible that there really aren't any great use cases,
but what I've noticed is that when someone comes up with an idea that a
tool doesn't support, they either give up or start creating ugly hacks.
I don't think we'll see anyone doing anything truly interesting with a
variant type until one actually exists.

Now, if a few years from now there's still no one using variant, then I
guess there really is no point. :) But I don't see us discovering it
without the type existing in the first place.
--
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com