Thread: Disallow arrays with non-standard lower bounds

Disallow arrays with non-standard lower bounds

From
Jim Nasby
Date:
ISTM that allowing users to pick arbitrary lower array bounds was a huge mistake. I've never seen anyone make use of
it,can't think of any legitimate use cases for it, and hate the stupendous amount of extra code needed to deal with
it.

Obviously we can't just drop support, but what about an initdb (or hell, even configure) option to disallow arrays with
alower bound <> 1? Unfortunately we can't do this with a GUC since you can store arrays in a table.
 
-- 
Jim C. Nasby, Data Architect                       jim@nasby.net
512.569.9461 (cell)                         http://jim.nasby.net



Re: Disallow arrays with non-standard lower bounds

From
Peter Geoghegan
Date:
On Thu, Jan 9, 2014 at 2:30 PM, Jim Nasby <jim@nasby.net> wrote:
> ISTM that allowing users to pick arbitrary lower array bounds was a huge
> mistake. I've never seen anyone make use of it, can't think of any
> legitimate use cases for it, and hate the stupendous amount of extra code
> needed to deal with it.

I agree with this, but I think it's too late. I don't think the answer
is any type of parameter.


-- 
Peter Geoghegan



Re: Disallow arrays with non-standard lower bounds

From
David Fetter
Date:
On Thu, Jan 09, 2014 at 04:30:25PM -0600, Jim Nasby wrote:
> ISTM that allowing users to pick arbitrary lower array bounds was a
> huge mistake. I've never seen anyone make use of it, can't think of
> any legitimate use cases for it, and hate the stupendous amount of
> extra code needed to deal with it.
> 
> Obviously we can't just drop support, but what about an initdb (or
> hell, even configure) option to disallow arrays with a lower bound
> <> 1? Unfortunately we can't do this with a GUC since you can store
> arrays in a table.

We have dropped support, as you put it, for bigger and harder-hitting
mistakes than this.  Anybody whose code has this kind of silliness in
it will be in other kinds of trouble, too.

Cheers,
David (who is among that tiny minority who believe that arrays should
be indexed from 0.5 as a compromise ;)
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate



Re: Disallow arrays with non-standard lower bounds

From
Gavin Flower
Date:
On 10/01/14 12:41, David Fetter wrote:
[..]
> David (who is among that tiny minority who believe that arrays should 
> be indexed from 0.5 as a compromise ;) 
Clearly we should use 1/e as the starting index, where 'e' is Euler's 
constant 2.718...  :-)
(Much more mathematically profound!)


Cheers,
Gavin



Re: Disallow arrays with non-standard lower bounds

From
Peter Geoghegan
Date:
On Thu, Jan 9, 2014 at 3:41 PM, David Fetter <david@fetter.org> wrote:
> We have dropped support, as you put it, for bigger and harder-hitting
> mistakes than this.  Anybody whose code has this kind of silliness in
> it will be in other kinds of trouble, too.

While the decision to make it possible to set the lower bound index
value arbitrarily was made before I was active in the project, I
imagine it went something like this:

Person 1: We should make our arrays similar to those found in a
certain proprietary system's standard procedural language - with one
as a lower bound.

Person 2: I don't like that, it should always be zero.

Person 1: We can all be winners.

-- 
Peter Geoghegan



Re: Disallow arrays with non-standard lower bounds

From
Gavin Flower
Date:
On 10/01/14 12:55, Peter Geoghegan wrote:
> On Thu, Jan 9, 2014 at 3:41 PM, David Fetter <david@fetter.org> wrote:
>> We have dropped support, as you put it, for bigger and harder-hitting
>> mistakes than this.  Anybody whose code has this kind of silliness in
>> it will be in other kinds of trouble, too.
> While the decision to make it possible to set the lower bound index
> value arbitrarily was made before I was active in the project, I
> imagine it went something like this:
>
> Person 1: We should make our arrays similar to those found in a
> certain proprietary system's standard procedural language - with one
> as a lower bound.
>
> Person 2: I don't like that, it should always be zero.
>
> Person 1: We can all be winners.
>
Starting arrays at zero makes the most sense, as then you can calculate 
the displacement simply as (index) * (size of entry), and not have 
subtract one from the index first.  This would be my preference.

Both C & C++ use zero as the default, so Java naturally followed their 
example.

But I wouldn't push it, as people who mainly know SQL are more used to 
starting arrays at 1, and it is not worth the effort to change it as 
this stage.  At least we should be consistent how we treat arrays in  SQL.


Cheers,
Gavin



Re: Disallow arrays with non-standard lower bounds

From
Tom Lane
Date:
Jim Nasby <jim@nasby.net> writes:
> ISTM that allowing users to pick arbitrary lower array bounds was a huge mistake. I've never seen anyone make use of
it,can't think of any legitimate use cases for it, and hate the stupendous amount of extra code needed to deal with
it.

You lack imagination, sir.
        regards, tom lane



Re: Disallow arrays with non-standard lower bounds

From
Jim Nasby
Date:
On 1/9/14, 10:58 PM, Tom Lane wrote:
> Jim Nasby <jim@nasby.net> writes:
>> ISTM that allowing users to pick arbitrary lower array bounds was a huge mistake. I've never seen anyone make use of
it,can't think of any legitimate use cases for it, and hate the stupendous amount of extra code needed to deal with
it.
>
> You lack imagination, sir.

Considering what you'd normally want to do in SQL, the only example I can think of is to not have the argument over 0
vs1 based.
 

Actually, I was thinking there might be some computational problems where changing lower bound would be nice, but then
again,what other languages actually support this?
 
-- 
Jim C. Nasby, Data Architect                       jim@nasby.net
512.569.9461 (cell)                         http://jim.nasby.net



Re: Disallow arrays with non-standard lower bounds

From
Andrew Dunstan
Date:
On 01/10/2014 04:26 PM, Jim Nasby wrote:
> On 1/9/14, 10:58 PM, Tom Lane wrote:
>> Jim Nasby <jim@nasby.net> writes:
>>> ISTM that allowing users to pick arbitrary lower array bounds was a 
>>> huge mistake. I've never seen anyone make use of it, can't think of 
>>> any legitimate use cases for it, and hate the stupendous amount of 
>>> extra code needed to deal with it.
>>
>> You lack imagination, sir.
>
> Considering what you'd normally want to do in SQL, the only example I 
> can think of is to not have the argument over 0 vs 1 based.
>
> Actually, I was thinking there might be some computational problems 
> where changing lower bound would be nice, but then again, what other 
> languages actually support this?

Ada, for one. In fact, in Ada the index doesn't need to be an integer, 
just an enumerable type (e.g. an enum). You can iterate over 
one-dimensional arrays by saying:
    FOR i IN [REVERSE] my_array'range LOOP ...

cheers

andrew (who sadly hasn't used Ada in anger for about 20 years).





Re: Disallow arrays with non-standard lower bounds

From
David Fetter
Date:
On Fri, Jan 10, 2014 at 03:26:04PM -0600, Jim Nasby wrote:
> On 1/9/14, 10:58 PM, Tom Lane wrote:
> >Jim Nasby <jim@nasby.net> writes:
> >>ISTM that allowing users to pick arbitrary lower array bounds was
> >>a huge mistake. I've never seen anyone make use of it, can't think
> >>of any legitimate use cases for it, and hate the stupendous amount
> >>of extra code needed to deal with it.
> >
> >You lack imagination, sir.
> 
> Considering what you'd normally want to do in SQL, the only example
> I can think of is to not have the argument over 0 vs 1 based.
> 
> Actually, I was thinking there might be some computational problems
> where changing lower bound would be nice, but then again, what other
> languages actually support this?

Well, there's Perl, but that's not an argument *for* doing this, and I
say that as a long-time Perl user (who has never seen this feature
used in code worth not scrapping, by the way).

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate



Re: Disallow arrays with non-standard lower bounds

From
Kevin Grittner
Date:
Gavin Flower <GavinFlower@archidevsys.co.nz> wrote:

> Starting arrays at zero makes the most sense, as then you can
> calculate the displacement simply as (index) * (size of entry),
> and not have subtract one from the index first.  This would be my
> preference.

The SQL standard explicitly specifies that arrays positions range
from 1 to the cardinality of the array, with individual elements
referenced by position.  When implementing a language for which
there is an international standard, my preference is to conform to
the standard.

I don't have a problem with extensions to the language, and a
variable low bound is workable as an extension as long as the
standard ways to create an array default to a low bound of 1.

A bigger problem with our array implementation is that is is really
a multidimensional matrix, rather than an array which can contain
nested arrays.  That is both non-standard and limiting.

That said, I think it would be nice to have better support for
arrays defined with a single dimension and a low bound of 1, as the
standard requires.  Functions which throw an error when passed a
non-conforming parameter and can provide a simplified API as a
result would probably get used by me.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: Disallow arrays with non-standard lower bounds

From
Jeff Janes
Date:
On Fri, Jan 10, 2014 at 1:26 PM, Jim Nasby <jim@nasby.net> wrote:
On 1/9/14, 10:58 PM, Tom Lane wrote:
Jim Nasby <jim@nasby.net> writes:
ISTM that allowing users to pick arbitrary lower array bounds was a huge mistake. I've never seen anyone make use of it, can't think of any legitimate use cases for it, and hate the stupendous amount of extra code needed to deal with it.

You lack imagination, sir.

Considering what you'd normally want to do in SQL, the only example I can think of is to not have the argument over 0 vs 1 based.

Actually, I was thinking there might be some computational problems where changing lower bound would be nice, but then again, what other languages actually support this?

Perl does, though they regret it bitterly.

Cheers,

Jeff

Re: Disallow arrays with non-standard lower bounds

From
Merlin Moncure
Date:
On Fri, Jan 10, 2014 at 4:10 PM, Jeff Janes <jeff.janes@gmail.com> wrote:
> On Fri, Jan 10, 2014 at 1:26 PM, Jim Nasby <jim@nasby.net> wrote:
>>
>> On 1/9/14, 10:58 PM, Tom Lane wrote:
>>>
>>> Jim Nasby <jim@nasby.net> writes:
>>>>
>>>> ISTM that allowing users to pick arbitrary lower array bounds was a huge
>>>> mistake. I've never seen anyone make use of it, can't think of any
>>>> legitimate use cases for it, and hate the stupendous amount of extra code
>>>> needed to deal with it.
>>>
>>>
>>> You lack imagination, sir.
>>
>>
>> Considering what you'd normally want to do in SQL, the only example I can
>> think of is to not have the argument over 0 vs 1 based.
>>
>> Actually, I was thinking there might be some computational problems where
>> changing lower bound would be nice, but then again, what other languages
>> actually support this?
>
> Perl does, though they regret it bitterly.

What does it matter?  Our arrays have had the capability for years and
years and "because it's cleaner" is simply not justification to break
people's applications.  Why are we even considering this?

merlin



Re: Disallow arrays with non-standard lower bounds

From
Jim Nasby
Date:
On 1/10/14, 4:14 PM, Merlin Moncure wrote:
> On Fri, Jan 10, 2014 at 4:10 PM, Jeff Janes <jeff.janes@gmail.com> wrote:
>> On Fri, Jan 10, 2014 at 1:26 PM, Jim Nasby <jim@nasby.net> wrote:
>>>
>>> On 1/9/14, 10:58 PM, Tom Lane wrote:
>>>>
>>>> Jim Nasby <jim@nasby.net> writes:
>>>>>
>>>>> ISTM that allowing users to pick arbitrary lower array bounds was a huge
>>>>> mistake. I've never seen anyone make use of it, can't think of any
>>>>> legitimate use cases for it, and hate the stupendous amount of extra code
>>>>> needed to deal with it.
>>>>
>>>>
>>>> You lack imagination, sir.
>>>
>>>
>>> Considering what you'd normally want to do in SQL, the only example I can
>>> think of is to not have the argument over 0 vs 1 based.
>>>
>>> Actually, I was thinking there might be some computational problems where
>>> changing lower bound would be nice, but then again, what other languages
>>> actually support this?
>>
>> Perl does, though they regret it bitterly.
>
> What does it matter?  Our arrays have had the capability for years and
> years and "because it's cleaner" is simply not justification to break
> people's applications.  Why are we even considering this?

Because it's a foot-gun. So far no one has given a legitimate use case for it and supporting it *greatly* complicates
iteratingover arrays.
 

Also, just to be clear, I'd be fine with offering a better alternative and leaving existing arrays alone. I don't see
anyeasy way to do that, but maybe someone's got a good idea on that.
 
-- 
Jim C. Nasby, Data Architect                       jim@nasby.net
512.569.9461 (cell)                         http://jim.nasby.net



Re: Disallow arrays with non-standard lower bounds

From
Craig Ringer
Date:
On 01/10/2014 07:41 AM, David Fetter wrote:
> On Thu, Jan 09, 2014 at 04:30:25PM -0600, Jim Nasby wrote:
>> ISTM that allowing users to pick arbitrary lower array bounds was a
>> huge mistake. I've never seen anyone make use of it, can't think of
>> any legitimate use cases for it, and hate the stupendous amount of
>> extra code needed to deal with it.
>>
>> Obviously we can't just drop support, but what about an initdb (or
>> hell, even configure) option to disallow arrays with a lower bound
>> <> 1? Unfortunately we can't do this with a GUC since you can store
>> arrays in a table.
> 
> We have dropped support, as you put it, for bigger and harder-hitting
> mistakes than this.

Implicit casts to text, anybody?

Everybody used them. The project dropped them anyway, and did so pretty
much unconditionally at that.

This is a vastly less heavily used feature. I'd be really, really glad
to see it disabled by default at initdb time, with a warning saying the
initdb option to enable it will go away in a few versions.

-- Craig Ringer                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services



Re: Disallow arrays with non-standard lower bounds

From
Merlin Moncure
Date:
On Sun, Jan 12, 2014 at 4:38 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
> Implicit casts to text, anybody?

This backward compatibility break orphaned the company I work for on
8.1 until last year and very nearly caused postgres to be summarily
extirpated (only rescued at the last minute by my arrival). It cost
hundreds of thousands of dollars to qualify a sprawling java code base
so that it could be moved back into a supported version.  Breaking
compatibility sucks -- it hurts your users and costs people money.
Hacking type casts may not have been a mistake, but the arbitrary
introduction of the breakage certainly was.

This project has no deprecation policy, and I'd argue we'd need one
before considering breaking changes.  For example, maybe we could pull
out an occasional release for longer term support to help users that
caught out.   But really, the better way to go IMNSHO is to take a
hard line on compatibility issues pretty much always -- consider the
case of libc and win32 api.  There are certain limited exceptions to
this rule -- for example security problems or gross violations of the
standard (bringing row-wise comparison to spec comes to mind as an
example of that).

merlin



Re: Disallow arrays with non-standard lower bounds

From
Jim Nasby
Date:
On 1/13/14, 10:40 AM, Merlin Moncure wrote:
> On Sun, Jan 12, 2014 at 4:38 AM, Craig Ringer<craig@2ndquadrant.com>  wrote:
>> >Implicit casts to text, anybody?
> This backward compatibility break orphaned the company I work for on
> 8.1 until last year and very nearly caused postgres to be summarily
> extirpated (only rescued at the last minute by my arrival). It cost
> hundreds of thousands of dollars to qualify a sprawling java code base
> so that it could be moved back into a supported version.  Breaking
> compatibility sucks -- it hurts your users and costs people money.
> Hacking type casts may not have been a mistake, but the arbitrary
> introduction of the breakage certainly was.

IMHO, I see no reason we need to plan on removing support. Presumably it's not that much burden on our codebase; it's
onlya PITA for users writing correct code. (It'd be very interesting to see how much user code would blow up if
presentedwith anything other than 1 as the lower bound...)
 

I'd be perfectly happy with an initdb option to allow for lower bound support if you wanted it and disable it by
default.People the legitimately want/need <> 1 lower bounds can set that up, but our general user population will end
upprotected from a class of heisenbugs.
 
-- 
Jim C. Nasby, Data Architect                       jim@nasby.net
512.569.9461 (cell)                         http://jim.nasby.net



Re: Disallow arrays with non-standard lower bounds

From
David Fetter
Date:
On Mon, Jan 13, 2014 at 10:40:57AM -0600, Merlin Moncure wrote:
> On Sun, Jan 12, 2014 at 4:38 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
> > Implicit casts to text, anybody?
> 
> This backward compatibility break orphaned the company I work for on
> 8.1 until last year and very nearly caused postgres to be summarily
> extirpated (only rescued at the last minute by my arrival). It cost
> hundreds of thousands of dollars to qualify a sprawling java code base
> so that it could be moved back into a supported version.  Breaking
> compatibility sucks -- it hurts your users and costs people money.
> Hacking type casts may not have been a mistake, but the arbitrary
> introduction of the breakage certainly was.

With utmost respect, it was not.  Databases are no good if there are
fixable things in them that cause them to produce incorrect results at
random, as auto-casting to text did.

> This project has no deprecation policy,

I believe it actually does, although it's not a formal, written
policy.  Would you like to help draft one up?

> and I'd argue we'd need one
> before considering breaking changes.  For example, maybe we could pull
> out an occasional release for longer term support to help users that
> caught out.   But really, the better way to go IMNSHO is to take a
> hard line on compatibility issues pretty much always -- consider the
> case of libc and win32 api.

Could you please help remind us what that was?

> There are certain limited exceptions to this rule -- for example
> security problems

Probably not.

> or gross violations of the standard

We're definitely there on lower bounds of arrays.  The standard, for a
wonder, is clear and unambiguous about them.  Whether we should go
there on the rest of our array implementation is a question for
another thread.

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate



Re: Disallow arrays with non-standard lower bounds

From
Robert Haas
Date:
On Mon, Jan 13, 2014 at 4:45 PM, David Fetter <david@fetter.org> wrote:
> On Mon, Jan 13, 2014 at 10:40:57AM -0600, Merlin Moncure wrote:
>> On Sun, Jan 12, 2014 at 4:38 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
>> > Implicit casts to text, anybody?
>>
>> This backward compatibility break orphaned the company I work for on
>> 8.1 until last year and very nearly caused postgres to be summarily
>> extirpated (only rescued at the last minute by my arrival). It cost
>> hundreds of thousands of dollars to qualify a sprawling java code base
>> so that it could be moved back into a supported version.  Breaking
>> compatibility sucks -- it hurts your users and costs people money.
>> Hacking type casts may not have been a mistake, but the arbitrary
>> introduction of the breakage certainly was.
>
> With utmost respect, it was not.  Databases are no good if there are
> fixable things in them that cause them to produce incorrect results at
> random, as auto-casting to text did.

With a precisely equal level of respect, that's a load of bunk.  As
has been discussed here many times in the past, those changes broke
many applications that were just fine.  Mine included, EnterpriseDB's
included.  It was designed to maximize rather than minimize breakage
in ways that were completely unnecessary, and every time anyone dares
to argue that the change was less than perfect in every respect, the
reponses evince a level of dubiousness normally reserved for the
parole hearings of convicted murderers.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: Disallow arrays with non-standard lower bounds

From
Merlin Moncure
Date:
On Mon, Jan 13, 2014 at 3:45 PM, David Fetter <david@fetter.org> wrote:
> On Mon, Jan 13, 2014 at 10:40:57AM -0600, Merlin Moncure wrote:
>> This project has no deprecation policy,
>
> I believe it actually does, although it's not a formal, written
> policy.  Would you like to help draft one up?

Lack of 'formal, written, policy' is equivalent to 'no policy'.
Regardless, the way things we done in the 7.x/8.x series may no longer
apply today; the project has grown up and we need to be more serious
about things, at least, IMNSHO.

>> and I'd argue we'd need one
>> before considering breaking changes.  For example, maybe we could pull
>> out an occasional release for longer term support to help users that
>> caught out.   But really, the better way to go IMNSHO is to take a
>> hard line on compatibility issues pretty much always -- consider the
>> case of libc and win32 api.
>
> Could you please help remind us what that was?

Let's take gets() for example.  C11 finally ditched it 12 years (!)
after it was formally deprecated in C99 and informally deprecate in
endless man pages ("don't use this!") for decades before that.  And
even then most compilers, at least the decent ones, should allow to
request previous standards for some time beyond that.  The win32 API
is also remarkably stable; ancient code written for it beyond the dim
horizon of time will still compile and execute today.  These are
probably strong contenders for most popular APIs ever made -- see the
connection?  Now, comparing C APIs to an SQL implementation for
deprecation purposes isn't quite applies to apples, but I'll stand by
the analogy.

>> or gross violations of the standard
>
> We're definitely there on lower bounds of arrays.  The standard, for a
> wonder, is clear and unambiguous about them.  Whether we should go
> there on the rest of our array implementation is a question for
> another thread.

The SQL standard requests that standard syntax gives standard
behavior.  Alternate bounds is non-standard syntax giving non-standard
behavior and is thus excepted.  Naturally, non-standard syntax is
dangerous because the standard may later implement it in which case
you then have a real problem (that may be the case here: I don't
know).  Our array implementation is a real mess on multiple levels but
at least it's an internally consistent mess.  Maybe it really should
be 'fixed', but not before the super un-fun discussion of how to ease
the path for our hapless users happens first.

merlin



Re: Disallow arrays with non-standard lower bounds

From
Craig Ringer
Date:
On 01/14/2014 12:40 AM, Merlin Moncure wrote:
> On Sun, Jan 12, 2014 at 4:38 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
>> Implicit casts to text, anybody?
> 
> This backward compatibility break orphaned the company I work for on
> 8.1 until last year and very nearly caused postgres to be summarily
> extirpated (only rescued at the last minute by my arrival).

You're far from the only one, too. Until last year I was still seeing
people saying they "can't" upgrade because of this. OTOH, that was a
sudden and drastic change, with no BC switch like the removal of
implicit joins had, that affected wide swaths of code. Lets not do that
again.

Removal of lower bounds for arrays is unlikely to even get noticed by
the vast majority of users, and can be done progressively with BC features.

The real issue IMO is how to get those few to stop using it so it can be
truly removed. Past experience has shown that people just turn the
compatibility flag on and forget they're using the deprecated feature.
If there are warnings they'll silence them in their application and
still forget they're using the deprecated feature. If there are log
messages, they'll probably turn logging detail down to hide them and
STILL forget they're using the deprecated feature.

Then whine about it three years later when it gets removed.

So I guess the question is: Is it worth all that hassle to remove a
misfeature you have to go out of your way to use? Is support for non-1
lower bounds stopping us from doing something useful and important? Or
is it just an irritation that it exists?

-- Craig Ringer                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services



Re: Disallow arrays with non-standard lower bounds

From
Hannu Krosing
Date:
On 01/14/2014 12:33 AM, Craig Ringer wrote:
> On 01/14/2014 12:40 AM, Merlin Moncure wrote:
>> On Sun, Jan 12, 2014 at 4:38 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
>>> Implicit casts to text, anybody?
>> This backward compatibility break orphaned the company I work for on
>> 8.1 until last year and very nearly caused postgres to be summarily
>> extirpated (only rescued at the last minute by my arrival).
> You're far from the only one, too. Until last year I was still seeing
> people saying they "can't" upgrade because of this. OTOH, that was a
> sudden and drastic change, with no BC switch like the removal of
> implicit joins had, that affected wide swaths of code. Lets not do that
> again.
>
> Removal of lower bounds for arrays is unlikely to even get noticed by
> the vast majority of users, and can be done progressively with BC features.
>
> The real issue IMO is how to get those few to stop using it so it can be
> truly removed. Past experience has shown that people just turn the
> compatibility flag on and forget they're using the deprecated feature.
> If there are warnings they'll silence them in their application and
> still forget they're using the deprecated feature. If there are log
> messages, they'll probably turn logging detail down to hide them and
> STILL forget they're using the deprecated feature.
>
> Then whine about it three years later when it gets removed.
>
> So I guess the question is: Is it worth all that hassle to remove a
> misfeature you have to go out of your way to use? Is support for non-1
> lower bounds stopping us from doing something useful and important? Or
> is it just an irritation that it exists?
>
Let's just add user defined operator for '[]' (weirdly-positioned but
2 argument, almost infix :) ) and add that to JSON arrays to get
0-based ones into poastgresq  ;)

Cheers

-- 
Hannu Krosing
PostgreSQL Consultant
Performance, Scalability and High Availability
2ndQuadrant Nordic OÜ




Re: Disallow arrays with non-standard lower bounds

From
Jim Nasby
Date:
On 1/13/14, 5:33 PM, Craig Ringer wrote:
> So I guess the question is: Is it worth all that hassle to remove a
> misfeature you have to go out of your way to use? Is support for non-1
> lower bounds stopping us from doing something useful and important? Or
> is it just an irritation that it exists?

It's not an irritation to -hackers, but it is an irritation for anyone that cares about data quality, because you're
forcedto code all of your stuff to always look at array_lower().
 

Actually, now that I think about it, if you want to be really safe you would actually force your code to use a
differentlower bound so you're more likely to discover code that's broken.
 

So it really is a big pain for users that know what's going on. And it will become a big pain for users that don't know
ifthey ever accidentally end up with non-1 arrays. :)
 
-- 
Jim C. Nasby, Data Architect                       jim@nasby.net
512.569.9461 (cell)                         http://jim.nasby.net



Re: Disallow arrays with non-standard lower bounds

From
Tom Lane
Date:
Craig Ringer <craig@2ndquadrant.com> writes:
> So I guess the question is: Is it worth all that hassle to remove a
> misfeature you have to go out of your way to use? Is support for non-1
> lower bounds stopping us from doing something useful and important? Or
> is it just an irritation that it exists?

I think the argument really is that some people don't want to make their
application code work with such cases (which is fine) so they'd like an
inside-the-database guarantee that the app code won't ever see such cases.
Which is less fine, ISTM: if you fear some part of your app might be
generating such arrays, then you don't have such little use for the
feature after all, eh?

This is being camouflaged in a whole lot of utter BS about how nobody
could possibly be using the feature, nobody *should* want it, it's outside
the standard, etc etc.  If we ripped out every feature being used by less
than 10% of the user base, we'd have a much smaller and more maintainable
system, for sure ... but we'd probably piss off upwards of 90% of the
user base by doing that.  Your useless frammish is the next man's killer
feature.
        regards, tom lane



Re: Disallow arrays with non-standard lower bounds

From
Kevin Grittner
Date:
Tom Lane <tgl@sss.pgh.pa.us> wrote:

> I think the argument really is that some people don't want to
> make their application code work with such cases (which is fine)
> so they'd like an inside-the-database guarantee that the app code
> won't ever see such cases.  Which is less fine, ISTM: if you fear
> some part of your app might be generating such arrays, then you
> don't have such little use for the feature after all, eh?

While I don't see the argument for dropping support for lower
bounds other than 1 as long as standard syntax for creating arrays
yields arrays staring at 1, this argument holds no water at all.  I
have done too many conversions from databases which did not enforce
data integrity rules not to have a lot of sympathy for people
wanting to have the ability to easily constrain data to whatever
their business rules say it should look like.  The fact that some
day some new programmer might not be aware of all business rules,
or might choose to try to ignore them is the reason you add
constraints to columns and domains.

I think providing functions that make it easy to enforce this rule
make a lot of sense, as it would make a lot of application code
simpler, easier to read, and less fragile.  I support overloading
functions with simpler parameter lists for standard-conforming
arrays; I don't support dropping support for whoever might be using
non-conforming arrays.  A veriable lower bound is an extension to
the standard, and extensions are fine.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: Disallow arrays with non-standard lower bounds

From
Jim Nasby
Date:
On 1/13/14, 7:10 PM, Kevin Grittner wrote:
> Tom Lane<tgl@sss.pgh.pa.us>  wrote:
>
>> >I think the argument really is that some people don't want to
>> >make their application code work with such cases (which is fine)
>> >so they'd like an inside-the-database guarantee that the app code
>> >won't ever see such cases.  Which is less fine, ISTM: if you fear
>> >some part of your app might be generating such arrays, then you
>> >don't have such little use for the feature after all, eh?
> While I don't see the argument for dropping support for lower
> bounds other than 1 as long as standard syntax for creating arrays
> yields arrays staring at 1, this argument holds no water at all.  I
> have done too many conversions from databases which did not enforce
> data integrity rules not to have a lot of sympathy for people
> wanting to have the ability to easily constrain data to whatever
> their business rules say it should look like.  The fact that some
> day some new programmer might not be aware of all business rules,
> or might choose to try to ignore them is the reason you add
> constraints to columns and domains.
>
> I think providing functions that make it easy to enforce this rule
> make a lot of sense, as it would make a lot of application code
> simpler, easier to read, and less fragile.  I support overloading
> functions with simpler parameter lists for standard-conforming
> arrays; I don't support dropping support for whoever might be using
> non-conforming arrays.  A veriable lower bound is an extension to
> the standard, and extensions are fine.

To be clear: I am completely fine with continuing to support custom lower bounds (to quote my original email:
"Obviouslywe can't just drop support, but what about an initdb (or hell, even configure) option to disallow arrays with
alower bound <> 1?").
 

It might also be possible to protect users on the access side instead: provide alternate functions that protect you if
youpass them an array that does have a custom lower bound. But I'm not so sure that'll work out, since one of the
biggestoffenders here is:
 

FOR i IN 1..array_upper(some_array) LOOP.

Yes, today you can use FOREACH, but what about all your legacy code that's ass-u-ming that the lower bound is 1?
-- 
Jim C. Nasby, Data Architect                       jim@nasby.net
512.569.9461 (cell)                         http://jim.nasby.net



Re: Disallow arrays with non-standard lower bounds

From
Florian Pflug
Date:
On Jan14, 2014, at 00:33 , Craig Ringer <craig@2ndquadrant.com> wrote:
> So I guess the question is: Is it worth all that hassle to remove a
> misfeature you have to go out of your way to use? Is support for non-1
> lower bounds stopping us from doing something useful and important? Or
> is it just an irritation that it exists?

I don't think it's worh it - as you say, the actual risk of bugs is low,
because you have to go out of your way to end up with a lower bound other
than one.

Also, at least from my POV, the fact that we use one type do represent
arrays with an arbitrary number of dimensions is actually worse than
the lower-bound problem. So *if* we ever remove support for arbitrary
lower bounds, we should also add distinct types for different dimensions.
That'd probably required some extension of the type system though...

best regards,
Florian Pflug




Re: Disallow arrays with non-standard lower bounds

From
Florian Pflug
Date:
On Jan14, 2014, at 02:10 , Kevin Grittner <kgrittn@ymail.com> wrote:
> The fact that some
> day some new programmer might not be aware of all business rules,
> or might choose to try to ignore them is the reason you add
> constraints to columns and domains.

Well, for columns and domains that seems easy. We could have
 array_has_shape(value anyarray, variadic bounds int4range[])

and
 array_has_shape(value anyarray, variadic bounds int4[])

The first returns true if the value has length(bounds) dimensions
and each dimension's bounds match the corresponding range's bound,
where NULL means "arbitrary". The second one requires all lower
bounds to be 1, and checks the upper bounds against the bounds array.

Checking that an array is one-dimensional with lower bound 1 is then
accomplished by
 array_has_shape(myarray, int4range(1, NULL))

or simply
 array_has_shape(myarray, NULL);

best regards,
Florian Pflug