Thread: BUG #2694: Memory allocation error when selecting array of empty arrays
The following bug has been logged online: Bug reference: 2694 Logged by: Vitali Stupin Email address: Vitali.Stupin@ria.ee PostgreSQL version: 8.1.4 Operating system: sparc-sun-solaris2.10 Description: Memory allocation error when selecting array of empty arrays Details: The error "invalid memory alloc request size 4294967293" apears when selecting array of empty arrays: select ARRAY['{}'::text[],'{}'::text[]]; This bug can be also reproduced on: PostgreSQL 8.1.3 on sparc-sun-solaris2.10; PostgreSQL 8.1.3 on i686-pc-linux-gnu. The only difference is that linux version of postgre is trying to allocate 4294967294 (or 2^32 - 2) bytes. ...Sorry for posting the same bug report for the second time, but the previous post sent almost a month ago (bug reference 2641) still had not appeared in bug-list.
"Vitali Stupin" <Vitali.Stupin@ria.ee> writes: > The error "invalid memory alloc request size 4294967293" apears when > selecting array of empty arrays: > select ARRAY['{}'::text[],'{}'::text[]]; I can get a core dump off it too, sometimes. The problem is in ExecEvalArray, which computes the dimension of the result as [1:2] even though there are no elements to put in it. Joe, what do you think about this? Offhand I think that the only workable definition is that this case yields another zero-dimensional array, but maybe there is another choice? We should probably check all the other array operations to see if they have comparable problems. regards, tom lane
Tom Lane wrote: > "Vitali Stupin" <Vitali.Stupin@ria.ee> writes: > >>The error "invalid memory alloc request size 4294967293" apears when >>selecting array of empty arrays: >>select ARRAY['{}'::text[],'{}'::text[]]; > > I can get a core dump off it too, sometimes. The problem is in > ExecEvalArray, which computes the dimension of the result as [1:2] > even though there are no elements to put in it. > > Joe, what do you think about this? Offhand I think that the only > workable definition is that this case yields another zero-dimensional > array, but maybe there is another choice? Sorry for the slow response -- I'm at the airport just heading home from a marathon 30 day business trip. I think producing another zero-dimensional result is the only way that makes sense unless/until we change multidimensional arrays to really be arrays of array-datatype elements. Right now they're two different things. > We should probably check all the other array operations to see if they > have comparable problems. Yes -- I'll see if I can find the time over the next couple weeks while home. Joe
Joe Conway <mail@joeconway.com> writes: > Sorry for the slow response -- I'm at the airport just heading home from > a marathon 30 day business trip. Yow. Hope you get some time off... > Tom Lane wrote: >> "Vitali Stupin" <Vitali.Stupin@ria.ee> writes: >>> The error "invalid memory alloc request size 4294967293" apears when >>> selecting array of empty arrays: >>> select ARRAY['{}'::text[],'{}'::text[]]; >> >> Joe, what do you think about this? Offhand I think that the only >> workable definition is that this case yields another zero-dimensional >> array, but maybe there is another choice? > I think producing another zero-dimensional result is the only way that > makes sense unless/until we change multidimensional arrays to really be > arrays of array-datatype elements. Right now they're two different things. On looking at the code, I notice that this somewhat-related case works: regression=# select array[null::text[], null::text[]]; array ------- {} (1 row) The reason is that null inputs are just ignored in ExecEvalArray. So one pretty simple patch would be to ignore zero-dimensional inputs too. This would have implications for mixed inputs though: instead of regression=# select array['{}'::text[], '{a,b,c}'::text[]]; ERROR: multidimensional arrays must have array expressions with matching dimensions you'd get behavior like regression=# select array[null::text[], '{a,b,c}'::text[]]; array ----------- {{a,b,c}} (1 row) Which of these seems more sane? regards, tom lane
Tom Lane wrote: > Joe Conway <mail@joeconway.com> writes: >>Sorry for the slow response -- I'm at the airport just heading home from >>a marathon 30 day business trip. > > Yow. Hope you get some time off... Yeah, I just took a week. Next week I'm back to work and the week after that I'm back to Germany for a few... > On looking at the code, I notice that this somewhat-related case works: > > regression=# select array[null::text[], null::text[]]; > array > ------- > {} > (1 row) > > The reason is that null inputs are just ignored in ExecEvalArray. So > one pretty simple patch would be to ignore zero-dimensional inputs too. > This would have implications for mixed inputs though: instead of > > regression=# select array['{}'::text[], '{a,b,c}'::text[]]; > ERROR: multidimensional arrays must have array expressions with matching dimensions > > you'd get behavior like > > regression=# select array[null::text[], '{a,b,c}'::text[]]; > array > ----------- > {{a,b,c}} > (1 row) > > Which of these seems more sane? I'm not sure I love either. I would think both NULL and empty array expressions should be disallowed in this scenario, i.e.: regression=# select array['{}'::text[], '{a,b,c}'::text[]]; ERROR: multidimensional arrays must have array expressions with matching dimensions regression=# select array[NULL::text[], '{a,b,c}'::text[]]; ERROR: multidimensional arrays must have array expressions with matching dimensions In both cases you are trying to construct a multidimensional array with inconsistent dimensions. On the other hand, building an N-dimension array from entirely empty array expressions should just produce an empty array, while using all NULL expressions should produce an N-dim array full of NULLs. But as I've opined before, all of this seems to me to be much cleaner if arrays were always one-dimensional, and array elements could also be nested arrays (per SQL 2003). If we said that the cardinality of the nested array is an integral part of the datatype, then I think you would have: regression=# select array['{}'::text[], '{a,b,c}'::text[]]; ERROR: nested arrays must have array expressions with matching dimensions regression=# select array[NULL::text[], '{a,b,c}'::text[]]; array ----------- {NULL, {a,b,c}} (1 row) So maybe this is the behavior we should shoot for now? Joe
Joe Conway <mail@joeconway.com> writes: > ... But as I've opined before, all of this seems to me to be much cleaner if > arrays were always one-dimensional, and array elements could also be > nested arrays (per SQL 2003). If we said that the cardinality of the > nested array is an integral part of the datatype, then I think you would > have: > regression=# select array['{}'::text[], '{a,b,c}'::text[]]; > ERROR: nested arrays must have array expressions with matching > dimensions > regression=# select array[NULL::text[], '{a,b,c}'::text[]]; > array > ----------- > {NULL, {a,b,c}} > (1 row) > So maybe this is the behavior we should shoot for now? Perhaps that's something to think about for 8.3, but it's clearly infeasible for 8.2, to say nothing of the back branches. What I've just committed does this: * if all inputs are empty arrays, return an empty array (instead of failing) * if mix of empty and nonempty arrays, raise error (same as before) The case of NULL subarrays is a bit weird. In the back branches the ARRAY[] construct was strict and would return a NULL array if any input was NULL; I think it would be a bad idea to change that. The HEAD code was simply skipping over NULL subarrays, which seemed like a good idea at the time but on reflection is not so hot. What I've done is to make it treat NULL subarrays as if they were empty subarrays. regards, tom lane