Thread: RE: [HACKERS] MAX Query length
Well, I'm starting on this, so hopefully in a couple of weeks the length limit of the query buffer will fade into insignificance. Is somebody actively working on removing the tuple-length dependence on the block size? At present, disk blocks are set to 8k. Is it as easy as just adjusting the constant to enlarge this? Testing queries larger than 16k with only an 8k tuple size could be challenging. MikeA >> > This entire chain of logic will fall to the ground anyway >> once we support >> > tuples larger than a disk block, which I believe is going to happen >> > before too much longer. So, rather than argue about what >> the multiplier >> > ought to be, I think it's more productive to just press on >> with making >> > the query buffers dynamically resizable... >> >> Yes, even if we choose to make some other limit (like Vadim >> suggested around 64K), a query operating on them could be >> much bigger. I already had some progress with a data type >> that uses a simple, byte oriented lz compression buffer as >> internal representation. >> >> >> Jan >> >> -- >> >> #============================================================ >> ==========# >> # It's easier to get forgiveness for being wrong than for >> being right. # >> # Let's break this rule - forgive me. >> # >> #========================================= wieck@debis.com >> (Jan Wieck) # >> >> >>
"Ansley, Michael" <Michael.Ansley@intec.co.za> writes:
> At present, disk blocks are set to 8k. Is it as easy as just adjusting the
> constant to enlarge this? Testing queries larger than 16k with only an 8k
> tuple size could be challenging.
As of 6.5, it's just a matter of adjusting BLCKSZ in include/config.h,
rebuilding, and re-initdb-ing. The workable sizes are 8k 16k and 32k;
bigger than 32k fails for reasons I don't recall exactly (offsets
stored in signed shorts somewhere, no doubt).
> Is somebody actively working on removing the tuple-length dependence on the
> block size?
There was considerable discussion about it a few weeks ago, but I didn't
hear anyone actually committing to do the work :-(. Maybe when you've
made some progress on the text-length issues, someone will get excited
about the tuple-length issue...
regards, tom lane
I wrote:
> "Ansley, Michael" <Michael.Ansley@intec.co.za> writes:
>> At present, disk blocks are set to 8k. Is it as easy as just adjusting the
>> constant to enlarge this? Testing queries larger than 16k with only an 8k
>> tuple size could be challenging.
> As of 6.5, it's just a matter of adjusting BLCKSZ in include/config.h,
> rebuilding, and re-initdb-ing.
A further thought on this: if you increase BLCKSZ then at least some of
the fixed-size text buffers will get bigger, so it's not clear that you
will be stressing things all that hard if you take that route. Might be
easier to leave BLCKSZ alone and test with queries that are long and
complicated but don't actually require a large tuple size. Some
examples:
1. SELECT a,a,a,a,... FROM table;
2. SELECT a FROM table WHERE x = 1 OR x = 2 OR x = 3 OR ...;
3. Hugely complex CREATE TABLE commands (lots of constraints and defaults and indexes, which don't enlarge the size
ofan actual tuple of the table).
regards, tom lane