Thread: Disk block size issues.

A few things that I have noticed will be affected by allowing the
disk block size to be other than 8k. (4k, 8k, 16k or 32k)

1. Rules

The rule system currently stores plans as tuples in pg_rewrite.
Making the block size smaller will accordingly reduce the size of
the rules you can create.

But, the converse is also true...bigger blocks -> bigger rules.

Are the rules ever going to become large objects?  Is this something
to put on the TODO to investigate now that Peter has fixed them?


2. Attribute limits

Should the size limits of the varchar/char be driven by the chosen
block size?

Since the current max len is 4k, should I for now advise that the
block size not be made smaller than the current 8k?  Or could the
limit be dropped from 4096 to 4000 to allow 4k blocks?

Oracle has a limit of 2000 on their varchar since they allow blocks
of as little as 2k.

Seems there would be an inconsistency in there with telling the user
that the text/varchar/char limit is 4096 and then not letting them
store a value of that size because of the tuple/block size limit.

Perhaps mention this as a caveat also if using 4k blocks?  Are 4k
block something that someone would be beneficial or only 16k/32k?

On the flip-side of this, uping the max text size though will run
into the 8k packet size.

I've run thru the regression tests a few times with 4k blocks and
they seem to pass with the same differences.  Today I will try with
16k and 32k.  If those work, I'll submit the patch for perusal.

Comments welcome...

darrenk@insightdist.com

>
>
> A few things that I have noticed will be affected by allowing the
> disk block size to be other than 8k. (4k, 8k, 16k or 32k)
>
> 1. Rules
>
> The rule system currently stores plans as tuples in pg_rewrite.
> Making the block size smaller will accordingly reduce the size of
> the rules you can create.

I say make it match the given block size at compile time.

>
> But, the converse is also true...bigger blocks -> bigger rules.
>
> Are the rules ever going to become large objects?  Is this something
> to put on the TODO to investigate now that Peter has fixed them?
>
>
> 2. Attribute limits
>
> Should the size limits of the varchar/char be driven by the chosen
> block size?

Yes, they should be calculated based on the compile block size.

>
> Since the current max len is 4k, should I for now advise that the
> block size not be made smaller than the current 8k?  Or could the
> limit be dropped from 4096 to 4000 to allow 4k blocks?
>
> Oracle has a limit of 2000 on their varchar since they allow blocks
> of as little as 2k.
>
> Seems there would be an inconsistency in there with telling the user
> that the text/varchar/char limit is 4096 and then not letting them
> store a value of that size because of the tuple/block size limit.
>
> Perhaps mention this as a caveat also if using 4k blocks?  Are 4k
> block something that someone would be beneficial or only 16k/32k?

Just make the max size based on the block size.

>
> On the flip-side of this, uping the max text size though will run
> into the 8k packet size.

This is an interesting point.  While we can compute most of the changes
at compile time, we will have to communicate with clients that were
compiled with different max limits.

I recommend we increase the max client buffer size to what we believe is
the largest block size anyone would ever reasonably choose.  That way,
all can communicate.  I recommend you contact Peter Mount for JDBC,
Openlink for ODBC, and all the other client maintainers and let them
know the changes will be in 6.3 so they can be ready with new version
when 6.3 starts beta on February 1.

>
> I've run thru the regression tests a few times with 4k blocks and
> they seem to pass with the same differences.  Today I will try with
> 16k and 32k.  If those work, I'll submit the patch for perusal.

Great.

--
Bruce Momjian
maillist@candle.pha.pa.us

Darren King wrote:
>
> A few things that I have noticed will be affected by allowing the
> disk block size to be other than 8k. (4k, 8k, 16k or 32k)
>
> 1. Rules
>
> The rule system currently stores plans as tuples in pg_rewrite.
> Making the block size smaller will accordingly reduce the size of
> the rules you can create.
>
> But, the converse is also true...bigger blocks -> bigger rules.
>
> Are the rules ever going to become large objects?  Is this something
> to put on the TODO to investigate now that Peter has fixed them?

It's better to implement multi-representation feature for all verlena
types. We could use on-disk vl_len < 0 to flag that data of size ABS(vl_len)
are in large object specified in vl_data. It seems very easy to do.

This will also resolve item 2 below.

Vadim

>
> 2. Attribute limits
>
> Should the size limits of the varchar/char be driven by the chosen
> block size?
>
> Since the current max len is 4k, should I for now advise that the
> block size not be made smaller than the current 8k?  Or could the
> limit be dropped from 4096 to 4000 to allow 4k blocks?
>
> Oracle has a limit of 2000 on their varchar since they allow blocks
> of as little as 2k.
>
> Seems there would be an inconsistency in there with telling the user
> that the text/varchar/char limit is 4096 and then not letting them
> store a value of that size because of the tuple/block size limit.
>
> Perhaps mention this as a caveat also if using 4k blocks?  Are 4k
> block something that someone would be beneficial or only 16k/32k?
>
> On the flip-side of this, uping the max text size though will run
> into the 8k packet size.
>
> I've run thru the regression tests a few times with 4k blocks and
> they seem to pass with the same differences.  Today I will try with
> 16k and 32k.  If those work, I'll submit the patch for perusal.
>
> Comments welcome...
>
> darrenk@insightdist.com