Thread: \dt+ sizes don't include TOAST data

\dt+ sizes don't include TOAST data

From
Florian Weimer
Date:
The sizes displayed by \dt+ in version 8.4.2 do not take TOAST tables
into account, presumably because the pg_relation_size does not reflect
that, either.  I think this is a bit surprising.  From a user
perspective, these are part of the table storage (I understand that
the indices might be a different story, but TOAST table are a fairly
deep implementation detail and should perhaps be hidden here).

--
Florian Weimer                <fweimer@bfk.de>
BFK edv-consulting GmbH       http://www.bfk.de/
Kriegsstraße 100              tel: +49-721-96201-1
D-76133 Karlsruhe             fax: +49-721-96201-99

Re: \dt+ sizes don't include TOAST data

From
Greg Smith
Date:
Florian Weimer wrote:
> The sizes displayed by \dt+ in version 8.4.2 do not take TOAST tables
> into account, presumably because the pg_relation_size does not reflect
> that, either.  I think this is a bit surprising.  From a user
> perspective, these are part of the table storage (I understand that
> the indices might be a different story, but TOAST table are a fairly
> deep implementation detail and should perhaps be hidden here).
>

As of last week there's a new pg_table_size available that does what you
want here:
http://archives.postgresql.org/pgsql-committers/2010-01/msg00288.php

I don't believe \dt+ has been updated yet to use that though; that's
worth considering for a minute, not sure anybody thought about it yet.

--
Greg Smith    2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com  www.2ndQuadrant.com


Re: \dt+ sizes don't include TOAST data

From
Tom Lane
Date:
Greg Smith <greg@2ndquadrant.com> writes:
> Florian Weimer wrote:
>> The sizes displayed by \dt+ in version 8.4.2 do not take TOAST tables
>> into account, presumably because the pg_relation_size does not reflect
>> that, either.  I think this is a bit surprising.  From a user
>> perspective, these are part of the table storage (I understand that
>> the indices might be a different story, but TOAST table are a fairly
>> deep implementation detail and should perhaps be hidden here).

> As of last week there's a new pg_table_size available that does what you
> want here:
> http://archives.postgresql.org/pgsql-committers/2010-01/msg00288.php

> I don't believe \dt+ has been updated yet to use that though; that's
> worth considering for a minute, not sure anybody thought about it yet.

We could only use pg_table_size against a backend >= 9.0, which would
mean that the displayed results mean something different depending on
which backend version psql is being used with.  That's not necessarily
a deal-breaker, but it does seem a bit evil.

An alternative worth thinking about is to make it use
pg_total_relation_size instead of pg_relation_size.  That's available,
with similar semantics, in all versions that have pg_relation_size
either (ie, >= 8.1).  Also, this is arguably more nearly the right thing
since at the level of \dt+ I think people would expect indexes to get
folded in too.

            regards, tom lane

Re: \dt+ sizes don't include TOAST data

From
Alvaro Herrera
Date:
Tom Lane wrote:
> Greg Smith <greg@2ndquadrant.com> writes:
> > Florian Weimer wrote:
> >> The sizes displayed by \dt+ in version 8.4.2 do not take TOAST tables
> >> into account, presumably because the pg_relation_size does not reflect
> >> that, either.  I think this is a bit surprising.  From a user
> >> perspective, these are part of the table storage (I understand that
> >> the indices might be a different story, but TOAST table are a fairly
> >> deep implementation detail and should perhaps be hidden here).
>
> > As of last week there's a new pg_table_size available that does what you
> > want here:
> > http://archives.postgresql.org/pgsql-committers/2010-01/msg00288.php
>
> > I don't believe \dt+ has been updated yet to use that though; that's
> > worth considering for a minute, not sure anybody thought about it yet.
>
> We could only use pg_table_size against a backend >= 9.0, which would
> mean that the displayed results mean something different depending on
> which backend version psql is being used with.  That's not necessarily
> a deal-breaker, but it does seem a bit evil.

Perhaps we can emulate pg_table_size on earlier server versions, using a
query which provides the sum of table plus toast items.  It would be a
bit slower, but the normal case of using the same server version would
be fast.

--
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Re: \dt+ sizes don't include TOAST data

From
Bruce Momjian
Date:
Alvaro Herrera wrote:
> Tom Lane wrote:
> > Greg Smith <greg@2ndquadrant.com> writes:
> > > Florian Weimer wrote:
> > >> The sizes displayed by \dt+ in version 8.4.2 do not take TOAST tables
> > >> into account, presumably because the pg_relation_size does not reflect
> > >> that, either.  I think this is a bit surprising.  From a user
> > >> perspective, these are part of the table storage (I understand that
> > >> the indices might be a different story, but TOAST table are a fairly
> > >> deep implementation detail and should perhaps be hidden here).
> >
> > > As of last week there's a new pg_table_size available that does what you
> > > want here:
> > > http://archives.postgresql.org/pgsql-committers/2010-01/msg00288.php
> >
> > > I don't believe \dt+ has been updated yet to use that though; that's
> > > worth considering for a minute, not sure anybody thought about it yet.
> >
> > We could only use pg_table_size against a backend >= 9.0, which would
> > mean that the displayed results mean something different depending on
> > which backend version psql is being used with.  That's not necessarily
> > a deal-breaker, but it does seem a bit evil.
>
> Perhaps we can emulate pg_table_size on earlier server versions, using a
> query which provides the sum of table plus toast items.  It would be a
> bit slower, but the normal case of using the same server version would
> be fast.

Added to TODO:

    Consider showing TOAST and index sizes in \dt+

        * http://archives.postgresql.org/pgsql-general/2010-01/msg00912.php

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +