Thread: performance using table partitions in Postgres 8.2.6
Hi. I am looking for information about using table partitions in Postgres,
in particular w.r.t. performance when querying a partitioned table.
I implemented table partitioning following the documentation, which
is quite good and easy to follow (Chapter 5.9). I am doing some
testing, so at this point my test dataset has 1M records; in the table,
each row will have an autoincrement integer index, two numerics,
one integer (DMID), and one smallint.
I created five master tables and created a different number of
partitions for each one (2, 4, 8, 16, 32). I am using a range partition
for the integer, DMID, which represents a file index. The values of
DMID range from 0 to 180360. I also create and index for DMID.
I don't understand the timing results that I am getting. I got these
times by averaging the results of querying the database from within
a loop in a Perl script:
no. of partitions constraint_exclusion off constraint_exclusion on
2 0.597 ms 0.427 ms
4 0.653 ms 0.414 ms
8 0.673 ms 0.654 ms
16 1.068 ms 1.014 ms
32 2.301 ms 1.537 ms
I expected that the query time would decrease as the number of
partitions increases, but that's not what I am seeing. I get better results
(0.29 ms) if I simply index DMID and don't use the partitions.
When I run "explain analyze"on a query, the results (with and without
constraint_exclusion set) indicate that fewer partitions are being scanned
when constraint_exclusion is set to on.
I am testing table partitioning in Postgres against table partitioning
using MySQL. The results for MySQL make sense: more partitions,
faster query times.
The underlying application is a file index. It is expected that groups
of files in selected ranges of DMID values will be accessed more
often, but this is not the key implementation issue.
This is basically a "write once, read often" database. We expect that
the database will grow to 50M records in a few years, and I thought
that using range partitions for the DMID value might decrease the
query time.
Should I be using many more partitions? Am I expecting too much in
terms of performance when using partitions? Do these results point to
some obvious implementation error?
Thank you for any help/suggestions you can give.
Janet Jacobsen
--
Janet Jacobsen
NERSC Analytics/HPCRD Visualization Group
Lawrence Berkeley National Laboratory
A Dilluns 07 Abril 2008, Janet Jacobsen va escriure: > no. of partitions constraint_exclusion off constraint_exclusion on > 2 0.597 ms 0.427 ms > 4 0.653 ms 0.414 ms > 8 0.673 ms 0.654 ms > 16 1.068 ms 1.014 ms > 32 2.301 ms 1.537 ms > > I expected that the query time would decrease as the number of > partitions increases, but that's not what I am seeing. I get better > results (0.29 ms) if I simply index DMID and don't use the partitions. I see really small times here so probably the overhead that partitioning imposes isn't worth yet. Maybe with 50M rows it'll help, you could try feeding those 50M tuples and test again. -- Albert Cervera Areny Dept. Informàtica Sedifa, S.L. Av. Can Bordoll, 149 08202 - Sabadell (Barcelona) Tel. 93 715 51 11 Fax. 93 715 51 12 ==================================================================== ........................ AVISO LEGAL ............................ La presente comunicación y sus anexos tiene como destinatario la persona a la que va dirigida, por lo que si usted lo recibe por error debe notificarlo al remitente y eliminarlo de su sistema, no pudiendo utilizarlo, total o parcialmente, para ningún fin. Su contenido puede tener información confidencial o protegida legalmente y únicamente expresa la opinión del remitente. El uso del correo electrónico vía Internet no permite asegurar ni la confidencialidad de los mensajes ni su correcta recepción. En el caso de que el destinatario no consintiera la utilización del correo electrónico, deberá ponerlo en nuestro conocimiento inmediatamente. ==================================================================== ........................... DISCLAIMER ............................. This message and its attachments are intended exclusively for the named addressee. If you receive this message in error, please immediately delete it from your system and notify the sender. You may not use this message or any part of it for any purpose. The message may contain information that is confidential or protected by law, and any opinions expressed are those of the individual sender. Internet e-mail guarantees neither the confidentiality nor the proper receipt of the message sent. If the addressee of this message does not consent to the use of internet e-mail, please inform us inmmediately. ====================================================================