Re: 2D partitioning of VLDB - sane or not? - Mailing list pgsql-hackers

From Gregory Stark
Subject Re: 2D partitioning of VLDB - sane or not?
Date
Msg-id 87hcn2mofu.fsf@oxford.xeocode.com
Whole thread Raw
In response to Re: 2D partitioning of VLDB - sane or not?  ("Zeugswetter Andreas ADI SD" <Andreas.Zeugswetter@s-itsolutions.at>)
List pgsql-hackers
"Zeugswetter Andreas ADI SD" <Andreas.Zeugswetter@s-itsolutions.at> writes:

> I'd say that that point currently is well below 2000 partitions for all
> common db systems.

I think it will depend heavily on the type of queries you're talking about.
Postgres's constraint_exclusion is a linear search and does quite a bit of
work for each constraint. So it isn't terribly efficient for more than 1,000
partitions or so.

*But* that only affects planning time. If your queries are always effectively
pruned to few partitions and you execute them thousands of times then you not
care about slow planning time.

And if the overall table is large enough and you're dropping and loading
partitions then you may still be benefiting from partitioning by keeping all
the loaded records together and allowing dropping a partition to be constant
time.

--  Gregory Stark EnterpriseDB          http://www.enterprisedb.com


pgsql-hackers by date:

Previous
From: "Zeugswetter Andreas ADI SD"
Date:
Subject: Re: 2D partitioning of VLDB - sane or not?
Next
From: Gregory Stark
Date:
Subject: Re: Testing the async-commit patch