I have been asked to deliver a seminar on use of PostgreSQL in large data environments. I have decided to send this to the general list as well because I think it may be of general interest here,
We will be talking about how problems and solutions for PostgreSQL when using it to manage 10-12TB of data under heavy load.
In particular, 4 specific problems we faced will be discussed in technical detail:
1. How table partitioning solved a number of our performance problems.
2. How non-1NF designs solved some problems and created some new ones.
3. How PL/Perl was used to address large data sets of semi-structured data.
4. How use of PostgreSQL as job queue store raised unexpected problems, and their solutions.
The discussion surrounds a case study which shows how well PostgreSQL scales, but offers a number of lessons for database management in smaller sizes as well.
