Re: [HACKERS] Schema boggle... - Mailing list pgsql-sql

Hi Marc,

I was actually leaning towards schema's as a method to partition the data 
out when I was beginning to plan AMS 2, your suggestions of schema's 
confirmed this as a good approach for what we were looking for so I started 
to code the initial layout for the Schema in April/May, but at the time I 
did not have the full picture in my head on dealing with some sections of 
the database....

To address Josh's comments though.
 Monolithic tables have their uses, I don't and wont contest that, however 
we've been using a monolithic structure in our current AMS distribution and 
in a lot of areas it has been quite a pain to try and bring some uniformity 
into the structure. First off, I have an ever changing platform and one of 
the issues I had was that development under the monolithic structure 
because problematic and very difficult to merge new elements in, granted 
the SQL schema was not all that great and a number of areas could have used 
improvement, but over all it was more trouble then it was worth.
 By breaking out our client content into a schema structure, I've 
simplified the management of my data immensely, I can easily delete 
old/unused data with out having to worry about updating scripts that have 
been written to clean out structures. I can give clients greater control 
over their own data with out worrying about some "security" issue that I 
might have missed in my programming, so there's and added level of safety 
in an already tight structure. I've "objectified" out the content into an 
easily changeable/"update friendly" system of tables, views and functions. 
I've reduced a fair amount of overhead by limiting the order of tables, For 
instance, with our original monolithic dataset, we have approx 90+ tables 
handling all of the content that we use on a regular basis. With this new 
structure I've reduced that down to a total of 30 tables, 8 of which are 
repeated across the schemas. The method we are using the schemas in also 
allows me to work in some assumptions that were tricky to code under the 
monolithic structure, with the schema system, simplicity of the table set 
allowed allowed me to simulate those assumptions with out having to 
re-write code each time I have an update, thus giving me a highly dynamic 
dataset. That and I'm more confident working on a 3D level, versus 2D, the 
schema levels introduce the third dimension into my structure and simplify 
visualizing how I want things to work.
 Within that third dimensional structure, it made sense to be able to code 
out an SQL statement that would have searched the schema set for matching 
patterns to that given in the SQL query, similar to the way one can search 
for a pattern on data in a column. But Tom has given me an idea that will 
allow me to work out how to search multiple schemas within a dynamic 
plpgsql function that figures out all my patterned schemas and executes the 
resulting query as he suggested, I just need to learn plpgsql programming 
so that I can work out all of the details (Tom, I might bug you about that 
once in a while).


At 09:45 PM 11/5/03 -0400, Marc G. Fournier wrote:

>Actually, the use of schema's was my idea, to speed up some dreadfully
>slow queries dealing with traffic stats from a table that was growing
>painfully monolithic ... the idea is/was that it would be easier to
>backup/remove all data pertaining to a specific client if they decided to
>close their account ...
>
>On Wed, 5 Nov 2003, Josh Berkus wrote:
>
> > Chris,
> >
> > >  I work with Marc Fournier, for those who don't know, and have been
> > > working extensively with the schemas feature for Hub.Org's new Account
> > > Management package. Each client's data is stored in a seperate schema
> > > set asside just for them (though they will never have direct access to
> > > it, it helps us "balance" out the data. Each schema has the same set of
> > > tables, with which different data is stored, depending on the client.
> >
> > While this is an interesting use of Schema, it is not, IMHO, a good way to
> > partition client data.    Far better to have monolithic tables with a
> > "client_id" column and then construct *views* which are available in each
> > schema only to that client.   Otherwise, as you've discovered, top-level
> > management becomes a royal pain.
> >
> > I'd be happy to discuss this further on PGSQL-SQL, which is really the
> > appropriate mailing list.
> >
> > --
> > -Josh Berkus
> >  Aglio Database Solutions
> >  San Francisco
> >
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 8: explain analyze is your friend
> >