Re: msession for PostgreSQL? - Mailing list pgsql-hackers

> On Fri, Jun 11, 2004 at 11:51:04AM -0400, pgsql@mohawksoft.com wrote:
>
>> The best part of it could be that it could replace the whole msession C
>> API with PostgreSQL. You can join against the various data, and it
>> should
>> be very fast with no MVCC overhead for those aspects of your project
>> that
>> don't need it while still allowing them to be incorporated with the data
>> that does.
>
> I don't get it.  If msession is already successful as it is, why do you
> want to change it?
>
> It seems to me you are looking for problems to solve.  If you want to
> code, there are lots of ideas in PostgreSQL's TODO.

Actually, *all* problems have been more or less solved given enough tools
time to piece together the solution. SQL itself didn't really *solve* a
problem, it only made an easier solution.

In a highly interactive web site or other complex DB project, there are
different classes of data storage and services. High volatility, low
volatility, temporary, more long lasting. PostgreSQL falls apart on the
high volatility temporary data class of problem.

In 1999-2002, I was working for a Web company that needed to spread
session information across a lot of boxes. The target was thousands of
hits per second, tens or hundreds of thousands of active users. Think
about the scalability of that problem. Most of the transactions would be
wasted. It was a huge waste to try to make sure all these actions were
saved. We could lose the temporary information once in a while with no
problems, but we had to keep the perminent information.

So, one class of data, the instantaneous "what they did last" and maybe
shopping cart information was important, but could be lost once in a very
great while without any loss of company revenue or reputation.

The next class of problem was the transaction class of problem, credid
card numbers, billing, and accounting, you can't screw around with that
stuff.

The next class of problem were the fixed inventory tables, product IDs,
links to jpegs, etc.

The next service was replication of persistent data (user info, inventory,
etc.) to the slaves.

The next service was full text search of the product inventory.

The last service was a recommendations system.

As Tom and Bruce will probably remember, I've been a real PITA. I was
pushing for sets of rows being able to be returned from functions since
7.0.

Anyway, what we finally made was a real kludge. I eventually used my
msession server, Oracle, PostgreSQL, PHP, my own full text search engine
(FTSS), and my own recommendations system (CGR).

The hard part of this system was that I was the only one who understood it
all. Having written most of it, I was always the one to fix it or debug
it. It worked, it didn't cost near a million dollars, but it was way too
complicated. Where was the data? In Oracle? in PostgreSQL? In Msession? In
FTSS? In CGR? How did you join information in msession to rows in Oracle?
How did you join rows in Oracle to rows in PostgreSQL? And so on.

The purpose of these various diatribes and rants is to address some issues
with PostgreSQL that limited its application and thus forced us to use
other technologies for different classes of problem. Currently being a
consultant, and advising on these classes of problem, I am still
continuing to create what I consider to be overly complicated systems.

Ideally, I would like to have a single interface to the required data and
functionality. It would be fantastic if users didn't all each have to
solve these problems. It would be a huge plus for PostgreSQL as it would
become *the* web DB of choice over *any* competitor.