Re: [HACKERS] Re: [QUESTIONS] Does Storage Manager support >2GB tables? - Mailing list pgsql-hackers

From ocie@paracel.com
Subject Re: [HACKERS] Re: [QUESTIONS] Does Storage Manager support >2GB tables?
Date
Msg-id 9803122257.AA29650@dolomite.paracel.com
Whole thread Raw
In response to Re: [HACKERS] Re: [QUESTIONS] Does Storage Manager support >2GB tables?  (dg@illustra.com (David Gould))
List pgsql-hackers
David Gould wrote:
>
> Bruce Momjian writes:
> > I am adding this to the TODO list:
> >
> >     * Do async I/O to do better read-ahead of data
>
> Good.
>
> > Because we are not threaded, we really can't do anything else while we
> > are waiting for I/O, but we can pre-request data we know we will need.
>
> Threading is a bit like raw devices. It sounds like a really good idea,
> particularly with M$ banging the "NT, now with threads" drum, but in real
> life there are some very good reasons not to thread. Particularly with an
> extensible product like Postgres where J-Random routine gets loaded at
> runtime. In a threaded system, J-Random routine needs to be pretty well
> perfect or the whole system comes down. In a process based system, unless
> it trashes something in the shared memory, only the one connection instance
> needs to come down. My experience with Illustra says that this is fairly
> important.

I tend to agree.  I think threads are more usefull in applications
software, where the programmer has at least some idea of the use of
the program.

On a more practical/implementation note, it might be nice to have one
process that can perform prefetches for the others.  I.E.  If I know
that I will need page X, I queue a request to this process (in shared
memory?) and it will read in this page for us (or even write a page to
disk for us).

Asynchronous I/O is interesting, but I don't know how widely it has
been implemented (Linux has none, unless the development version is
working on it, and Solaris has aio, but not the POSIX flavor yet).


Ocie Mitchell

pgsql-hackers by date:

Previous
From: dg@illustra.com (David Gould)
Date:
Subject: Re: [HACKERS] Re: [QUESTIONS] Does Storage Manager support >2GB tables?
Next
From: Bruce Momjian
Date:
Subject: Re: [HACKERS] Re: [QUESTIONS] Does Storage Manager support >2GB tables?