Thread: Guide to PostgreSQL source tree
Hi all,
For a few months now I've been thinking about whether or not a guide
('line-by-line') to the Postgres source tree would be of any value.
Such a guide would, most probably, trace an 'ultimate' query (ie,
one which requires the use of all source level functionality) through the
source as well as reference appendices guides to underlying functionality
(backend lib, transactions, macros) and client interfaces (inc. FE/BE
protocol, internals of the libs etc), procedural language interfaces, SPI
and any other part of the source I have left out.
The guide would look at all non-trivial functions and code segments in the
source. Examination would involve explanation of complex code (at a
line-by-line level), background information of reasoning behind the
code-level design of important functionality, analysis of algorithms and
what ever else seems helpful to people approaching the source.
I think it would be most useful as a non-commercial project intended for
distribution from the Postgres Web site.
Obviously such a project would take a very long time and would have to
involve more people than myself. So the questions which go unanswered
are: Would such a work be of any real use? Would it be of use to enough
people? Is this the right way to go about introducing people to the
source? Is it desirable to introduce (lots of) people to the source?
For my part, I think such a project would be a good way of countering two
things which are affecting Postgres's popularity. Firstly, in Australia
(and I imagine other parts of the world) University courses dealing with
database/information systems (whether it be as basic as an introduction to
SQL or as detailed as developing complicated/sophisticated data storage
systems) are more often than not sponsored by vendors (Oracle, IBM,
Sybase) or 'consultancy' companies who may as well be sales reps for
vendors. Which ever it is, in the end courses are full of marketing drivel
and very little analysis and exploration of real concepts/problems.
Secondly, where Postgres really out performs proprietary databases is in
its source being open. Problems which cause major functionality problems
and downtime in critical vendor installations could often easily be
resolved if developers had the source. I think a thorough source guide
would go some way to bolstering the appeal of Postgres to such developers
whilst countering some of the arguments for throwing hefty sums of money
at support.
So, what do people think?
Gavin
----- Original Message -----
From: Gavin Sherry <swm@linuxworld.com.au>
Sent: Sunday, August 19, 2001 3:40 AM
> Hi all,
>
> For a few months now I've been thinking about whether or not a guide
> ('line-by-line') to the Postgres source tree would be of any value.
[snip]
> So, what do people think?
Such a guide would be nice to have handy as a reference, but how
are you going to keep up with frequent code changes
and the new code added? Keeping this thing up-to-date
is an enormous effort even if you're not the only
one involved.
And what's wrong in automated guide generation? Like
if put specially formatted comments (responsibility of the author of the piece)
for the guide in the source, and run through the source tree
your guide generator from time to time (like on every beta, RC, and release perhaps?).
This way the guide maintenance will be much easier, but the quality will solely
depend on comments supplied by the author.
My two Canadian cents.
Serguei
"Serguei Mokhov" <sa_mokho@alcor.concordia.ca> writes:
> And what's wrong in automated guide generation?
Even more to the point, what's wrong with looking at the source code?
(Why would you want a "line by line" guide if you're not looking at the
source code, anyway?)
We could probably do with more extensive high-level documentation than
we have, to point people to the parts of the code that they need to read
for a particular purpose. But I agree with Sergei that it's hopeless
to try to divorce low-level documentation from the code itself.
One thing that I find absolutely essential for dealing with any large
project is a full-text indexer (I use Glimpse, but I think there are
others out there). Being able to quickly look at every use of a
particular identifier goes a long way towards answering questions.
regards, tom lane
On Sun, 19 Aug 2001, Tom Lane wrote: > One thing that I find absolutely essential for dealing with any large > project is a full-text indexer (I use Glimpse, but I think there are > others out there). Being able to quickly look at every use of a > particular identifier goes a long way towards answering questions. Agreed -- you can't find your way around PostgreSQL without such a program. Personally, I use Source Navigator which you can grab at http://sources.redhat.com/sourcenav/ . The really useful thing about source navigator is that it parses the source into functions, variables, etc. rather than just indexing it all as text. This means when you are looking at a source file with it, you can do neat things like click on a function call and then see things like the declaration and a x-ref tree. Very handy. Neil -- Neil Padgett Red Hat Canada Ltd. E-Mail: npadgett@redhat.com 2323 Yonge Street, Suite #300, Toronto, ON M4P 2C9
> Hi all,
>
> For a few months now I've been thinking about whether or not a guide
> ('line-by-line') to the Postgres source tree would be of any value.
>
> Such a guide would, most probably, trace an 'ultimate' query (ie,
> one which requires the use of all source level functionality) through the
> source as well as reference appendices guides to underlying functionality
> (backend lib, transactions, macros) and client interfaces (inc. FE/BE
> protocol, internals of the libs etc), procedural language interfaces, SPI
> and any other part of the source I have left out.
I wonder if we should just work down from where we are now. We have the
FAQ, backend flowchard, and "PostgreSQL Internals through Pictures" that
I wrote, plus a presentation by Tom Lane. Where do we go from there?
> Secondly, where Postgres really out performs proprietary databases is in
> its source being open. Problems which cause major functionality problems
> and downtime in critical vendor installations could often easily be
> resolved if developers had the source. I think a thorough source guide
> would go some way to bolstering the appeal of Postgres to such developers
> whilst countering some of the arguments for throwing hefty sums of money
> at support.
I didn't realize that was happening.
-- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610)
853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill,
Pennsylvania19026
> On Sun, 19 Aug 2001, Tom Lane wrote: > > One thing that I find absolutely essential for dealing with any large > > project is a full-text indexer (I use Glimpse, but I think there are > > others out there). Being able to quickly look at every use of a > > particular identifier goes a long way towards answering questions. > > Agreed -- you can't find your way around PostgreSQL without such a > program. Personally, I use Source Navigator which you can grab at > http://sources.redhat.com/sourcenav/ . The really useful thing about This is the cygnus programmer's editor, right? It is a nice piece of software. > source navigator is that it parses the source into functions, variables, > etc. rather than just indexing it all as text. This means when you are > looking at a source file with it, you can do neat things like click on a > function call and then see things like the declaration and a x-ref > tree. Very handy. I have found I need tags support too. You can do project-wide function/identifier searching either in your editor, if it supports that, or using an external program like glimpse or idutils. Syntax colorization helps too. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026