Thread: source documentation tool doxygen

source documentation tool doxygen

From
Joachim Wieland
Date:
I've created a browsable source tree "documentation", it's done with the
doxygen tool.

http://www.mcknight.de/pgsql-doxygen/cvshead/html/

There was a discussion about this some time ago, Jonathan Gardner proposed
it here:

http://archives.postgresql.org/pgsql-hackers/2004-03/msg00748.php

quite a few people found it useful but it somehow got stuck. The reason was
apparently that you'd have to tweak your comments in order to profit from
it as much as possible.

However I still think it's a nice-to-have among the online documentation
and it gives people quite new to the code the possibility to easily check
what is where defined and how... What do you think?

doxygen can also produce a pdf but I haven't succeeded in doing that so far,
pdflatex keeps bailing out. Has anybody else succeeded building this yet?



Joachim

--
Joachim Wieland                                              joe@mcknight.de
C/ Usandizaga 12 1°B                                           ICQ: 37225940
20002 Donostia / San Sebastian (Spain)                     GPG key available


Re: source documentation tool doxygen

From
Thomas Hallgren
Date:
I wish I've had this when I started working with PostgreSQL. This looks really good. Very 
useful indeed, even without the comments. What kind of changes are needed in order to get 
the comments in?

Regards,
Thomas Hallgren

Joachim Wieland wrote:
> I've created a browsable source tree "documentation", it's done with the
> doxygen tool.
> 
> http://www.mcknight.de/pgsql-doxygen/cvshead/html/
> 
> There was a discussion about this some time ago, Jonathan Gardner proposed
> it here:
> 
> http://archives.postgresql.org/pgsql-hackers/2004-03/msg00748.php
> 
> quite a few people found it useful but it somehow got stuck. The reason was
> apparently that you'd have to tweak your comments in order to profit from
> it as much as possible.
> 
> However I still think it's a nice-to-have among the online documentation
> and it gives people quite new to the code the possibility to easily check
> what is where defined and how... What do you think?
> 
> doxygen can also produce a pdf but I haven't succeeded in doing that so far,
> pdflatex keeps bailing out. Has anybody else succeeded building this yet?
> 
> 
> 
> Joachim
> 



Re: source documentation tool doxygen

From
Kim Bisgaard
Date:
Try following the link (the Doxygen icon) - it has both a tutorial and
extensive doc.

Regards,
Kim Bisgaard

Thomas Hallgren wrote:

> I wish I've had this when I started working with PostgreSQL. This 
> looks really good. Very useful indeed, even without the comments. What 
> kind of changes are needed in order to get the comments in?
>
> Regards,
> Thomas Hallgren
>
> Joachim Wieland wrote:
>
>> I've created a browsable source tree "documentation", it's done with the
>> doxygen tool.
>>
>> http://www.mcknight.de/pgsql-doxygen/cvshead/html/
>>
>> There was a discussion about this some time ago, Jonathan Gardner 
>> proposed
>> it here:
>>
>> http://archives.postgresql.org/pgsql-hackers/2004-03/msg00748.php
>>
>> quite a few people found it useful but it somehow got stuck. The 
>> reason was
>> apparently that you'd have to tweak your comments in order to profit 
>> from
>> it as much as possible.
>>
>> However I still think it's a nice-to-have among the online documentation
>> and it gives people quite new to the code the possibility to easily 
>> check
>> what is where defined and how... What do you think?
>>
>> doxygen can also produce a pdf but I haven't succeeded in doing that 
>> so far,
>> pdflatex keeps bailing out. Has anybody else succeeded building this 
>> yet?
>>
>>
>>
>> Joachim
>>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
>



Re: source documentation tool doxygen

From
"Andrew Dunstan"
Date:
Thomas Hallgren said:
> I wish I've had this when I started working with PostgreSQL. This looks
> really good. Very  useful indeed, even without the comments. What kind
> of changes are needed in order to get  the comments in?
>

I too have done this. But retrofitting Doxygen style comments to the
PostgreSQL source code would be a big undertaking. Maintaining it, which
would be another task for reviewers/committers, would also be a pain unless
there were some automated checking tool.

cheers

andrew




Re: source documentation tool doxygen

From
Thomas Hallgren
Date:
Andrew,
Why not publish it as it stands today? Changing comments can be done in due time, no need to 
rush it. Or are the comments in some special format today that is used by some other tool?

What I'm trying to say is that for people like me, this would be very useful. Just clicking 
on a structure and see what file it's defined in is great. I don't care much if the comments 
are not perfect from day one (or ever for that matter). I can click on the file and read the 
comments right there if I want to.

If the comments have no special format today, well then perhaps the Doxygen format could be 
a good format to gradually adopt? No need to add checking etc. at this point. Perhaps later 
on (years from now) when most of the comments use that format.

Regards,
Thomas Hallgren

Andrew Dunstan wrote:
> Thomas Hallgren said:
>> I wish I've had this when I started working with PostgreSQL. This looks
>> really good. Very  useful indeed, even without the comments. What kind
>> of changes are needed in order to get  the comments in?
>>
> 
> I too have done this. But retrofitting Doxygen style comments to the
> PostgreSQL source code would be a big undertaking. Maintaining it, which
> would be another task for reviewers/committers, would also be a pain unless
> there were some automated checking tool.
> 
> cheers
> 
> andrew
> 
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
> 
>                http://www.postgresql.org/docs/faq
> 



Re: source documentation tool doxygen

From
Neil Conway
Date:
On Mon, 2006-01-16 at 07:57 -0600, Andrew Dunstan wrote:
> I too have done this. But retrofitting Doxygen style comments to the
> PostgreSQL source code would be a big undertaking. Maintaining it, which
> would be another task for reviewers/committers, would also be a pain unless
> there were some automated checking tool.

I don't think it would be all that painful. There would be no need to
convert the entire source tree to use proper Doxygen-style comments in
one fell swoop: individual files and modules can be converted whenever
anyone gets the inclination to do so. I don't think the maintenance
burden would be very substantial, either.

-Neil




Re: source documentation tool doxygen

From
Tom Lane
Date:
Neil Conway <neilc@samurai.com> writes:
> I don't think it would be all that painful. There would be no need to
> convert the entire source tree to use proper Doxygen-style comments in
> one fell swoop: individual files and modules can be converted whenever
> anyone gets the inclination to do so. I don't think the maintenance
> burden would be very substantial, either.

In the previous go-round on this topic, I seem to recall some concern
about side-effects, to wit reducing the readability of the comments for
ordinary non-doxygen code browsing.  I'd be quite against taking any
noticeable hit in that direction.

A quick look through the doxygen manual doesn't make it sound too
invasive, but I am worried about how well it will coexist with pgindent.
It seems both tools think they can dictate the meaning of the characters
immediately after /* of a comment block.
        regards, tom lane


Re: source documentation tool doxygen

From
Michael Glaesemann
Date:
On Jan 17, 2006, at 3:51 , Tom Lane wrote:

> A quick look through the doxygen manual doesn't make it sound too
> invasive, but I am worried about how well it will coexist with  
> pgindent.
> It seems both tools think they can dictate the meaning of the  
> characters
> immediately after /* of a comment block.

I haven't looked at it yet, but might there not be a way to have a  
preprocessing step where the current comment format is converted to  
something doxygen-friendly? pg_indent2doxygen or something? Then the  
current comment style could be maintained and doxygen could get what  
it likes.

Michael Glaesemann
grzm myrealbox com





Re: source documentation tool doxygen

From
Joachim Wieland
Date:
On Tue, Jan 17, 2006 at 06:51:15AM +0900, Michael Glaesemann wrote:
> I haven't looked at it yet, but might there not be a way to have a
> preprocessing step where the current comment format is converted to
> something doxygen-friendly? pg_indent2doxygen or something? Then the
> current comment style could be maintained and doxygen could get what
> it likes.

It's not about getting in all the comments, it's mostly about comments
describing functions / makros / typedefs / structures / global variables
and the like.

Comments in function bodies will just stay I guess.

However we could try to find some de facto rules about comments to get most
of the matches: For example a comment starting on column 0 of the line is
probably one you're looking for vs. one that has some whitespace at the
beginning is not.

Of course this would still need manual review for every file.


Joachim

--
Joachim Wieland                                              joe@mcknight.de
C/ Usandizaga 12 1°B                                           ICQ: 37225940
20002 Donostia / San Sebastian (Spain)                     GPG key available


Re: source documentation tool doxygen

From
Bruce Momjian
Date:
Wow, looks great.  Is that URL stable?  Can we link to it from the
PostgreSQL developers page?

    http://www.postgresql.org/developer/coding

---------------------------------------------------------------------------

Joachim Wieland wrote:
> I've created a browsable source tree "documentation", it's done with the
> doxygen tool.
>
> http://www.mcknight.de/pgsql-doxygen/cvshead/html/
>
> There was a discussion about this some time ago, Jonathan Gardner proposed
> it here:
>
> http://archives.postgresql.org/pgsql-hackers/2004-03/msg00748.php
>
> quite a few people found it useful but it somehow got stuck. The reason was
> apparently that you'd have to tweak your comments in order to profit from
> it as much as possible.
>
> However I still think it's a nice-to-have among the online documentation
> and it gives people quite new to the code the possibility to easily check
> what is where defined and how... What do you think?
>
> doxygen can also produce a pdf but I haven't succeeded in doing that so far,
> pdflatex keeps bailing out. Has anybody else succeeded building this yet?
>
>
>
> Joachim
>
> --
> Joachim Wieland                                              joe@mcknight.de
> C/ Usandizaga 12 1?B                                           ICQ: 37225940
> 20002 Donostia / San Sebastian (Spain)                     GPG key available
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
>                http://www.postgresql.org/docs/faq
>

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: [pgsql-www] source documentation tool doxygen

From
"Marc G. Fournier"
Date:
the only question I have ... is there any way of improving that right
index?  Love the 'detail pages', but somehow making the right index more
'tree like' might make navigation a bit easier ...



On Mon, 16 Jan 2006, Bruce Momjian wrote:

>
> Wow, looks great.  Is that URL stable?  Can we link to it from the
> PostgreSQL developers page?
>
>     http://www.postgresql.org/developer/coding
>
> ---------------------------------------------------------------------------
>
> Joachim Wieland wrote:
>> I've created a browsable source tree "documentation", it's done with the
>> doxygen tool.
>>
>> http://www.mcknight.de/pgsql-doxygen/cvshead/html/
>>
>> There was a discussion about this some time ago, Jonathan Gardner proposed
>> it here:
>>
>> http://archives.postgresql.org/pgsql-hackers/2004-03/msg00748.php
>>
>> quite a few people found it useful but it somehow got stuck. The reason was
>> apparently that you'd have to tweak your comments in order to profit from
>> it as much as possible.
>>
>> However I still think it's a nice-to-have among the online documentation
>> and it gives people quite new to the code the possibility to easily check
>> what is where defined and how... What do you think?
>>
>> doxygen can also produce a pdf but I haven't succeeded in doing that so far,
>> pdflatex keeps bailing out. Has anybody else succeeded building this yet?
>>
>>
>>
>> Joachim
>>
>> --
>> Joachim Wieland                                              joe@mcknight.de
>> C/ Usandizaga 12 1?B                                           ICQ: 37225940
>> 20002 Donostia / San Sebastian (Spain)                     GPG key available
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 3: Have you checked our extensive FAQ?
>>
>>                http://www.postgresql.org/docs/faq
>>
>
> --
>  Bruce Momjian                        |  http://candle.pha.pa.us
>  pgman@candle.pha.pa.us               |  (610) 359-1001
>  +  If your life is a hard drive,     |  13 Roberts Road
>  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings
>

----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org           Yahoo!: yscrappy              ICQ: 7615664

Re: [pgsql-www] source documentation tool doxygen

From
Michael Glaesemann
Date:
On Jan 17, 2006, at 8:40 , Marc G. Fournier wrote:

>
> the only question I have ... is there any way of improving that
> right index?  Love the 'detail pages', but somehow making the right
> index more 'tree like' might make navigation a bit easier ...

Along those lines, I wonder if a CSS couldn't be worked up to
integrate the look with the rest of the site.

Michael Glaesemann
grzm myrealbox com




Re: [pgsql-www] source documentation tool doxygen

From
Robert Treat
Date:
This was my plan all along, just been waiting for someone to make it work with
the postgresql code and then send instructions to the postgresql web team on
how to set it up.

Robert Treat

On Monday 16 January 2006 18:32, Bruce Momjian wrote:
> Wow, looks great.  Is that URL stable?  Can we link to it from the
> PostgreSQL developers page?
>
>  http://www.postgresql.org/developer/coding
>
> ---------------------------------------------------------------------------
>
> Joachim Wieland wrote:
> > I've created a browsable source tree "documentation", it's done with the
> > doxygen tool.
> >
> > http://www.mcknight.de/pgsql-doxygen/cvshead/html/
> >
> > There was a discussion about this some time ago, Jonathan Gardner
> > proposed it here:
> >
> > http://archives.postgresql.org/pgsql-hackers/2004-03/msg00748.php
> >
> > quite a few people found it useful but it somehow got stuck. The reason
> > was apparently that you'd have to tweak your comments in order to profit
> > from it as much as possible.
> >
> > However I still think it's a nice-to-have among the online documentation
> > and it gives people quite new to the code the possibility to easily check
> > what is where defined and how... What do you think?
> >
> > doxygen can also produce a pdf but I haven't succeeded in doing that so
> > far, pdflatex keeps bailing out. Has anybody else succeeded building this
> > yet?
> >
> >
> >
> > Joachim
> >
> > --
> > Joachim Wieland
> > joe@mcknight.de C/ Usandizaga 12 1?B
> >      ICQ: 37225940 20002 Donostia / San Sebastian (Spain)
> >     GPG key available
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 3: Have you checked our extensive FAQ?
> >
> >                http://www.postgresql.org/docs/faq

--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

Re: source documentation tool doxygen

From
Jonathan Gardner
Date:
On Monday 16 January 2006 10:51, Tom Lane wrote:
> Neil Conway <neilc@samurai.com> writes:
> > I don't think it would be all that painful. There would be no need to
> > convert the entire source tree to use proper Doxygen-style comments in
> > one fell swoop: individual files and modules can be converted whenever
> > anyone gets the inclination to do so. I don't think the maintenance
> > burden would be very substantial, either.
> 
> In the previous go-round on this topic, I seem to recall some concern
> about side-effects, to wit reducing the readability of the comments for
> ordinary non-doxygen code browsing.  I'd be quite against taking any
> noticeable hit in that direction.
> 
> A quick look through the doxygen manual doesn't make it sound too
> invasive, but I am worried about how well it will coexist with pgindent.
> It seems both tools think they can dictate the meaning of the characters
> immediately after /* of a comment block.
> 

In my experimentation, I found that there are really three paths for using 
Doxygen on code.

(1) Do nothing to the code. Doxygen in this state works pretty well. You can 
browse the code and you can find references to things. Some of the comments 
are picked up correctly as-is. It does a really good job of formatting the 
code for HTML.

(2) Do minimally invasive changes to make Doxygen recognize comments better. 
Example: Move comments from behind to in-front, or vice-versa. Put an extra 
'*' after '/*' to denote special documentation. This makes Doxygen especially 
useful, because most things already have proper documentation and bringing 
that out to the person who browses Doxygen's output is very good. All of a 
sudden, we have a manual that reads like a book describing how PostgreSQL is 
put together PLUS we still have the marked-up code.

(3) Do maximally invasive changes to the comments. This includes things like 
using Doxygen-specific markup, putting in LaTeX for formulas, and putting in 
cross-references. This is a huge amount of work (initial + maintenance) and 
doesn't add much benefit beyond what (2) has. It makes the book read more 
like a book, but it was already pretty readable.

It's not a question of WHETHER to use Doxygen. You can use it whether or not 
anyone else wants to. It's really a question of how far do we bend backwards 
to accomodate Doxygen. I'd say we just stick to moving comments around and 
putting in the extra '*'s. Let's not encourage people to put in Doxygen 
markup that isn't obvious. If someone contributes code with Doxygen mark-up, 
we won't turn them down but we won't promise we'll keep their markup as 
pretty as it was when they submitted it.

-- 
Jonathan Gardner
jgardner@jonathangardner.net


Re: source documentation tool doxygen

From
Tom Lane
Date:
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Wow, looks great.  Is that URL stable?  Can we link to it from the
> PostgreSQL developers page?

The thing seems to have only the very vaguest grasp on whether it is
parsing C or C++ ... or should I say that it is convinced it is parsing
C++ despite all evidence to the contrary?  I'd be happier with the
pretty pictures if they had some connection to reality.

            regards, tom lane

Re: source documentation tool doxygen

From
Joachim Wieland
Date:
On Tue, Jan 17, 2006 at 12:15:02AM -0500, Tom Lane wrote:
> The thing seems to have only the very vaguest grasp on whether it is
> parsing C or C++ ... or should I say that it is convinced it is parsing
> C++ despite all evidence to the contrary?  I'd be happier with the
> pretty pictures if they had some connection to reality.

True. Doxygen is developped for documenting C++ source code. There is
however an option OPTIMIZE_OUTPUT_FOR_C which is set. There are a few more
options for class and collaboration graphs as well as a switch to create a
graphical class hierarchy, all of them are turned on at the moment. So it
might get more concise but we'll have to play around with it to see what is
useful and what isn't.


Joachim

--
Joachim Wieland                                              joe@mcknight.de
C/ Usandizaga 12 1°B                                           ICQ: 37225940
20002 Donostia / San Sebastian (Spain)                     GPG key available

Re: [pgsql-www] source documentation tool doxygen

From
Joachim Wieland
Date:
On Mon, Jan 16, 2006 at 07:42:35PM -0500, Robert Treat wrote:
> This was my plan all along, just been waiting for someone to make it work
> with the postgresql code and then send instructions to the postgresql web
> team on how to set it up.

I volunteer to tell you after I've found out for myself ;-)

Some details: What I have put online on my website occupies about 240 MBs of
disk space and gets built in 1.5h on my PIII 800 Laptop. Removing useless
graphs this can be reduced to less than 200 MBs I imagine.

Do you want to put it on the postgresql.org site nevertheless? Is it too big
to be mirrored and should be recreated on every webserver? We might need
one copy for the last version of every major release as well as one for
cvs. The latter should get updated regularly of course but I figure it would
be sufficient to do that once a week...


Joachim

--
Joachim Wieland                                              joe@mcknight.de
C/ Usandizaga 12 1°B                                           ICQ: 37225940
20002 Donostia / San Sebastian (Spain)                     GPG key available

Re: [pgsql-www] source documentation tool doxygen

From
Joachim Wieland
Date:
On Tue, Jan 17, 2006 at 09:34:06AM +0900, Michael Glaesemann wrote:
> Along those lines, I wonder if a CSS couldn't be worked up to
> integrate the look with the rest of the site.

Yes, it's stylesheet based. However I don't know yet to what extend you can
change the look. It allows for a custom header and footer as well. The
postgresql logo on top would be nice but the navigation menu on the left has
to be sacrificed for more space.


Joachim

--
Joachim Wieland                                              joe@mcknight.de
C/ Usandizaga 12 1°B                                           ICQ: 37225940
20002 Donostia / San Sebastian (Spain)                     GPG key available

Re: [pgsql-www] source documentation tool doxygen

From
"Andrew Dunstan"
Date:
Joachim Wieland said:
> Do you want to put it on the postgresql.org site nevertheless? Is it
> too big to be mirrored and should be recreated on every webserver? We
> might need one copy for the last version of every major release as well
> as one for cvs. The latter should get updated regularly of course but I
> figure it would be sufficient to do that once a week...
>


The overwhelming amount of development work gets done against HEAD. I would
start with a once a day run against HEAD, and possibly one against the
latest stable branch (currently REL8_1_STABLE in cvs). That would get you
99% of the possible benefit, I think. I don't see any virtue in doing it
against a release as opposed to a stable branch - this is to help
development efforts so it should be against what people should be basing
their development efforts on.


cheers

andrew




Re: [pgsql-www] source documentation tool doxygen

From
Tom Lane
Date:
"Andrew Dunstan" <andrew@dunslane.net> writes:
> The overwhelming amount of development work gets done against HEAD. I would
> start with a once a day run against HEAD, and possibly one against the
> latest stable branch (currently REL8_1_STABLE in cvs). That would get you
> 99% of the possible benefit, I think.

I agree --- I see no reason for us to maintain such documentation for
anything except HEAD.  If somebody really wants documentation for a
stable branch, they can build it themselves.

            regards, tom lane