Thread: Where can I find the doxyfile?

Where can I find the doxyfile?

From
Xin Wang
Date:
Hi,
I don't know where I can find the doxyfile which generate
"doxygen.postgresql.org" web site. I found that when reading code the
doxygen source code is quite helpful. However, I want to generate an
off-line copy of doxygen docs myself, but I can't find the doxyfile in
the lastest source release.
Thanks!


Re: Where can I find the doxyfile?

From
Zdenek Kotala
Date:
Xin Wang napsal(a):
> Hi,
> I don't know where I can find the doxyfile which generate
> "doxygen.postgresql.org" web site. I found that when reading code the
> doxygen source code is quite helpful. However, I want to generate an
> off-line copy of doxygen docs myself, but I can't find the doxyfile in
> the lastest source release.

I think it is good idea. Stefan, what's about put it on the wiki?
    Zdenek




Re: Where can I find the doxyfile?

From
Stefan Kaltenbrunner
Date:
Zdenek Kotala wrote:
> Xin Wang napsal(a):
>> Hi,
>> I don't know where I can find the doxyfile which generate
>> "doxygen.postgresql.org" web site. I found that when reading code the
>> doxygen source code is quite helpful. However, I want to generate an
>> off-line copy of doxygen docs myself, but I can't find the doxyfile in
>> the lastest source release.
> 
> I think it is good idea. Stefan, what's about put it on the wiki?

don't think the wiki is a good place (we would have to maintain it in
addition to the main file) so I simply added a link on
http:/doxygen.postgresql.org that contains the current copy of the
configuration file that was used to generate a particular set of docs.
However some of the things there are specific to our install so a bit
(like some of the paths) of manual change will be required anyway.


Stefan


Re: Where can I find the doxyfile?

From
Bogdan Mart
Date:
On 2008-06-02 18:33:42 Stefan Kaltenbrunner wrote:
>
> Zdenek Kotala wrote:
> > Xin Wang napsal(a):
> >> Hi,
> >> I don't know where I can find the doxyfile which generate
> >> "doxygen.postgresql.org" web site. I found that when reading code the
> >> doxygen source code is quite helpful. However, I want to generate an
> >> off-line copy of doxygen docs myself, but I can't find the doxyfile in
> >> the lastest source release.
> >
> > I think it is good idea. Stefan, what's about put it on the wiki?
>
> don't think the wiki is a good place (we would have to maintain it in
> addition to the main file) so I simply added a link on
> http:/doxygen.postgresql.org that contains the current copy of the
> configuration file that was used to generate a particular set of docs.
> However some of the things there are specific to our install so a bit
> (like some of the paths) of manual change will be required anyway.

Hello, sorry for resurrecting this old discussion, and necroposting,
but this was relevant.

It turns out that way back in 2008 it was possible to download Doxyfile
from https://doxygen.postgresql.org/ , which can be seen at:
https://web.archive.org/web/20080915132924/https://doxygen.postgresql.org/

I was able to download this Doxyfile from the web archive, and it actually
worked for me, after I changed the path to postgres sources and removed
header template. It even produced same result as on current web-site.

It would be nice if this doxygen file would be placed somewhere
on the current site.

P.S. I've tried to run doxygen on postgres sources without config,
and it didn't find any source file, this archived thread helped.

Perhaps doxygen file can be just placed into postgres git repository?

Best regards,
Bohdan Mart.



Re: Where can I find the doxyfile?

From
Bruce Momjian
Date:
On Fri, Oct  6, 2023 at 10:50:25PM +0300, Bogdan Mart wrote:
> On 2008-06-02 18:33:42 Stefan Kaltenbrunner wrote:
> >
> > Zdenek Kotala wrote:
> > > Xin Wang napsal(a):
> > >> Hi,
> > >> I don't know where I can find the doxyfile which generate
> > >> "doxygen.postgresql.org" web site. I found that when reading code the
> > >> doxygen source code is quite helpful. However, I want to generate an
> > >> off-line copy of doxygen docs myself, but I can't find the doxyfile in
> > >> the lastest source release.
> > >
> > > I think it is good idea. Stefan, what's about put it on the wiki?
> >
> > don't think the wiki is a good place (we would have to maintain it in
> > addition to the main file) so I simply added a link on
> > http:/doxygen.postgresql.org that contains the current copy of the
> > configuration file that was used to generate a particular set of docs.
> > However some of the things there are specific to our install so a bit
> > (like some of the paths) of manual change will be required anyway.
> 
> Hello, sorry for resurrecting this old discussion, and necroposting,
> but this was relevant.
> 
> It turns out that way back in 2008 it was possible to download Doxyfile
> from https://doxygen.postgresql.org/ , which can be seen at:
> https://web.archive.org/web/20080915132924/https://doxygen.postgresql.org/
> 
> I was able to download this Doxyfile from the web archive, and it actually
> worked for me, after I changed the path to postgres sources and removed
> header template. It even produced same result as on current web-site.
> 
> It would be nice if this doxygen file would be placed somewhere
> on the current site.
> 
> P.S. I've tried to run doxygen on postgres sources without config,
> and it didn't find any source file, this archived thread helped.
> 
> Perhaps doxygen file can be just placed into postgres git repository?

We do link to a Doxygen build from our website:

    https://www.postgresql.org/developer/coding/

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.



Re: Where can I find the doxyfile?

From
mart.bogdan@gmail.com
Date:

On 07.10.23 00:06, Bruce Momjian wrote:
> On Fri, Oct  6, 2023 at 10:50:25PM +0300, Bogdan Mart wrote:
> > On 2008-06-02 18:33:42 Stefan Kaltenbrunner wrote:
> >>
> >> Zdenek Kotala wrote:
> >>> Xin Wang napsal(a):
> >>>> Hi,
> >>>> I don't know where I can find the doxyfile which generate
> >>>> "doxygen.postgresql.org" web site. I found that when reading code the
> >>>> doxygen source code is quite helpful. However, I want to generate an
> >>>> off-line copy of doxygen docs myself, but I can't find the doxyfile in
> >>>> the lastest source release.
> >>>
> >>> I think it is good idea. Stefan, what's about put it on the wiki?
> >>
> >> don't think the wiki is a good place (we would have to maintain it in
> >> addition to the main file) so I simply added a link on
> >> http:/doxygen.postgresql.org that contains the current copy of the
> >> configuration file that was used to generate a particular set of docs.
> >> However some of the things there are specific to our install so a bit
> >> (like some of the paths) of manual change will be required anyway.
> >
> > Hello, sorry for resurrecting this old discussion, and necroposting,
> > but this was relevant.
> >
> > It turns out that way back in 2008 it was possible to download Doxyfile
> > from https://doxygen.postgresql.org/ , which can be seen at:
> > https://web.archive.org/web/20080915132924/https://doxygen.postgresql.org/
> >
> > I was able to download this Doxyfile from the web archive, and it actually
> > worked for me, after I changed the path to postgres sources and removed
> > header template. It even produced same result as on current web-site.
> >
> > It would be nice if this doxygen file would be placed somewhere
> > on the current site.
> >
> > P.S. I've tried to run doxygen on postgres sources without config,
> > and it didn't find any source file, this archived thread helped.
> >
> > Perhaps doxygen file can be just placed into postgres git repository?
> 
> We do link to a Doxygen build from our website:
> 
>     https://www.postgresql.org/developer/coding/

Yes, there is link to deployed documentation generated by Doxygen
(unless I'm missing something), but I am talking about config file
for Doxygen to generate same documentation locally. 
There are no entries on Wiki regarding doxygen,
and not in build instructions. 

I was able to locate needed config file in Web Archive:
https://web.archive.org/web/20081210081808if_/http://doxygen.postgresql.org:80/pgsql.conf

But it would be nice if it was readily available if anybody other would like
to build docs locally. 



RE: Where can I find the doxyfile?

From
Date:
Sometime earlier, I created a filter to annotate regular C comments as doxy
comments.  I'll attach it along with a sample doxyfile for running it.  Just
in case it looks useful.

I've never been a big fan of Doxygen, but it seems to have gotten better
over time. Now that some IDEs display doxy comments on hover, I'm beginning
to appreciate it.



Attachment

RE: Where can I find the doxyfile?

From
Date:
Sometime earlier, I created a filter to annotate regular C comments as doxy comments.
I'll attach it along with a sample doxyfile for running it.  Just in case it looks useful.

I've never been a big fan of Doxygen, but it seems to have gotten better over time.
Now that some editors are displaying doxygen comments on hover, I'm beginning to appreciate it.




d

Attachment

Re: Where can I find the doxyfile?

From
Bohdan Mart
Date:
On 07.10.23 00:29, postgres@coyotebush.net wrote:
> Sometime earlier, I created a filter to annotate regular C comments as doxy
> comments.  I'll attach it along with a sample doxyfile for running it.  Just
> in case it looks useful.
>
> I've never been a big fan of Doxygen, but it seems to have gotten better
> over time. Now that some IDEs display doxy comments on hover, I'm beginning
> to appreciate it.

Thank you for providing this `flex` filter! It is actually useful. I've 
tested it on one

file from postures source base, and it actually converted regular

comments to doc-comments! If this filter could be used by official

Doxygen generation of Postgres, it would highly increase quality

of online documentation of Postgres!


My initial idea was to create patches to upstream with changing comments

to documented comments, because current online dock is lacking comments,

but with this filter it would be unneeded, as documentation would be 
able to

be generated from current sources!


To illustrate the point:

~~~~~~~~~~~

This function has Doxy style comment

```

src/interfaces/ecpg/compatlib/informix.c
672:/**
673- * initialize the struct, which holds the different forms
674- * of the long value
675- */
676-static int
677-initValue(long lng_val)

```

And it is rendered properly:

https://doxygen.postgresql.org/informix_8c.html#a0dad4c2ee52a831d3aa1bf1133e0e17d


But  function like this

```

src/backend/foreign/foreign.c

  /*
   * get_foreign_server_oid - given a server name, look up the OID
   *
   * If missing_ok is false, throw an error if name not found.  If true, 
just
   * return InvalidOid.
   */
  Oid
  get_foreign_server_oid(const char *servername, bool missing_ok)

```

https://doxygen.postgresql.org/foreign_8c.html#a7959c56969be440c25b62c3c98ce2a78

doesn't have rendered documentation.


P.S. Was this original message from postgres@coyotebush.net intended to 
be sent on John's Morris behalf?




Re: Where can I find the doxyfile?

From
Stefan Kaltenbrunner
Date:
On 07.10.23 00:23, Bohdan Mart wrote:
> 
> On 07.10.23 00:29, postgres@coyotebush.net wrote:
>> Sometime earlier, I created a filter to annotate regular C comments as 
>> doxy
>> comments.  I'll attach it along with a sample doxyfile for running 
>> it.  Just
>> in case it looks useful.
>>
>> I've never been a big fan of Doxygen, but it seems to have gotten better
>> over time. Now that some IDEs display doxy comments on hover, I'm 
>> beginning
>> to appreciate it.
> 
> Thank you for providing this `flex` filter! It is actually useful. I've 
> tested it on one
> 
> file from postures source base, and it actually converted regular
> 
> comments to doc-comments! If this filter could be used by official
> 
> Doxygen generation of Postgres, it would highly increase quality
> 
> of online documentation of Postgres!


I have not actually looked at the code of the filter itself but 
personally I'm not super happy with doing a C-based filter as part of 
our doxygen documentation generation step - that seems like an 
additional maintainance burden infrastrukcture wise - is there nothing 
more lightweight available(assuming we even want that)?




Stefan



Re: Where can I find the doxyfile?

From
Bohdan Mart
Date:
On 09.10.23 11:38, Stefan Kaltenbrunner wrote:
> I have not actually looked at the code of the filter itself but 
> personally I'm not super happy with doing a C-based filter as part of 
> our doxygen documentation generation step - that seems like an 
> additional maintainance burden infrastrukcture wise - is there nothing 
> more lightweight available(assuming we even want that)?

Hello, Stefan.

This indeed can require some maintenace, unfortunately. But I think it 
is worth it! Please see screenshots in attachments. On the left there 
are currently published doxygen, and on right is documentation I have 
generated with John's config. (It doesn't show source, but I believe it 
is discrepancy in config, not issue from filter).

There could be some false positive, like in this code:

/* Functions in access/index/amapi.c */
extern IndexAmRoutine *GetIndexAmRoutine(Oid amhandler);
extern IndexAmRoutine *GetIndexAmRoutineByAmId(Oid amoid, bool noerror);

Doxygen would concat `Functions in access/index/amapi.c` with actual 
docs from .c file, and resulting docs would become:

> GetIndexAmRoutine - call the specified access method handler routine to get its IndexAmRoutine struct, which will be
palloc'din the caller's context.
 
> 
> Functions in access/index/amapi.c.
> 
> Note that if the amhandler function is built-in, this will not involve any catalog access. It's therefore safe to use
thiswhile bootstrapping indexes for the system catalogs. relcache.c relies on that.
 
> 
> Definition at line 33 of file amapi.c.

But that's not big problem, IMHO. Better at least some docs misrendered, 
than no comments in docs.

Other option would be to manually clean up documentation to provide more 
concise comments. I can actually try doing it, few files at a time.
But that would damage git history and git-blame, unless we use
.git-blame-ignore-revs-entry

I have tested, and if we add such commit to ignoreblame, and don't 
change actual line numbers, git-blame works fine.

So I have created draft patches(don't pay atention to commit message in 
patch, it's just draft) with way, how this comemnts can be changed. It 
can be done semi manually, with applying that filter by hands and then 
commiting to git. I am willing to do that, if this option is preferable 
over using filter during build process.


P.S. Editors, such as VsCode and CLion parses comments in any format and 
show them as documentation, so if use standalone tool to view code 
instead of Doxygen it is good experience. But having proper 
documentation in Doxygen would help new people who haven't downloaded 
code and configured IDE yet.

P.S. My original question was to have current doxygen config to be 
published, like it was in 2008.

Regards,
Bohdan Mart.
Attachment

Re: Where can I find the doxyfile?

From
John Morris
Date:

Here’s a first pass at integrating that doxygen “C filter” into the meson build. As a prototype, it “seems to work”.

 

The command “ninja doxygen” generates html files under doc/doxygen/html.

Attachment

Re: Where can I find the doxyfile?

From
Bohdan Mart
Date:
On 12.10.23 09:07, John Morris wrote:
> Here’s a first pass at integrating that doxygen “C filter” into the 
> meson build. As a prototype, it “seems to work”.
> 
> The command “ninja doxygen” generates html files under doc/doxygen/html.

Thanks. I have applied this patch and build (`ninja doxygen`) works.

But this Doxyfile don't generate dependency graph, which is present in 
official doxygen, see screenshot-1.png.

Also it doesn't include sources from `contrib` dir.

I guess, @Stefan, we need original Doxyfile that is used to build 
https://doxygen.postgresql.org/ currently.

I've also run it with ninja and no args, and observed following behavior:
* filter was compiled
* Doxyfile generated from template
* Doxygen generation didn't happen

I believe filter compilation and Doxygen generation shouldn't happen in 
this case as well.
Attachment

Re: Where can I find the doxyfile?

From
John Morris
Date:
Thank you for trying the patch out and commenting on it.  

I'm starting to think of it as a project. Here's a quick project statement.

The purpose is to generate improved  Doxygen output while making maximal use of how Postgres currently does program comments.

Thinking in terms of specific steps, and adding to what you have mentioned:
  • Configure Doxyfile so it produces output similar to previous output.
  • Only build Doxygen output if requested
  • Don't compile the filter or configure the Doxyfile  if they aren't needed
  • Include contrib in the sources to document
  • Provide options for other (non-html) output. (Which ones?)
  • Update Postgres documentation to include instructions for creating Doxygen output.
  • Mention it in developer guidelines and provide sample code showing a "desired" commenting style.
Does that list seem complete? I don't want people to think we're imposing new standards or legislating new commenting styles.  It's more a matter of describing what we currently do, maybe with some minor suggestions for improving.
  • John Morris

Re: Where can I find the doxyfile?

From
Bohdan Mart
Date:
On 14.10.23 21:54, John Morris wrote:
> Thank you for trying the patch out and commenting on it.

Thank you! I even didn't know such filters are possible, and after that 
googling filter for C/C++ didn't gave good results.

> I'm starting to think of it as a project. Here's a quick project statement.

👍

> The purpose is to generate improved  Doxygen output while making maximal 
> use of how Postgres currently does program comments.

Yes, it shouldn't alter existing workflow.

>   * Provide options for other (non-html) output. (Which ones?)

I guess for now it is not needed, there were only HTML doxygen.

>   * Mention it in developer guidelines and provide sample code showing a
>     "desired" commenting style.

I think no changes are needed in guidelines at this point, just allow 
existing comments to be collected.

In future some documentation from Doxygen style can be copied to 
describe how to document function's parameter, etc.

> Does that list seem complete? I don't want people to think we're 
> imposing new standards or legislating new commenting styles.  It's more 
> a matter of describing what we currently do, maybe with some minor 
> suggestions for improving.

Yes, that's all I can think about at first stage, at least. Except last 
bullet is too much. Better to not impose any new standards at this point.
Just get as much comments as possible from existing code-base.

And allow people build it locally.

Later on we can find problematic places where comments that should not 
be pulled are pulled, and add some syntax into filter to exclude such 
comments.

This are places, like section comments, like:
      "all following fields are ***"

But at first step it's just good to have as much as possible comments 
collected, even if some are misplaced.

So done is better than perfect.

----
Best regards,
Bohdan Mart.

Attachment

Including Doxyfile and Meson script for docs into main source tree

From
Bohdan Mart
Date:
Hello, Stefan!

What do you think about this? See quoted message from John Morris.

P.S. Also we would like you to provide currently used Doxyfile, as 
Postgres doxigen's home page no longer have link to download said file.

Regards,
    Bohdan.

On 14.10.23 21:54, John Morris wrote:
> Thank you for trying the patch out and commenting on it.
> 
> I'm starting to think of it as a project. Here's a quick project statement.
> 
> The purpose is to generate improved  Doxygen output while making maximal 
> use of how Postgres currently does program comments.
> 
> Thinking in terms of specific steps, and adding to what you have mentioned:
> 
>   * Configure Doxyfile so it produces output similar to previous output.
>   * Only build Doxygen output if requested
>   * Don't compile the filter or configure the Doxyfile  if they aren't
>     needed
>   * Include contrib in the sources to document
>   * Provide options for other (non-html) output. (Which ones?)
>   * Update Postgres documentation to include instructions for creating
>     Doxygen output.
>   * Mention it in developer guidelines and provide sample code showing a
>     "desired" commenting style.
> 
> Does that list seem complete? I don't want people to think we're 
> imposing new standards or legislating new commenting styles.  It's more 
> a matter of describing what we currently do, maybe with some minor 
> suggestions for improving.
> 
>   * John Morris
> 



Re: Where can I find the doxyfile?

From
John Morris
Date:

Here is an updated patch for generating Doxygen files from existing PostgreSQL source code.

  • “ninja doxygen” builds Doxygen output under doc/doxygen/html.
  • The C filter is only compiled when doxygen output is requested, and compiler warnings are now fixed.
  • “meson configure -Ddoxygen_graphs=true” enables (caller,callee,include) graphs. Generating graphs is slow, so this feature is turned off by default.
  • The output is closer to, but not yet identical to, https://doxygen.postgresql.org.
  • As a personal preference, source code is not included inline in the documentation. Instead, it can be accessed by clicking a link.
  • Added brief mention to the PostgreSQL guide on how to access or create Doxygen documentation.
Attachment

Re: Including Doxyfile and Meson script for docs into main source tree

From
Stefan Kaltenbrunner
Date:
Hi Bohdan!

I will see about making th ecurrent doxygen configuration available as a 
download again in the next few days. Not sure whether there is an 
additional actionable item yet as far as the project from John Morris 
goes wrt. the postgresql.org sysadmin team.

The patch proposed is an enhancement of the build process in core 
postgresql and I think we would look into utilizing that also for the 
"official" build once applied.

I dont think we would want to locally maintain a C-based filter and a 
custom build procedure otherwise.

On the patch itself I'm somewhat unconvinced that it is a good idea or 
long-term maintainable to actually have a kinda random copy of the 
configuration file(!) of an external software (doxygen in this) case in 
our code that might need continous updating and maintainance to keep up 
with new upstream releases.
As it stands the patch right now is against 1.9.4, with 1.9.1 running on 
our installation and 1.9.8 being the latest release...


Stefan


On 30.10.23 14:57, Bohdan Mart wrote:
> Hello, Stefan!
> 
> What do you think about this? See quoted message from John Morris.
> 
> P.S. Also we would like you to provide currently used Doxyfile, as 
> Postgres doxigen's home page no longer have link to download said file.
> 
> Regards,
>      Bohdan.
> 
> On 14.10.23 21:54, John Morris wrote:
>> Thank you for trying the patch out and commenting on it.
>>
>> I'm starting to think of it as a project. Here's a quick project 
>> statement.
>>
>> The purpose is to generate improved  Doxygen output while making 
>> maximal use of how Postgres currently does program comments.
>>
>> Thinking in terms of specific steps, and adding to what you have 
>> mentioned:
>>
>>   * Configure Doxyfile so it produces output similar to previous output.
>>   * Only build Doxygen output if requested
>>   * Don't compile the filter or configure the Doxyfile  if they aren't
>>     needed
>>   * Include contrib in the sources to document
>>   * Provide options for other (non-html) output. (Which ones?)
>>   * Update Postgres documentation to include instructions for creating
>>     Doxygen output.
>>   * Mention it in developer guidelines and provide sample code showing a
>>     "desired" commenting style.
>>
>> Does that list seem complete? I don't want people to think we're 
>> imposing new standards or legislating new commenting styles.  It's 
>> more a matter of describing what we currently do, maybe with some 
>> minor suggestions for improving.
>>
>>   * John Morris
>>




Re: Including Doxyfile and Meson script for docs into main source tree

From
John Morris
Date:

>On the patch itself I'm somewhat unconvinced that it is a good idea or

>long-term maintainable to actually have a kinda random copy of the
>configuration file(!) of an external software (doxygen in this)

 

Rather than copying the Doxyfile, we could just publish the non-default values. As a mostly non-Doxygen user, I found it useful to see the config file in its entirety, but I agree it is a bit much.

Would only showing modified values make more sense?

 

 

 

From: Stefan Kaltenbrunner <stefan@kaltenbrunner.cc>
Date: Thursday, November 2, 2023 at 2:51 AM
To: Bohdan Mart <mart.bogdan@gmail.com>
Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>, 'Bruce Momjian' <bruce@momjian.us>, postgres@coyotebush.net <postgres@coyotebush.net>, John Morris <john.morris@crunchydata.com>
Subject: Re: Including Doxyfile and Meson script for docs into main source tree

Hi Bohdan!

I will see about making th ecurrent doxygen configuration available as a
download again in the next few days. Not sure whether there is an
additional actionable item yet as far as the project from John Morris
goes wrt. the postgresql.org sysadmin team.

The patch proposed is an enhancement of the build process in core
postgresql and I think we would look into utilizing that also for the
"official" build once applied.

I dont think we would want to locally maintain a C-based filter and a
custom build procedure otherwise.

On the patch itself I'm somewhat unconvinced that it is a good idea or
long-term maintainable to actually have a kinda random copy of the
configuration file(!) of an external software (doxygen in this) case in
our code that might need continous updating and maintainance to keep up
with new upstream releases.
As it stands the patch right now is against 1.9.4, with 1.9.1 running on
our installation and 1.9.8 being the latest release...


Stefan


On 30.10.23 14:57, Bohdan Mart wrote:
> Hello, Stefan!
>
> What do you think about this? See quoted message from John Morris.
>
> P.S. Also we would like you to provide currently used Doxyfile, as
> Postgres doxigen's home page no longer have link to download said file.
>
> Regards,
>      Bohdan.
>
> On 14.10.23 21:54, John Morris wrote:
>> Thank you for trying the patch out and commenting on it.
>>
>> I'm starting to think of it as a project. Here's a quick project
>> statement.
>>
>> The purpose is to generate improved  Doxygen output while making
>> maximal use of how Postgres currently does program comments.
>>
>> Thinking in terms of specific steps, and adding to what you have
>> mentioned:
>>
>>   * Configure Doxyfile so it produces output similar to previous output.
>>   * Only build Doxygen output if requested
>>   * Don't compile the filter or configure the Doxyfile  if they aren't
>>     needed
>>   * Include contrib in the sources to document
>>   * Provide options for other (non-html) output. (Which ones?)
>>   * Update Postgres documentation to include instructions for creating
>>     Doxygen output.
>>   * Mention it in developer guidelines and provide sample code showing a
>>     "desired" commenting style.
>>
>> Does that list seem complete? I don't want people to think we're
>> imposing new standards or legislating new commenting styles.  It's
>> more a matter of describing what we currently do, maybe with some
>> minor suggestions for improving.
>>
>>   * John Morris
>>

Re: Where can I find the doxyfile?

From
John Morris
Date:

Another update, this time with an abbreviated Doxyfile. Rather than including the full Doxyfile, this updated version only includes modified settings. It is more compact and more likely to survive across future doxygen versions.

 

  • John Morris
Attachment

Re: Where can I find the doxyfile?

From
vignesh C
Date:
On Tue, 7 Nov 2023 at 12:23, John Morris <john.morris@crunchydata.com> wrote:
>
> Another update, this time with an abbreviated Doxyfile. Rather than including the full Doxyfile, this updated version
onlyincludes modified settings. It is more compact and more likely to survive across future doxygen versions.
 

Meson build is failing in CFbot at [1] and [2] with:
Message: Install doxygen if you wish to create Doxygen documentation
Program dot found: NO
Configuring Doxyfile using configuration

doc/doxygen/meson.build:52:0: ERROR: Tried to use not-found external
program in "command"

[1] - https://cirrus-ci.com/task/5823573617541120
[2] - https://api.cirrus-ci.com/v1/artifact/task/5823573617541120/meson_log/build/meson-logs/meson-log.txt

Regards,
Vignesh



Re: Where can I find the doxyfile?

From
vignesh C
Date:
On Thu, 18 Jan 2024 at 06:39, vignesh C <vignesh21@gmail.com> wrote:
>
> On Tue, 7 Nov 2023 at 12:23, John Morris <john.morris@crunchydata.com> wrote:
> >
> > Another update, this time with an abbreviated Doxyfile. Rather than including the full Doxyfile, this updated
versiononly includes modified settings. It is more compact and more likely to survive across future doxygen versions.
 
>
> Meson build is failing in CFbot at [1] and [2] with:
> Message: Install doxygen if you wish to create Doxygen documentation
> Program dot found: NO
> Configuring Doxyfile using configuration
>
> doc/doxygen/meson.build:52:0: ERROR: Tried to use not-found external
> program in "command"
>

With no update to the thread and the build still failing I'm marking
this as returned with feedback.  Please feel free to resubmit to the
next CF when there is a new version of the patch.

Regards,
Vignesh



Re: Where can I find the doxyfile?

From
John Morris
Date:

Here is the updated patch. It should fix the meson issue when no doxygen is present.

Attachment

Re: Where can I find the doxyfile?

From
Alvaro Herrera
Date:
On 2024-Feb-02, John Morris wrote:

> Here is the updated patch. It should fix the meson issue when no
> doxygen is present.

I wish you'd stop proposing the lex filter so that we can discuss the
Doxyfile.in file and the accompanying Meson rule.

I think there's pretty much zero chance that we'd accept the
doxy_filter.l thingy in our source tree.  It seems something you want to
keep for your personal use.  I think this filter is the kind of genious
idea that it's OK if you can do it at home, but having something that
purports to display our source code and then show something that is
*not* our source tree sounds insane.  Who knows what artifacts are going
to show up in the doxygen output, and then we'll be on the hook to fix
those.  I propose to add the file to the wiki, maybe link to it from the
Developer_FAQ page.

Also, I'm not sure about the use case of people who wants to study the
Postgres source code but doesn't have time to download it into VSCode or
whatever.

In short: -1 from me.

I see the Doxy thing as roughly parallel to the coverage build rules ...
but the coverage rules seem more much closely intertwined with the tree
in a way that the Doxygen processing is not.  Speaking from my personal
point of view, I make very little use of our Doxygen pages, but I do use
them --- and very occassionally I would like to see what the changes
would be with some patch applied.  However, again, maybe having the .in
file in the wiki is enough, where you can download it and have your own
script to run it, and tweak the prefs however you want them and so on?

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/



Re: Where can I find the doxyfile?

From
John Morris
Date:

>> I wish you'd stop proposing the lex filter so that we can discuss the
>> Doxyfile.in file and the accompanying Meson rule.


I see no obstacle to discussing Doxyfile.in and the meson.build file. Your suggestions are welcome. If you can find the original Doxyfile, it

would make sense to incorporate it.

>> but having something that purports to display our source code and
>> then show something that is*not* our source tree sounds insane.

 

The filter enables doxygen to interpret Postgres comments. Any code displayed is the original code, not the filtered code.

 

>> Speaking from my personal point of view, I make very little use

>> of our Doxygen pages,

 

Same for me. The pages are missing critical information, such as function descriptions and descriptions of structure fields. This filter is an attempt at fixing those problems.

 

Of course, nothing will fix doxygen when there are no comments to begin with. If we want comprehensive doxygen output, we need to add comments. However, we don’t need to add those irritating extra symbols.

 

 

Re: Where can I find the doxyfile?

From
John Morris
Date:

>> It seems something you want to keep for your personal use. 

>> I think this filter is the kind of genious idea that it's OK if you

>> can do it at home

 

I don’t agree with your characterization.

 

The purpose of the filter is to bring existing Postgres comments into the doxygen output. While I haven’t done a full survey, the majority of Postgres code has comments describing functions, globals, macros and structure fields.

 

Currently, those comments are thrown away. They do not appear in the existing Doxygen output.

 

The filter itself is not rocket science. It is a straightforward scanner built using the flex tool. I understand many people are more comfortable with perl or python, but flex is uniquely appropriate for creating small, efficient scanners.

 

The resulting output is not perfect. However, it is a major step forward from what we are currently producing. Over time, we can gradually update Postgres comments to fill in missing information. Eventually, doxygen output could become a “first look” tool for looking at code.

 

Will it be my “first look” tool. Not likely. Like you, I am comfortable with Intellij and VsCode. I have my build environments already set up.

 

I do not see this as a high-risk endeavor. It is “off to the side” of the main Postgres code. It takes an existing tool, doxygen, which is mostly useless, and starts to make it usable.

 

As an aside, I have contacted Intellij about allowing custom Doxygen files for hover information. No reply yet, but it would allow us to tailor our IDEs to display the information we need most.

 

 

Re: Where can I find the doxyfile?

From
Peter Eisentraut
Date:
On 05.02.24 18:29, John Morris wrote:
> The purpose of the filter is to bring existing Postgres comments into 
> the doxygen output. While I haven’t done a full survey, the majority of 
> Postgres code has comments describing functions, globals, macros and 
> structure fields.
> 
> Currently, those comments are thrown away. They do not appear in the 
> existing Doxygen output.

Maybe this is something that can be tweaked on the doxygen side?

For example, clangd can also process doxygen-style comments.  But it can 
also process non-decorated comments, because it knows that the comment 
just before a declaration is probably the comment describing the thing. 
Maybe doxygen could have that functionality as well.




Re: Where can I find the doxyfile?

From
Peter Eisentraut
Date:
On 02.02.24 01:19, John Morris wrote:
> Here is the updated patch. It should fix the meson issue when no doxygen 
> is present.

I think all the explanatory messages in doc/doxygen/meson.build are a 
bit much.  I think it's enough to just not define the target when the 
required conditions (dependencies, options) are not there.  Maybe 
something like docs_pdf can serve as an example.




Re: Where can I find the doxyfile?

From
John Morris
Date:

>> I think all the explanatory messages in doc/doxygen/meson.build
>> are a bit much.  I think it's enough to just not define the target
>> when the required conditions (dependencies, options) are not there. 
>> Maybe something like docs_pdf can serve as an example.

Makes sense, and it is simpler.

Let’s continue after I take a look at docs_pdf.

 

Thanks,

    John

 

Re: Where can I find the doxyfile?

From
John Morris
Date:

>> Maybe this is something that can be tweaked on the doxygen side?

 

I’ll look into that idea. Doxygen is a separate project with its own developers to persuade. I could imagine a special “C” or “C++” mode. I wanted to keep things simple, and the filter mechanism is how they extend doxygen to support other languages.

 

One alternative is to add the filter to doxygen’s webpage. They currently list a number of filters, none of which address this particular issue. The filter would become a build dependency like doxygen itself.

 

In the meantime, I’d like to address Alvaro’s concern about generating artifacts. It’s a bit tedious, but a visual survey of the output may add confidence or show areas needing improvement.

Re: Where can I find the doxyfile?

From
John Morris
Date:

>> I think all the explanatory messages in doc/doxygen/meson.build
>> are a bit much.  I think it's enough to just not define the target

 

Here is a patch with an updated meson.build as you suggested. I agree the messages were a bit much.
On the other hand, I would like to see clear error messages when dot or doxygen are not installed,
but I’ll leave that for a future discussion.

 

  • John

---------------------------------------------------------------------

Attachment

Re: Where can I find the doxyfile?

From
Andres Freund
Date:
Hi,

On 2024-02-08 18:30:01 +0000, John Morris wrote:
> +FILTER_PATTERNS        = *.c *.h

We also have a handful of .cpp files.

> +/*
> + * Custom banner character to be removed from comments.
> + * We'll hardcode it to suit postgreSQL, but it should be set through a command line arg.
> + */
> +char customBanner = '-';
> +
> +/*************************************************************************************************
> +A simple program which reads a file, updates the comments, and writes to stdout.
> +This is intended to be used as a doxygen filter, converting existing comments to doxygen comments.
> +**********************************************************************************************/
> +int main(int argc, char**argv) {
> +
> +    // Verify we have a single argument.

This file isn't really following postgres coding style - any reason not to do
so?



> diff --git a/doc/doxygen/meson.build b/doc/doxygen/meson.build
> new file mode 100644
> index 0000000000..e5539b7854
> --- /dev/null
> +++ b/doc/doxygen/meson.build
> @@ -0,0 +1,73 @@
> +# Generate doxygen pages for PostgreSQL using "ninja doxygen"
> +#
> +# Doxygen pages are optional. Nothing in this script should
> +# cause PostgreSQL builds to fail.
> +#
> +# Currently there are no explicit error messages
> +#   - If doxygen is not found, the doxygen target will not be defined.
> +#   - If dot is not found, no graphs will be generated.
> +#   - flex is already required, so we don't check for it.
> +#
> +# As a future enhancement, display meaningful error messages
> +# when doxygen or dot are not found. Meson does not
> +# support build time error messages, but they can be displayed
> +# using a python custom target.

What do you mean with "build time error messages" specifically? Why would we
want to output anything at build time (rather than configure time)?


> +# Find the doxygen command. If not found, stop and don't define the target.
> +doxygen_cmd = find_program('doxygen', required: false)

I'd add "native: true", it'd not work to find the program in the target
environment of a cross build.

> +
> +# Find the dot command. If not found, no graphs will be generated.
> +dot_cmd = find_program('dot', required: false)

Dito.


Greetings,

Andres Freund



Re: Where can I find the doxyfile?

From
John Morris
Date:

>> We also have a handful of .cpp files.

Fortunately, our .cpp files are simple and do not include C++ specific features.  
We shouldn’t have any problem adding .cpp to the filter list.

 

At some point it will be necessary to support general C++, but I wasn’t planning to do it yet.


>> This file isn't really following postgres coding style - any reason not to do so?

I’ll update it to Postgres commenting. The filter started as a standalone project.


>> What do you mean with "build time error messages" specifically? Why would we
>> want to output anything at build time (rather than configure time)?

“build time error messages” occur after typing “ninja”.

 

Documentation is usually produced long after meson setup messages have been forgotten.
It is much more useful to see “Please install the doxygen program” after typing “ninja doxygen”
than to see “target not supported”, or to have to look through the setup log.

 

It’s personal preference, but I think the sgml docs should display a similar message.


>> I'd add "native: true", it'd not work to find the program in the target
>> environment of a cross build.
Got it. Thanks.

Re: Where can I find the doxyfile?

From
John Morris
Date:

Update:  another patch with 1) suggested changes, 2) delete old html before generating new, and 3) added flex names for the more complex regular expressions.

 

Exercising the “ninja doxygen” command, brought up some issues.

  1. The generated html directory contained old as well as new pages.
    The build script now deletes old pages before building new ones.
  2. Using a line of dashes to suppress text formatting is not implemented.
    Some dashed comments are harder to read, especially around code samples.
  3. Comments at the end of “#define” are considered part of the define
    and not annotated as doxygen comments.

 

The first issue was very confusing, so it has been fixed.
My preference is to make the other two issues part of a future enhancement. Thoughts?

 

I’ve also been scanning the generated pages looking for anomalies.

  1. About 1/3 of pages examined.
  2. In most cases, the filter works as expected.
  3. Many places are missing information, so the output has blank fields (as expected).
  4. A few places have obvious nonsense. (eg. a group comment applied to one element)
  5. No situations where the output would be misleading.
  6. In all cases, the source code is “a click away”.

 

While I had planned to look at *every* page, I’ll probably stop at the 1/3 sample – unless someone wants to help scan through the pages with me.

 

I also heard back from Jetbrains about incorporating custom Doxyfiles. They do their own rendering and do not invoke the doxygen command. Custom Doxyfiles are not going to happen. (It’s probably the same for VS.)

 

On my Mac M1, generating doxygen takes about 20 seconds. When graphs are added, it takes 5 minutes.

 

 

Attachment

Re: Where can I find the doxyfile?

From
Bruce Momjian
Date:
On Mon, Feb  5, 2024 at 05:29:08PM +0000, John Morris wrote:
> >> It seems something you want to keep for your personal use. 
> 
> >> I think this filter is the kind of genious idea that it's OK if you
> 
> >> can do it at home
> 
> I don’t agree with your characterization.
> 
> The purpose of the filter is to bring existing Postgres comments into the
> doxygen output. While I haven’t done a full survey, the majority of Postgres
> code has comments describing functions, globals, macros and structure fields.
> 
> Currently, those comments are thrown away. They do not appear in the existing
> Doxygen output.

I have found it very strange that a tool like doxygen which can create
all sorts of call graphs, just ignores some comments.  The comments
above function are very important.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.



Re: Where can I find the doxyfile?

From
John Morris
Date:

>> I have found it very strange that a tool like doxygen which can create
>> all sorts of call graphs, just ignores some comments.  The comments
>> above function are very important.

 

I agree with you . I hated doxygen for decades because of the irritating annotations it required.  When I discovered IDEs were creating doxygen-like popups from conventional comments, I tried to figure out what they were doing. In the process, I discovered filters, and I created a filter to match what I thought the IDEs were doing.

 

(As it turns out, IDEs implement their own rendering independent of doxygen.)

 

I find it ironic I’ve gone from being a long-time hater of doxygen to being its advocate.

 

  • John Morris

 

Tiny tidbit of history. Back in the 70’s, I created a comment extractor for the language Ratfor. We used it to maintain an alphabetical index of functions and to display pseudo-code. We drew our call graphs by hand and saved them in manila folders. Hard to imagine now.