Thread: Where can I find the doxyfile?
Hi, I don't know where I can find the doxyfile which generate "doxygen.postgresql.org" web site. I found that when reading code the doxygen source code is quite helpful. However, I want to generate an off-line copy of doxygen docs myself, but I can't find the doxyfile in the lastest source release. Thanks!
Xin Wang napsal(a): > Hi, > I don't know where I can find the doxyfile which generate > "doxygen.postgresql.org" web site. I found that when reading code the > doxygen source code is quite helpful. However, I want to generate an > off-line copy of doxygen docs myself, but I can't find the doxyfile in > the lastest source release. I think it is good idea. Stefan, what's about put it on the wiki? Zdenek
Zdenek Kotala wrote: > Xin Wang napsal(a): >> Hi, >> I don't know where I can find the doxyfile which generate >> "doxygen.postgresql.org" web site. I found that when reading code the >> doxygen source code is quite helpful. However, I want to generate an >> off-line copy of doxygen docs myself, but I can't find the doxyfile in >> the lastest source release. > > I think it is good idea. Stefan, what's about put it on the wiki? don't think the wiki is a good place (we would have to maintain it in addition to the main file) so I simply added a link on http:/doxygen.postgresql.org that contains the current copy of the configuration file that was used to generate a particular set of docs. However some of the things there are specific to our install so a bit (like some of the paths) of manual change will be required anyway. Stefan
On 2008-06-02 18:33:42 Stefan Kaltenbrunner wrote: > > Zdenek Kotala wrote: > > Xin Wang napsal(a): > >> Hi, > >> I don't know where I can find the doxyfile which generate > >> "doxygen.postgresql.org" web site. I found that when reading code the > >> doxygen source code is quite helpful. However, I want to generate an > >> off-line copy of doxygen docs myself, but I can't find the doxyfile in > >> the lastest source release. > > > > I think it is good idea. Stefan, what's about put it on the wiki? > > don't think the wiki is a good place (we would have to maintain it in > addition to the main file) so I simply added a link on > http:/doxygen.postgresql.org that contains the current copy of the > configuration file that was used to generate a particular set of docs. > However some of the things there are specific to our install so a bit > (like some of the paths) of manual change will be required anyway. Hello, sorry for resurrecting this old discussion, and necroposting, but this was relevant. It turns out that way back in 2008 it was possible to download Doxyfile from https://doxygen.postgresql.org/ , which can be seen at: https://web.archive.org/web/20080915132924/https://doxygen.postgresql.org/ I was able to download this Doxyfile from the web archive, and it actually worked for me, after I changed the path to postgres sources and removed header template. It even produced same result as on current web-site. It would be nice if this doxygen file would be placed somewhere on the current site. P.S. I've tried to run doxygen on postgres sources without config, and it didn't find any source file, this archived thread helped. Perhaps doxygen file can be just placed into postgres git repository? Best regards, Bohdan Mart.
On Fri, Oct 6, 2023 at 10:50:25PM +0300, Bogdan Mart wrote: > On 2008-06-02 18:33:42 Stefan Kaltenbrunner wrote: > > > > Zdenek Kotala wrote: > > > Xin Wang napsal(a): > > >> Hi, > > >> I don't know where I can find the doxyfile which generate > > >> "doxygen.postgresql.org" web site. I found that when reading code the > > >> doxygen source code is quite helpful. However, I want to generate an > > >> off-line copy of doxygen docs myself, but I can't find the doxyfile in > > >> the lastest source release. > > > > > > I think it is good idea. Stefan, what's about put it on the wiki? > > > > don't think the wiki is a good place (we would have to maintain it in > > addition to the main file) so I simply added a link on > > http:/doxygen.postgresql.org that contains the current copy of the > > configuration file that was used to generate a particular set of docs. > > However some of the things there are specific to our install so a bit > > (like some of the paths) of manual change will be required anyway. > > Hello, sorry for resurrecting this old discussion, and necroposting, > but this was relevant. > > It turns out that way back in 2008 it was possible to download Doxyfile > from https://doxygen.postgresql.org/ , which can be seen at: > https://web.archive.org/web/20080915132924/https://doxygen.postgresql.org/ > > I was able to download this Doxyfile from the web archive, and it actually > worked for me, after I changed the path to postgres sources and removed > header template. It even produced same result as on current web-site. > > It would be nice if this doxygen file would be placed somewhere > on the current site. > > P.S. I've tried to run doxygen on postgres sources without config, > and it didn't find any source file, this archived thread helped. > > Perhaps doxygen file can be just placed into postgres git repository? We do link to a Doxygen build from our website: https://www.postgresql.org/developer/coding/ -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Only you can decide what is important to you.
On 07.10.23 00:06, Bruce Momjian wrote: > On Fri, Oct 6, 2023 at 10:50:25PM +0300, Bogdan Mart wrote: > > On 2008-06-02 18:33:42 Stefan Kaltenbrunner wrote: > >> > >> Zdenek Kotala wrote: > >>> Xin Wang napsal(a): > >>>> Hi, > >>>> I don't know where I can find the doxyfile which generate > >>>> "doxygen.postgresql.org" web site. I found that when reading code the > >>>> doxygen source code is quite helpful. However, I want to generate an > >>>> off-line copy of doxygen docs myself, but I can't find the doxyfile in > >>>> the lastest source release. > >>> > >>> I think it is good idea. Stefan, what's about put it on the wiki? > >> > >> don't think the wiki is a good place (we would have to maintain it in > >> addition to the main file) so I simply added a link on > >> http:/doxygen.postgresql.org that contains the current copy of the > >> configuration file that was used to generate a particular set of docs. > >> However some of the things there are specific to our install so a bit > >> (like some of the paths) of manual change will be required anyway. > > > > Hello, sorry for resurrecting this old discussion, and necroposting, > > but this was relevant. > > > > It turns out that way back in 2008 it was possible to download Doxyfile > > from https://doxygen.postgresql.org/ , which can be seen at: > > https://web.archive.org/web/20080915132924/https://doxygen.postgresql.org/ > > > > I was able to download this Doxyfile from the web archive, and it actually > > worked for me, after I changed the path to postgres sources and removed > > header template. It even produced same result as on current web-site. > > > > It would be nice if this doxygen file would be placed somewhere > > on the current site. > > > > P.S. I've tried to run doxygen on postgres sources without config, > > and it didn't find any source file, this archived thread helped. > > > > Perhaps doxygen file can be just placed into postgres git repository? > > We do link to a Doxygen build from our website: > > https://www.postgresql.org/developer/coding/ Yes, there is link to deployed documentation generated by Doxygen (unless I'm missing something), but I am talking about config file for Doxygen to generate same documentation locally. There are no entries on Wiki regarding doxygen, and not in build instructions. I was able to locate needed config file in Web Archive: https://web.archive.org/web/20081210081808if_/http://doxygen.postgresql.org:80/pgsql.conf But it would be nice if it was readily available if anybody other would like to build docs locally.
Sometime earlier, I created a filter to annotate regular C comments as doxy comments. I'll attach it along with a sample doxyfile for running it. Just in case it looks useful. I've never been a big fan of Doxygen, but it seems to have gotten better over time. Now that some IDEs display doxy comments on hover, I'm beginning to appreciate it.
Attachment
Sometime earlier, I created a filter to annotate regular C comments as doxy comments. I'll attach it along with a sample doxyfile for running it. Just in case it looks useful. I've never been a big fan of Doxygen, but it seems to have gotten better over time. Now that some editors are displaying doxygen comments on hover, I'm beginning to appreciate it. d
Attachment
On 07.10.23 00:29, postgres@coyotebush.net wrote: > Sometime earlier, I created a filter to annotate regular C comments as doxy > comments. I'll attach it along with a sample doxyfile for running it. Just > in case it looks useful. > > I've never been a big fan of Doxygen, but it seems to have gotten better > over time. Now that some IDEs display doxy comments on hover, I'm beginning > to appreciate it. Thank you for providing this `flex` filter! It is actually useful. I've tested it on one file from postures source base, and it actually converted regular comments to doc-comments! If this filter could be used by official Doxygen generation of Postgres, it would highly increase quality of online documentation of Postgres! My initial idea was to create patches to upstream with changing comments to documented comments, because current online dock is lacking comments, but with this filter it would be unneeded, as documentation would be able to be generated from current sources! To illustrate the point: ~~~~~~~~~~~ This function has Doxy style comment ``` src/interfaces/ecpg/compatlib/informix.c 672:/** 673- * initialize the struct, which holds the different forms 674- * of the long value 675- */ 676-static int 677-initValue(long lng_val) ``` And it is rendered properly: https://doxygen.postgresql.org/informix_8c.html#a0dad4c2ee52a831d3aa1bf1133e0e17d But function like this ``` src/backend/foreign/foreign.c /* * get_foreign_server_oid - given a server name, look up the OID * * If missing_ok is false, throw an error if name not found. If true, just * return InvalidOid. */ Oid get_foreign_server_oid(const char *servername, bool missing_ok) ``` https://doxygen.postgresql.org/foreign_8c.html#a7959c56969be440c25b62c3c98ce2a78 doesn't have rendered documentation. P.S. Was this original message from postgres@coyotebush.net intended to be sent on John's Morris behalf?
On 07.10.23 00:23, Bohdan Mart wrote: > > On 07.10.23 00:29, postgres@coyotebush.net wrote: >> Sometime earlier, I created a filter to annotate regular C comments as >> doxy >> comments. I'll attach it along with a sample doxyfile for running >> it. Just >> in case it looks useful. >> >> I've never been a big fan of Doxygen, but it seems to have gotten better >> over time. Now that some IDEs display doxy comments on hover, I'm >> beginning >> to appreciate it. > > Thank you for providing this `flex` filter! It is actually useful. I've > tested it on one > > file from postures source base, and it actually converted regular > > comments to doc-comments! If this filter could be used by official > > Doxygen generation of Postgres, it would highly increase quality > > of online documentation of Postgres! I have not actually looked at the code of the filter itself but personally I'm not super happy with doing a C-based filter as part of our doxygen documentation generation step - that seems like an additional maintainance burden infrastrukcture wise - is there nothing more lightweight available(assuming we even want that)? Stefan
On 09.10.23 11:38, Stefan Kaltenbrunner wrote: > I have not actually looked at the code of the filter itself but > personally I'm not super happy with doing a C-based filter as part of > our doxygen documentation generation step - that seems like an > additional maintainance burden infrastrukcture wise - is there nothing > more lightweight available(assuming we even want that)? Hello, Stefan. This indeed can require some maintenace, unfortunately. But I think it is worth it! Please see screenshots in attachments. On the left there are currently published doxygen, and on right is documentation I have generated with John's config. (It doesn't show source, but I believe it is discrepancy in config, not issue from filter). There could be some false positive, like in this code: /* Functions in access/index/amapi.c */ extern IndexAmRoutine *GetIndexAmRoutine(Oid amhandler); extern IndexAmRoutine *GetIndexAmRoutineByAmId(Oid amoid, bool noerror); Doxygen would concat `Functions in access/index/amapi.c` with actual docs from .c file, and resulting docs would become: > GetIndexAmRoutine - call the specified access method handler routine to get its IndexAmRoutine struct, which will be palloc'din the caller's context. > > Functions in access/index/amapi.c. > > Note that if the amhandler function is built-in, this will not involve any catalog access. It's therefore safe to use thiswhile bootstrapping indexes for the system catalogs. relcache.c relies on that. > > Definition at line 33 of file amapi.c. But that's not big problem, IMHO. Better at least some docs misrendered, than no comments in docs. Other option would be to manually clean up documentation to provide more concise comments. I can actually try doing it, few files at a time. But that would damage git history and git-blame, unless we use .git-blame-ignore-revs-entry I have tested, and if we add such commit to ignoreblame, and don't change actual line numbers, git-blame works fine. So I have created draft patches(don't pay atention to commit message in patch, it's just draft) with way, how this comemnts can be changed. It can be done semi manually, with applying that filter by hands and then commiting to git. I am willing to do that, if this option is preferable over using filter during build process. P.S. Editors, such as VsCode and CLion parses comments in any format and show them as documentation, so if use standalone tool to view code instead of Doxygen it is good experience. But having proper documentation in Doxygen would help new people who haven't downloaded code and configured IDE yet. P.S. My original question was to have current doxygen config to be published, like it was in 2008. Regards, Bohdan Mart.
Attachment
Here’s a first pass at integrating that doxygen “C filter” into the meson build. As a prototype, it “seems to work”.
The command “ninja doxygen” generates html files under doc/doxygen/html.
Attachment
On 12.10.23 09:07, John Morris wrote: > Here’s a first pass at integrating that doxygen “C filter” into the > meson build. As a prototype, it “seems to work”. > > The command “ninja doxygen” generates html files under doc/doxygen/html. Thanks. I have applied this patch and build (`ninja doxygen`) works. But this Doxyfile don't generate dependency graph, which is present in official doxygen, see screenshot-1.png. Also it doesn't include sources from `contrib` dir. I guess, @Stefan, we need original Doxyfile that is used to build https://doxygen.postgresql.org/ currently. I've also run it with ninja and no args, and observed following behavior: * filter was compiled * Doxyfile generated from template * Doxygen generation didn't happen I believe filter compilation and Doxygen generation shouldn't happen in this case as well.
Attachment
- Configure Doxyfile so it produces output similar to previous output.
- Only build Doxygen output if requested
- Don't compile the filter or configure the Doxyfile if they aren't needed
- Include contrib in the sources to document
- Provide options for other (non-html) output. (Which ones?)
- Update Postgres documentation to include instructions for creating Doxygen output.
- Mention it in developer guidelines and provide sample code showing a "desired" commenting style.
- John Morris
On 14.10.23 21:54, John Morris wrote: > Thank you for trying the patch out and commenting on it. Thank you! I even didn't know such filters are possible, and after that googling filter for C/C++ didn't gave good results. > I'm starting to think of it as a project. Here's a quick project statement. 👍 > The purpose is to generate improved Doxygen output while making maximal > use of how Postgres currently does program comments. Yes, it shouldn't alter existing workflow. > * Provide options for other (non-html) output. (Which ones?) I guess for now it is not needed, there were only HTML doxygen. > * Mention it in developer guidelines and provide sample code showing a > "desired" commenting style. I think no changes are needed in guidelines at this point, just allow existing comments to be collected. In future some documentation from Doxygen style can be copied to describe how to document function's parameter, etc. > Does that list seem complete? I don't want people to think we're > imposing new standards or legislating new commenting styles. It's more > a matter of describing what we currently do, maybe with some minor > suggestions for improving. Yes, that's all I can think about at first stage, at least. Except last bullet is too much. Better to not impose any new standards at this point. Just get as much comments as possible from existing code-base. And allow people build it locally. Later on we can find problematic places where comments that should not be pulled are pulled, and add some syntax into filter to exclude such comments. This are places, like section comments, like: "all following fields are ***" But at first step it's just good to have as much as possible comments collected, even if some are misplaced. So done is better than perfect. ---- Best regards, Bohdan Mart.
Attachment
Hello, Stefan! What do you think about this? See quoted message from John Morris. P.S. Also we would like you to provide currently used Doxyfile, as Postgres doxigen's home page no longer have link to download said file. Regards, Bohdan. On 14.10.23 21:54, John Morris wrote: > Thank you for trying the patch out and commenting on it. > > I'm starting to think of it as a project. Here's a quick project statement. > > The purpose is to generate improved Doxygen output while making maximal > use of how Postgres currently does program comments. > > Thinking in terms of specific steps, and adding to what you have mentioned: > > * Configure Doxyfile so it produces output similar to previous output. > * Only build Doxygen output if requested > * Don't compile the filter or configure the Doxyfile if they aren't > needed > * Include contrib in the sources to document > * Provide options for other (non-html) output. (Which ones?) > * Update Postgres documentation to include instructions for creating > Doxygen output. > * Mention it in developer guidelines and provide sample code showing a > "desired" commenting style. > > Does that list seem complete? I don't want people to think we're > imposing new standards or legislating new commenting styles. It's more > a matter of describing what we currently do, maybe with some minor > suggestions for improving. > > * John Morris >
Here is an updated patch for generating Doxygen files from existing PostgreSQL source code.
- “ninja doxygen” builds Doxygen output under doc/doxygen/html.
- The C filter is only compiled when doxygen output is requested, and compiler warnings are now fixed.
- “meson configure -Ddoxygen_graphs=true” enables (caller,callee,include) graphs. Generating graphs is slow, so this feature is turned off by default.
- The output is closer to, but not yet identical to, https://doxygen.postgresql.org.
- As a personal preference, source code is not included inline in the documentation. Instead, it can be accessed by clicking a link.
- Added brief mention to the PostgreSQL guide on how to access or create Doxygen documentation.
Attachment
Re: Including Doxyfile and Meson script for docs into main source tree
Hi Bohdan! I will see about making th ecurrent doxygen configuration available as a download again in the next few days. Not sure whether there is an additional actionable item yet as far as the project from John Morris goes wrt. the postgresql.org sysadmin team. The patch proposed is an enhancement of the build process in core postgresql and I think we would look into utilizing that also for the "official" build once applied. I dont think we would want to locally maintain a C-based filter and a custom build procedure otherwise. On the patch itself I'm somewhat unconvinced that it is a good idea or long-term maintainable to actually have a kinda random copy of the configuration file(!) of an external software (doxygen in this) case in our code that might need continous updating and maintainance to keep up with new upstream releases. As it stands the patch right now is against 1.9.4, with 1.9.1 running on our installation and 1.9.8 being the latest release... Stefan On 30.10.23 14:57, Bohdan Mart wrote: > Hello, Stefan! > > What do you think about this? See quoted message from John Morris. > > P.S. Also we would like you to provide currently used Doxyfile, as > Postgres doxigen's home page no longer have link to download said file. > > Regards, > Bohdan. > > On 14.10.23 21:54, John Morris wrote: >> Thank you for trying the patch out and commenting on it. >> >> I'm starting to think of it as a project. Here's a quick project >> statement. >> >> The purpose is to generate improved Doxygen output while making >> maximal use of how Postgres currently does program comments. >> >> Thinking in terms of specific steps, and adding to what you have >> mentioned: >> >> * Configure Doxyfile so it produces output similar to previous output. >> * Only build Doxygen output if requested >> * Don't compile the filter or configure the Doxyfile if they aren't >> needed >> * Include contrib in the sources to document >> * Provide options for other (non-html) output. (Which ones?) >> * Update Postgres documentation to include instructions for creating >> Doxygen output. >> * Mention it in developer guidelines and provide sample code showing a >> "desired" commenting style. >> >> Does that list seem complete? I don't want people to think we're >> imposing new standards or legislating new commenting styles. It's >> more a matter of describing what we currently do, maybe with some >> minor suggestions for improving. >> >> * John Morris >>
>On the patch itself I'm somewhat unconvinced that it is a good idea or
>long-term maintainable to actually have a kinda random copy of the
>configuration file(!) of an external software (doxygen in this)
Rather than copying the Doxyfile, we could just publish the non-default values. As a mostly non-Doxygen user, I found it useful to see the config file in its entirety, but I agree it is a bit much.
Would only showing modified values make more sense?
From: Stefan Kaltenbrunner <stefan@kaltenbrunner.cc>
Date: Thursday, November 2, 2023 at 2:51 AM
To: Bohdan Mart <mart.bogdan@gmail.com>
Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>, 'Bruce Momjian' <bruce@momjian.us>, postgres@coyotebush.net <postgres@coyotebush.net>, John Morris <john.morris@crunchydata.com>
Subject: Re: Including Doxyfile and Meson script for docs into main source tree
Hi Bohdan!
I will see about making th ecurrent doxygen configuration available as a
download again in the next few days. Not sure whether there is an
additional actionable item yet as far as the project from John Morris
goes wrt. the postgresql.org sysadmin team.
The patch proposed is an enhancement of the build process in core
postgresql and I think we would look into utilizing that also for the
"official" build once applied.
I dont think we would want to locally maintain a C-based filter and a
custom build procedure otherwise.
On the patch itself I'm somewhat unconvinced that it is a good idea or
long-term maintainable to actually have a kinda random copy of the
configuration file(!) of an external software (doxygen in this) case in
our code that might need continous updating and maintainance to keep up
with new upstream releases.
As it stands the patch right now is against 1.9.4, with 1.9.1 running on
our installation and 1.9.8 being the latest release...
Stefan
On 30.10.23 14:57, Bohdan Mart wrote:
> Hello, Stefan!
>
> What do you think about this? See quoted message from John Morris.
>
> P.S. Also we would like you to provide currently used Doxyfile, as
> Postgres doxigen's home page no longer have link to download said file.
>
> Regards,
> Bohdan.
>
> On 14.10.23 21:54, John Morris wrote:
>> Thank you for trying the patch out and commenting on it.
>>
>> I'm starting to think of it as a project. Here's a quick project
>> statement.
>>
>> The purpose is to generate improved Doxygen output while making
>> maximal use of how Postgres currently does program comments.
>>
>> Thinking in terms of specific steps, and adding to what you have
>> mentioned:
>>
>> * Configure Doxyfile so it produces output similar to previous output.
>> * Only build Doxygen output if requested
>> * Don't compile the filter or configure the Doxyfile if they aren't
>> needed
>> * Include contrib in the sources to document
>> * Provide options for other (non-html) output. (Which ones?)
>> * Update Postgres documentation to include instructions for creating
>> Doxygen output.
>> * Mention it in developer guidelines and provide sample code showing a
>> "desired" commenting style.
>>
>> Does that list seem complete? I don't want people to think we're
>> imposing new standards or legislating new commenting styles. It's
>> more a matter of describing what we currently do, maybe with some
>> minor suggestions for improving.
>>
>> * John Morris
>>
Another update, this time with an abbreviated Doxyfile. Rather than including the full Doxyfile, this updated version only includes modified settings. It is more compact and more likely to survive across future doxygen versions.
- John Morris
Attachment
On Tue, 7 Nov 2023 at 12:23, John Morris <john.morris@crunchydata.com> wrote: > > Another update, this time with an abbreviated Doxyfile. Rather than including the full Doxyfile, this updated version onlyincludes modified settings. It is more compact and more likely to survive across future doxygen versions. Meson build is failing in CFbot at [1] and [2] with: Message: Install doxygen if you wish to create Doxygen documentation Program dot found: NO Configuring Doxyfile using configuration doc/doxygen/meson.build:52:0: ERROR: Tried to use not-found external program in "command" [1] - https://cirrus-ci.com/task/5823573617541120 [2] - https://api.cirrus-ci.com/v1/artifact/task/5823573617541120/meson_log/build/meson-logs/meson-log.txt Regards, Vignesh
On Thu, 18 Jan 2024 at 06:39, vignesh C <vignesh21@gmail.com> wrote: > > On Tue, 7 Nov 2023 at 12:23, John Morris <john.morris@crunchydata.com> wrote: > > > > Another update, this time with an abbreviated Doxyfile. Rather than including the full Doxyfile, this updated versiononly includes modified settings. It is more compact and more likely to survive across future doxygen versions. > > Meson build is failing in CFbot at [1] and [2] with: > Message: Install doxygen if you wish to create Doxygen documentation > Program dot found: NO > Configuring Doxyfile using configuration > > doc/doxygen/meson.build:52:0: ERROR: Tried to use not-found external > program in "command" > With no update to the thread and the build still failing I'm marking this as returned with feedback. Please feel free to resubmit to the next CF when there is a new version of the patch. Regards, Vignesh
Here is the updated patch. It should fix the meson issue when no doxygen is present.
Attachment
On 2024-Feb-02, John Morris wrote: > Here is the updated patch. It should fix the meson issue when no > doxygen is present. I wish you'd stop proposing the lex filter so that we can discuss the Doxyfile.in file and the accompanying Meson rule. I think there's pretty much zero chance that we'd accept the doxy_filter.l thingy in our source tree. It seems something you want to keep for your personal use. I think this filter is the kind of genious idea that it's OK if you can do it at home, but having something that purports to display our source code and then show something that is *not* our source tree sounds insane. Who knows what artifacts are going to show up in the doxygen output, and then we'll be on the hook to fix those. I propose to add the file to the wiki, maybe link to it from the Developer_FAQ page. Also, I'm not sure about the use case of people who wants to study the Postgres source code but doesn't have time to download it into VSCode or whatever. In short: -1 from me. I see the Doxy thing as roughly parallel to the coverage build rules ... but the coverage rules seem more much closely intertwined with the tree in a way that the Doxygen processing is not. Speaking from my personal point of view, I make very little use of our Doxygen pages, but I do use them --- and very occassionally I would like to see what the changes would be with some patch applied. However, again, maybe having the .in file in the wiki is enough, where you can download it and have your own script to run it, and tweak the prefs however you want them and so on? -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
>> I wish you'd stop proposing the lex filter so that we can discuss the
>> Doxyfile.in file and the accompanying Meson rule.
I see no obstacle to discussing Doxyfile.in and the meson.build file. Your suggestions are welcome. If you can find the original Doxyfile, it
would make sense to incorporate it.
>> but having something that purports to display our source code and
>> then show something that is*not* our source tree sounds insane.
The filter enables doxygen to interpret Postgres comments. Any code displayed is the original code, not the filtered code.
>> Speaking from my personal point of view, I make very little use
>> of our Doxygen pages,
Same for me. The pages are missing critical information, such as function descriptions and descriptions of structure fields. This filter is an attempt at fixing those problems.
Of course, nothing will fix doxygen when there are no comments to begin with. If we want comprehensive doxygen output, we need to add comments. However, we don’t need to add those irritating extra symbols.
>> It seems something you want to keep for your personal use.
>> I think this filter is the kind of genious idea that it's OK if you
>> can do it at home
I don’t agree with your characterization.
The purpose of the filter is to bring existing Postgres comments into the doxygen output. While I haven’t done a full survey, the majority of Postgres code has comments describing functions, globals, macros and structure fields.
Currently, those comments are thrown away. They do not appear in the existing Doxygen output.
The filter itself is not rocket science. It is a straightforward scanner built using the flex tool. I understand many people are more comfortable with perl or python, but flex is uniquely appropriate for creating small, efficient scanners.
The resulting output is not perfect. However, it is a major step forward from what we are currently producing. Over time, we can gradually update Postgres comments to fill in missing information. Eventually, doxygen output could become a “first look” tool for looking at code.
Will it be my “first look” tool. Not likely. Like you, I am comfortable with Intellij and VsCode. I have my build environments already set up.
I do not see this as a high-risk endeavor. It is “off to the side” of the main Postgres code. It takes an existing tool, doxygen, which is mostly useless, and starts to make it usable.
As an aside, I have contacted Intellij about allowing custom Doxygen files for hover information. No reply yet, but it would allow us to tailor our IDEs to display the information we need most.
On 05.02.24 18:29, John Morris wrote: > The purpose of the filter is to bring existing Postgres comments into > the doxygen output. While I haven’t done a full survey, the majority of > Postgres code has comments describing functions, globals, macros and > structure fields. > > Currently, those comments are thrown away. They do not appear in the > existing Doxygen output. Maybe this is something that can be tweaked on the doxygen side? For example, clangd can also process doxygen-style comments. But it can also process non-decorated comments, because it knows that the comment just before a declaration is probably the comment describing the thing. Maybe doxygen could have that functionality as well.
On 02.02.24 01:19, John Morris wrote: > Here is the updated patch. It should fix the meson issue when no doxygen > is present. I think all the explanatory messages in doc/doxygen/meson.build are a bit much. I think it's enough to just not define the target when the required conditions (dependencies, options) are not there. Maybe something like docs_pdf can serve as an example.
>> I think all the explanatory messages in doc/doxygen/meson.build
>> are a bit much. I think it's enough to just not define the target
>> when the required conditions (dependencies, options) are not there.
>> Maybe something like docs_pdf can serve as an example.
Makes sense, and it is simpler.
Let’s continue after I take a look at docs_pdf.
Thanks,
John
>> Maybe this is something that can be tweaked on the doxygen side?
I’ll look into that idea. Doxygen is a separate project with its own developers to persuade. I could imagine a special “C” or “C++” mode. I wanted to keep things simple, and the filter mechanism is how they extend doxygen to support other languages.
One alternative is to add the filter to doxygen’s webpage. They currently list a number of filters, none of which address this particular issue. The filter would become a build dependency like doxygen itself.
In the meantime, I’d like to address Alvaro’s concern about generating artifacts. It’s a bit tedious, but a visual survey of the output may add confidence or show areas needing improvement.
>> I think all the explanatory messages in doc/doxygen/meson.build
>> are a bit much. I think it's enough to just not define the target
Here is a patch with an updated meson.build as you suggested. I agree the messages were a bit much.
On the other hand, I would like to see clear error messages when dot or doxygen are not installed,
but I’ll leave that for a future discussion.
- John
---------------------------------------------------------------------
Attachment
Hi, On 2024-02-08 18:30:01 +0000, John Morris wrote: > +FILTER_PATTERNS = *.c *.h We also have a handful of .cpp files. > +/* > + * Custom banner character to be removed from comments. > + * We'll hardcode it to suit postgreSQL, but it should be set through a command line arg. > + */ > +char customBanner = '-'; > + > +/************************************************************************************************* > +A simple program which reads a file, updates the comments, and writes to stdout. > +This is intended to be used as a doxygen filter, converting existing comments to doxygen comments. > +**********************************************************************************************/ > +int main(int argc, char**argv) { > + > + // Verify we have a single argument. This file isn't really following postgres coding style - any reason not to do so? > diff --git a/doc/doxygen/meson.build b/doc/doxygen/meson.build > new file mode 100644 > index 0000000000..e5539b7854 > --- /dev/null > +++ b/doc/doxygen/meson.build > @@ -0,0 +1,73 @@ > +# Generate doxygen pages for PostgreSQL using "ninja doxygen" > +# > +# Doxygen pages are optional. Nothing in this script should > +# cause PostgreSQL builds to fail. > +# > +# Currently there are no explicit error messages > +# - If doxygen is not found, the doxygen target will not be defined. > +# - If dot is not found, no graphs will be generated. > +# - flex is already required, so we don't check for it. > +# > +# As a future enhancement, display meaningful error messages > +# when doxygen or dot are not found. Meson does not > +# support build time error messages, but they can be displayed > +# using a python custom target. What do you mean with "build time error messages" specifically? Why would we want to output anything at build time (rather than configure time)? > +# Find the doxygen command. If not found, stop and don't define the target. > +doxygen_cmd = find_program('doxygen', required: false) I'd add "native: true", it'd not work to find the program in the target environment of a cross build. > + > +# Find the dot command. If not found, no graphs will be generated. > +dot_cmd = find_program('dot', required: false) Dito. Greetings, Andres Freund
>> We also have a handful of .cpp files.
Fortunately, our .cpp files are simple and do not include C++ specific features.
We shouldn’t have any problem adding .cpp to the filter list.
At some point it will be necessary to support general C++, but I wasn’t planning to do it yet.
>> This file isn't really following postgres coding style - any reason not to do so?
I’ll update it to Postgres commenting. The filter started as a standalone project.
>> What do you mean with "build time error messages" specifically? Why would we
>> want to output anything at build time (rather than configure time)?
“build time error messages” occur after typing “ninja”.
Documentation is usually produced long after meson setup messages have been forgotten.
It is much more useful to see “Please install the doxygen program” after typing “ninja doxygen”
than to see “target not supported”, or to have to look through the setup log.
It’s personal preference, but I think the sgml docs should display a similar message.
>> I'd add "native: true", it'd not work to find the program in the target
>> environment of a cross build.
Got it. Thanks.
Update: another patch with 1) suggested changes, 2) delete old html before generating new, and 3) added flex names for the more complex regular expressions.
Exercising the “ninja doxygen” command, brought up some issues.
- The generated html directory contained old as well as new pages.
The build script now deletes old pages before building new ones. - Using a line of dashes to suppress text formatting is not implemented.
Some dashed comments are harder to read, especially around code samples. - Comments at the end of “#define” are considered part of the define
and not annotated as doxygen comments.
The first issue was very confusing, so it has been fixed.
My preference is to make the other two issues part of a future enhancement. Thoughts?
I’ve also been scanning the generated pages looking for anomalies.
- About 1/3 of pages examined.
- In most cases, the filter works as expected.
- Many places are missing information, so the output has blank fields (as expected).
- A few places have obvious nonsense. (eg. a group comment applied to one element)
- No situations where the output would be misleading.
- In all cases, the source code is “a click away”.
While I had planned to look at *every* page, I’ll probably stop at the 1/3 sample – unless someone wants to help scan through the pages with me.
I also heard back from Jetbrains about incorporating custom Doxyfiles. They do their own rendering and do not invoke the doxygen command. Custom Doxyfiles are not going to happen. (It’s probably the same for VS.)
On my Mac M1, generating doxygen takes about 20 seconds. When graphs are added, it takes 5 minutes.
Attachment
On Mon, Feb 5, 2024 at 05:29:08PM +0000, John Morris wrote: > >> It seems something you want to keep for your personal use. > > >> I think this filter is the kind of genious idea that it's OK if you > > >> can do it at home > > I don’t agree with your characterization. > > The purpose of the filter is to bring existing Postgres comments into the > doxygen output. While I haven’t done a full survey, the majority of Postgres > code has comments describing functions, globals, macros and structure fields. > > Currently, those comments are thrown away. They do not appear in the existing > Doxygen output. I have found it very strange that a tool like doxygen which can create all sorts of call graphs, just ignores some comments. The comments above function are very important. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Only you can decide what is important to you.
>> I have found it very strange that a tool like doxygen which can create
>> all sorts of call graphs, just ignores some comments. The comments
>> above function are very important.
I agree with you . I hated doxygen for decades because of the irritating annotations it required. When I discovered IDEs were creating doxygen-like popups from conventional comments, I tried to figure out what they were doing. In the process, I discovered filters, and I created a filter to match what I thought the IDEs were doing.
(As it turns out, IDEs implement their own rendering independent of doxygen.)
I find it ironic I’ve gone from being a long-time hater of doxygen to being its advocate.
- John Morris
Tiny tidbit of history. Back in the 70’s, I created a comment extractor for the language Ratfor. We used it to maintain an alphabetical index of functions and to display pseudo-code. We drew our call graphs by hand and saved them in manila folders. Hard to imagine now.