Thread: FDW API / flow charts for the docs?
Hi, I've been experimenting with the new reworked FDW API to get familiar with it. The postgres_fdw is a great source of knowledge (huge thanks to Shigeru Hanada, KaiGai Kohei and everyone else who made this happen), but in the end I had to draw some flow charts in Dia, to understand how exactly do the functions interact, pass private data etc. in various scenarios. Attached is the set of flow charts, showing the sequence of callbacks for all the supported commands (i.e. SELECT, INSERT, UPDATE, DELETE, ANALYZE). Wouldn't it be useful to put something like this into the docs? I mean, the FDW API is not going to get any simpler, and for me this was a significant help. regards Tomas
Attachment
<div dir="ltr"><div class="gmail_extra">On Wed, Oct 16, 2013 at 8:35 PM, Tomas Vondra <<a href="mailto:tv@fuzzy.cz">tv@fuzzy.cz</a>>wrote:<br />><br />> [...]<br />><br />> Attached is the set offlow charts, showing the sequence of callbacks<br /> > for all the supported commands (i.e. SELECT, INSERT, UPDATE,DELETE,<br />> ANALYZE).<br /><br /></div><div class="gmail_extra">Thank you very much... this flow charts willhelp many people, including me ;-)<br /></div><div class="gmail_extra"> <br /><br />> Wouldn't it be useful to putsomething like this into the<br />> docs? I mean, the FDW API is not going to get any simpler, and for me<br />>this was a significant help.<br />><br /><br /></div><div class="gmail_extra"> +1 to add into docs.<br /></div><divclass="gmail_extra"><br /></div><div class="gmail_extra">I think we can add this flow charts to [1].<br /></div><divclass="gmail_extra"><br />Regards,<br /><br />[1] <a href="http://www.postgresql.org/docs/9.3/interactive/fdwhandler.html">http://www.postgresql.org/docs/9.3/interactive/fdwhandler.html</a><br /><br/>--<br />Fabrízio de Royes Mello<br />Consultoria/Coaching PostgreSQL<br />>> Timbira: <a href="http://www.timbira.com.br">http://www.timbira.com.br</a><br/>>> Blog sobre TI: <a href="http://fabriziomello.blogspot.com">http://fabriziomello.blogspot.com</a><br/> >> Perfil Linkedin: <a href="http://br.linkedin.com/in/fabriziomello">http://br.linkedin.com/in/fabriziomello</a><br/>>> Twitter: <a href="http://twitter.com/fabriziomello">http://twitter.com/fabriziomello</a></div></div>
Tomas Vondra wrote: > Attached is the set of flow charts, showing the sequence of callbacks > for all the supported commands (i.e. SELECT, INSERT, UPDATE, DELETE, > ANALYZE). Wouldn't it be useful to put something like this into the > docs? I mean, the FDW API is not going to get any simpler, and for me > this was a significant help. Please see this thread http://www.postgresql.org/message-id/4BB9E69F.9080203@usit.uio.no -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Alvaro, * Alvaro Herrera (alvherre@2ndquadrant.com) wrote: > Tomas Vondra wrote: > > Attached is the set of flow charts, showing the sequence of callbacks > > for all the supported commands (i.e. SELECT, INSERT, UPDATE, DELETE, > > ANALYZE). Wouldn't it be useful to put something like this into the > > docs? I mean, the FDW API is not going to get any simpler, and for me > > this was a significant help. > > Please see this thread > http://www.postgresql.org/message-id/4BB9E69F.9080203@usit.uio.no The conclusion of that thread appears to be "use dia", which was done here.. Am I missing something there (full disclosure- I haven't looked at the dia yet)? Also, for my part, I'd suggest putting it on the wiki initially anyway, as then it can be seen directly (load it as a png or what-have-you) and it becomes immediately available to users. The .dia should also be on the wiki, of course, and then included in the PG tree eventually if it's added as part of the official docs. Thanks, Stephen
On 17 Říjen 2013, 5:32, Stephen Frost wrote: > Alvaro, > > * Alvaro Herrera (alvherre@2ndquadrant.com) wrote: >> Tomas Vondra wrote: >> > Attached is the set of flow charts, showing the sequence of callbacks >> > for all the supported commands (i.e. SELECT, INSERT, UPDATE, DELETE, >> > ANALYZE). Wouldn't it be useful to put something like this into the >> > docs? I mean, the FDW API is not going to get any simpler, and for me >> > this was a significant help. >> >> Please see this thread >> http://www.postgresql.org/message-id/4BB9E69F.9080203@usit.uio.no > > The conclusion of that thread appears to be "use dia", which was done > here.. Am I missing something there (full disclosure- I haven't looked > at the dia yet)? My impression from that thread was that one of the requirements is reasonable versioning / diff support, and AFAIK that's not a good match for any GUI-based product. So while I like dia and I used it for drawing the charts I submitted today, I don't think it works with this (quite reasonable) requirement. The only tool that might be a good match is graphviz (also mentioned in that thread). It's text-based, widely available and quite customizable. I'm not a graphviz expert, but attached is a result of 5-minute work with graphviz. Not perfect, but I'm pretty sure we could get much better / nicer results in very short time. It's easy to edit by hand, do versioning and /or diff on that, etc. I doubt there's a better option available. > Also, for my part, I'd suggest putting it on the wiki initially anyway, > as then it can be seen directly (load it as a png or what-have-you) and > it becomes immediately available to users. The .dia should also be on > the wiki, of course, and then included in the PG tree eventually if it's > added as part of the official docs. No problem with that, but I'd like to know in advance if we're willing to put that into the docs / under what requirements etc. Otherwise it might result in a major effort just to get it from wiki into docs later. Tomas
Attachment
Tomas,
* Tomas Vondra (tv@fuzzy.cz) wrote:
> My impression from that thread was that one of the requirements is
> reasonable versioning / diff support, and AFAIK that's not a good match
> for any GUI-based product. So while I like dia and I used it for drawing
> the charts I submitted today, I don't think it works with this (quite
> reasonable) requirement.
I'm not sure why you feel that way wrt dia..? As was pointed out in the
thread, if you decompress the dia, it's pretty reasonable XML and diffs,
etc, will work reasonably well with it.
> > Also, for my part, I'd suggest putting it on the wiki initially anyway,
> > as then it can be seen directly (load it as a png or what-have-you) and
> > it becomes immediately available to users. The .dia should also be on
> > the wiki, of course, and then included in the PG tree eventually if it's
> > added as part of the official docs.
>
> No problem with that, but I'd like to know in advance if we're willing to
> put that into the docs / under what requirements etc. Otherwise it might
> result in a major effort just to get it from wiki into docs later.
I can't see it being a major effort to get it from the wiki into the
docs, though perhaps I'm being a bit over-optomistic wrt that. Still,
I'd much rather have it somewhere than not have it at all...
Thanks,
Stephen
On 10/18/13 5:35 PM, Stephen Frost wrote: > I can't see it being a major effort to get it from the wiki into the > docs, though perhaps I'm being a bit over-optomistic wrt that. Hah! Consider that an image would have to work with the following toolchains/output formats: - standalone HTML output - web site HTML output - reasonable mobile/small-screen experience - HTML XSLT - jadetex - pdfjadetex - FOP - Texinfo (at least not break it) - man (at least not break it) - EPUB In my mind, it's not worth the effort unless we have, say, at least a dozen really useful images to add. I don't want to go through this entire pain for one image on a pretty minor topic.
* Peter Eisentraut (peter_e@gmx.net) wrote:
> In my mind, it's not worth the effort unless we have, say, at least a
> dozen really useful images to add. I don't want to go through this
> entire pain for one image on a pretty minor topic.
These are certainly fair issues- my point was merely that once we have
support for such in the regular docs, the actual migration from the wiki
into the docs would hopefully not be too difficult.
As to your point about not wanting to do it for a single image- it seems
we could potentially say that for every individual image proposed, but
if we don't keep track of those images anywhere then we may not realize
that 5 or 10 have actually been done and proposed but never integrated.
If they're kept on the wiki then perhaps we would both keep track of the
ones proposed and realize when it's worthwhile to add support for them
to the doc build system.
As to Tom's point on the previous thread, that we would need to actually
maintain these images, that is helped by using dia, imv, since it's a
pretty simple tool to use and understand and is available on many
platforms. There's still some risk there, of course, but it could be
worth it in the end. Images really can explain things in a much better
way in many cases.
Thanks,
Stephen
On 2013-10-18 18:46:10 -0400, Stephen Frost wrote: > As to your point about not wanting to do it for a single image- it seems > we could potentially say that for every individual image proposed, but > if we don't keep track of those images anywhere then we may not realize > that 5 or 10 have actually been done and proposed but never integrated. > [...] Images really can explain things in a much better > way in many cases. +many Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
On 18.10.2013 23:35, Stephen Frost wrote: > Tomas, > > * Tomas Vondra (tv@fuzzy.cz) wrote: >> My impression from that thread was that one of the requirements is >> reasonable versioning / diff support, and AFAIK that's not a good >> match for any GUI-based product. So while I like dia and I used it >> for drawing the charts I submitted today, I don't think it works >> with this (quite reasonable) requirement. > > I'm not sure why you feel that way wrt dia..? As was pointed out in > the thread, if you decompress the dia, it's pretty reasonable XML and > diffs, etc, will work reasonably well with it. > >>> Also, for my part, I'd suggest putting it on the wiki initially >>> anyway, as then it can be seen directly (load it as a png or >>> what-have-you) and it becomes immediately available to users. >>> The .dia should also be on the wiki, of course, and then included >>> in the PG tree eventually if it's added as part of the official >>> docs. >> >> No problem with that, but I'd like to know in advance if we're >> willing to put that into the docs / under what requirements etc. >> Otherwise it might result in a major effort just to get it from >> wiki into docs later. > > I can't see it being a major effort to get it from the wiki into the > docs, though perhaps I'm being a bit over-optomistic wrt that. > Still, I'd much rather have it somewhere than not have it at all... I meant something a bit different. Imagine I start with a simple chart but spend a lot of time improving it over time (adding comments, formattting) only to find out later I need to redo that from scratch in a different tool. Wouldn't that be easier to start with the target tool in the first place? Tomas
On 18.10.2013 23:52, Peter Eisentraut wrote: > On 10/18/13 5:35 PM, Stephen Frost wrote: >> I can't see it being a major effort to get it from the wiki into >> the docs, though perhaps I'm being a bit over-optomistic wrt that. > > Hah! > > Consider that an image would have to work with the following > toolchains/output formats: > > - standalone HTML output > - web site HTML output > - reasonable mobile/small-screen experience > - HTML XSLT > - jadetex > - pdfjadetex > - FOP > - Texinfo (at least not break it) > - man (at least not break it) > - EPUB > > In my mind, it's not worth the effort unless we have, say, at least > a dozen really useful images to add. I don't want to go through this > entire pain for one image on a pretty minor topic. AFAIK graphviz can give you at least .ps .svg .fig .png .gif .dia formats (and some other). I believe that covers most (if not all) of the cases you've mentioned.
<p dir="ltr">I would be more inclined to let GraphViz into the process than Dia; the former fits *much* better into a Make-basedprocess.<p dir="ltr">It is worth observing that there are schema diagramming systems (SchemaSpy is mighty likable)that build diagrams using GraphViz. We have integrated this into internal deployments at Afilias.<p dir="ltr">RodTaylor's autodoc tool has some minimal Dia support, but it lacks the ability to do layout; if you use Dia output,you'll have to manually drag things around to have the tables *not* all atop one another. I'm a little surprisedDia hasn't integrated in GraphViz to help with that; note that the MacOS diagram tool OmniGraffle (proprietary,considered quite good, competitive with Visio) uses GraphViz to help do automatic object layout. That thisis integrated inside an interactive GUI app is pretty wild/cool.<p dir="ltr">I'd think it a fine idea to add rules for.dot files (the usual GraphViz language/format to describe graphs) to the PG Make rules.
On Fri, 2013-10-18 at 18:46 -0400, Stephen Frost wrote: > As to your point about not wanting to do it for a single image- it seems > we could potentially say that for every individual image proposed, but > if we don't keep track of those images anywhere then we may not realize > that 5 or 10 have actually been done and proposed but never integrated. > If they're kept on the wiki then perhaps we would both keep track of the > ones proposed and realize when it's worthwhile to add support for them > to the doc build system. Right. > As to Tom's point on the previous thread, that we would need to actually > maintain these images, that is helped by using dia, imv, since it's a > pretty simple tool to use and understand and is available on many > platforms. I'd also take a look at ditaa, which might make this even simpler.
On Sat, 2013-10-19 at 00:52 +0200, Tomas Vondra wrote: > AFAIK graphviz can give you at least .ps .svg .fig .png .gif .dia > formats (and some other). I believe that covers most (if not all) of > the cases you've mentioned. I'm not concerned about that. It's the code that would need to be written to implement those cases.
On 10/19/2013 02:22 AM, Christopher Browne wrote: > I would be more inclined to let GraphViz into the process than Dia; > the former fits *much* better into a Make-based process. I also cast my vote for Graphviz. -- Vik
On Sat, Oct 19, 2013 at 01:56:23PM -0400, Peter Eisentraut wrote: > On Fri, 2013-10-18 at 18:46 -0400, Stephen Frost wrote: > > As to your point about not wanting to do it for a single image- it seems > > we could potentially say that for every individual image proposed, but > > if we don't keep track of those images anywhere then we may not realize > > that 5 or 10 have actually been done and proposed but never integrated. > > If they're kept on the wiki then perhaps we would both keep track of the > > ones proposed and realize when it's worthwhile to add support for them > > to the doc build system. > > Right. > > > As to Tom's point on the previous thread, that we would need to actually > > maintain these images, that is helped by using dia, imv, since it's a > > pretty simple tool to use and understand and is available on many > > platforms. > > I'd also take a look at ditaa, which might make this even simpler. If we manage to keep Java out of our build toolchain, we will be much the better for it. Cheers, David. -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
On 10/21/2013 06:24 AM, David Fetter wrote: > On Sat, Oct 19, 2013 at 01:56:23PM -0400, Peter Eisentraut wrote: >> I'd also take a look at ditaa, which might make this even simpler. > If we manage to keep Java out of our build toolchain, we will be much > the better for it. Amen. -- Vik