Thread: FDW API / flow charts for the docs?

FDW API / flow charts for the docs?

From
Tomas Vondra
Date:
Hi,

I've been experimenting with the new reworked FDW API to get familiar
with it. The postgres_fdw is a great source of knowledge (huge thanks to
Shigeru Hanada, KaiGai Kohei and everyone else who made this happen),
but in the end I had to draw some flow charts in Dia, to understand how
exactly do the functions interact, pass private data etc. in various
scenarios.

Attached is the set of flow charts, showing the sequence of callbacks
for all the supported commands (i.e. SELECT, INSERT, UPDATE, DELETE,
ANALYZE). Wouldn't it be useful to put something like this into the
docs? I mean, the FDW API is not going to get any simpler, and for me
this was a significant help.

regards
Tomas

Attachment

Re: FDW API / flow charts for the docs?

From
Fabrízio de Royes Mello
Date:
<div dir="ltr"><div class="gmail_extra">On Wed, Oct 16, 2013 at 8:35 PM, Tomas Vondra <<a
href="mailto:tv@fuzzy.cz">tv@fuzzy.cz</a>>wrote:<br />><br />> [...]<br />><br />> Attached is the set
offlow charts, showing the sequence of callbacks<br /> > for all the supported commands (i.e. SELECT, INSERT,
UPDATE,DELETE,<br />> ANALYZE).<br /><br /></div><div class="gmail_extra">Thank you very much... this flow charts
willhelp many people, including me ;-)<br /></div><div class="gmail_extra">  <br /><br />> Wouldn't it be useful to
putsomething like this into the<br />> docs? I mean, the FDW API is not going to get any simpler, and for me<br
/>>this was a significant help.<br />><br /><br /></div><div class="gmail_extra"> +1 to add into docs.<br
/></div><divclass="gmail_extra"><br /></div><div class="gmail_extra">I think we can add this flow charts to [1].<br
/></div><divclass="gmail_extra"><br />Regards,<br /><br />[1] <a
href="http://www.postgresql.org/docs/9.3/interactive/fdwhandler.html">http://www.postgresql.org/docs/9.3/interactive/fdwhandler.html</a><br
/><br/>--<br />Fabrízio de Royes Mello<br />Consultoria/Coaching PostgreSQL<br />>> Timbira: <a
href="http://www.timbira.com.br">http://www.timbira.com.br</a><br/>>> Blog sobre TI: <a
href="http://fabriziomello.blogspot.com">http://fabriziomello.blogspot.com</a><br/> >> Perfil Linkedin: <a
href="http://br.linkedin.com/in/fabriziomello">http://br.linkedin.com/in/fabriziomello</a><br/>>> Twitter: <a
href="http://twitter.com/fabriziomello">http://twitter.com/fabriziomello</a></div></div>

Re: FDW API / flow charts for the docs?

From
Alvaro Herrera
Date:
Tomas Vondra wrote:

> Attached is the set of flow charts, showing the sequence of callbacks
> for all the supported commands (i.e. SELECT, INSERT, UPDATE, DELETE,
> ANALYZE). Wouldn't it be useful to put something like this into the
> docs? I mean, the FDW API is not going to get any simpler, and for me
> this was a significant help.

Please see this thread
http://www.postgresql.org/message-id/4BB9E69F.9080203@usit.uio.no

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services



Re: FDW API / flow charts for the docs?

From
Stephen Frost
Date:
Alvaro,

* Alvaro Herrera (alvherre@2ndquadrant.com) wrote:
> Tomas Vondra wrote:
> > Attached is the set of flow charts, showing the sequence of callbacks
> > for all the supported commands (i.e. SELECT, INSERT, UPDATE, DELETE,
> > ANALYZE). Wouldn't it be useful to put something like this into the
> > docs? I mean, the FDW API is not going to get any simpler, and for me
> > this was a significant help.
>
> Please see this thread
> http://www.postgresql.org/message-id/4BB9E69F.9080203@usit.uio.no

The conclusion of that thread appears to be "use dia", which was done
here..  Am I missing something there (full disclosure- I haven't looked
at the dia yet)?

Also, for my part, I'd suggest putting it on the wiki initially anyway,
as then it can be seen directly (load it as a png or what-have-you) and
it becomes immediately available to users.  The .dia should also be on
the wiki, of course, and then included in the PG tree eventually if it's
added as part of the official docs.
Thanks,
    Stephen

Re: FDW API / flow charts for the docs?

From
"Tomas Vondra"
Date:
On 17 Říjen 2013, 5:32, Stephen Frost wrote:
> Alvaro,
>
> * Alvaro Herrera (alvherre@2ndquadrant.com) wrote:
>> Tomas Vondra wrote:
>> > Attached is the set of flow charts, showing the sequence of callbacks
>> > for all the supported commands (i.e. SELECT, INSERT, UPDATE, DELETE,
>> > ANALYZE). Wouldn't it be useful to put something like this into the
>> > docs? I mean, the FDW API is not going to get any simpler, and for me
>> > this was a significant help.
>>
>> Please see this thread
>> http://www.postgresql.org/message-id/4BB9E69F.9080203@usit.uio.no
>
> The conclusion of that thread appears to be "use dia", which was done
> here..  Am I missing something there (full disclosure- I haven't looked
> at the dia yet)?

My impression from that thread was that one of the requirements is
reasonable versioning / diff support, and AFAIK that's not a good match
for any GUI-based product. So while I like dia and I used it for drawing
the charts I submitted today, I don't think it works with this (quite
reasonable) requirement.

The only tool that might be a good match is graphviz (also mentioned in
that thread). It's text-based, widely available and quite customizable.
I'm not a graphviz expert, but attached is a result of 5-minute work with
graphviz. Not perfect, but I'm pretty sure we could get much better /
nicer results in very short time.

It's easy to edit by hand, do versioning and /or diff on that, etc. I
doubt there's a better option available.

> Also, for my part, I'd suggest putting it on the wiki initially anyway,
> as then it can be seen directly (load it as a png or what-have-you) and
> it becomes immediately available to users.  The .dia should also be on
> the wiki, of course, and then included in the PG tree eventually if it's
> added as part of the official docs.

No problem with that, but I'd like to know in advance if we're willing to
put that into the docs / under what requirements etc. Otherwise it might
result in a major effort just to get it from wiki into docs later.

Tomas
Attachment

Re: FDW API / flow charts for the docs?

From
Stephen Frost
Date:
Tomas,

* Tomas Vondra (tv@fuzzy.cz) wrote:
> My impression from that thread was that one of the requirements is
> reasonable versioning / diff support, and AFAIK that's not a good match
> for any GUI-based product. So while I like dia and I used it for drawing
> the charts I submitted today, I don't think it works with this (quite
> reasonable) requirement.

I'm not sure why you feel that way wrt dia..?  As was pointed out in the
thread, if you decompress the dia, it's pretty reasonable XML and diffs,
etc, will work reasonably well with it.

> > Also, for my part, I'd suggest putting it on the wiki initially anyway,
> > as then it can be seen directly (load it as a png or what-have-you) and
> > it becomes immediately available to users.  The .dia should also be on
> > the wiki, of course, and then included in the PG tree eventually if it's
> > added as part of the official docs.
>
> No problem with that, but I'd like to know in advance if we're willing to
> put that into the docs / under what requirements etc. Otherwise it might
> result in a major effort just to get it from wiki into docs later.

I can't see it being a major effort to get it from the wiki into the
docs, though perhaps I'm being a bit over-optomistic wrt that.  Still,
I'd much rather have it somewhere than not have it at all...
Thanks,
    Stephen

Re: FDW API / flow charts for the docs?

From
Peter Eisentraut
Date:
On 10/18/13 5:35 PM, Stephen Frost wrote:
> I can't see it being a major effort to get it from the wiki into the
> docs, though perhaps I'm being a bit over-optomistic wrt that.

Hah!

Consider that an image would have to work with the following
toolchains/output formats:

- standalone HTML output
- web site HTML output
- reasonable mobile/small-screen experience
- HTML XSLT
- jadetex
- pdfjadetex
- FOP
- Texinfo (at least not break it)
- man (at least not break it)
- EPUB

In my mind, it's not worth the effort unless we have, say, at least a
dozen really useful images to add.  I don't want to go through this
entire pain for one image on a pretty minor topic.




Re: FDW API / flow charts for the docs?

From
Stephen Frost
Date:
* Peter Eisentraut (peter_e@gmx.net) wrote:
> In my mind, it's not worth the effort unless we have, say, at least a
> dozen really useful images to add.  I don't want to go through this
> entire pain for one image on a pretty minor topic.

These are certainly fair issues- my point was merely that once we have
support for such in the regular docs, the actual migration from the wiki
into the docs would hopefully not be too difficult.

As to your point about not wanting to do it for a single image- it seems
we could potentially say that for every individual image proposed, but
if we don't keep track of those images anywhere then we may not realize
that 5 or 10 have actually been done and proposed but never integrated.
If they're kept on the wiki then perhaps we would both keep track of the
ones proposed and realize when it's worthwhile to add support for them
to the doc build system.

As to Tom's point on the previous thread, that we would need to actually
maintain these images, that is helped by using dia, imv, since it's a
pretty simple tool to use and understand and is available on many
platforms.  There's still some risk there, of course, but it could be
worth it in the end.  Images really can explain things in a much better
way in many cases.
Thanks,
    Stephen

Re: FDW API / flow charts for the docs?

From
Andres Freund
Date:
On 2013-10-18 18:46:10 -0400, Stephen Frost wrote:
> As to your point about not wanting to do it for a single image- it seems
> we could potentially say that for every individual image proposed, but
> if we don't keep track of those images anywhere then we may not realize
> that 5 or 10 have actually been done and proposed but never integrated.

> [...] Images really can explain things in a much better
> way in many cases.

+many

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services



Re: FDW API / flow charts for the docs?

From
Tomas Vondra
Date:
On 18.10.2013 23:35, Stephen Frost wrote:
> Tomas,
> 
> * Tomas Vondra (tv@fuzzy.cz) wrote:
>> My impression from that thread was that one of the requirements is 
>> reasonable versioning / diff support, and AFAIK that's not a good
>> match for any GUI-based product. So while I like dia and I used it
>> for drawing the charts I submitted today, I don't think it works
>> with this (quite reasonable) requirement.
> 
> I'm not sure why you feel that way wrt dia..?  As was pointed out in
> the thread, if you decompress the dia, it's pretty reasonable XML and
> diffs, etc, will work reasonably well with it.
> 
>>> Also, for my part, I'd suggest putting it on the wiki initially
>>> anyway, as then it can be seen directly (load it as a png or
>>> what-have-you) and it becomes immediately available to users.
>>> The .dia should also be on the wiki, of course, and then included
>>> in the PG tree eventually if it's added as part of the official
>>> docs.
>> 
>> No problem with that, but I'd like to know in advance if we're
>> willing to put that into the docs / under what requirements etc.
>> Otherwise it might result in a major effort just to get it from
>> wiki into docs later.
> 
> I can't see it being a major effort to get it from the wiki into the 
> docs, though perhaps I'm being a bit over-optomistic wrt that.
> Still, I'd much rather have it somewhere than not have it at all...

I meant something a bit different. Imagine I start with a simple chart
but spend a lot of time improving it over time (adding comments,
formattting) only to find out later I need to redo that from scratch in
a different tool. Wouldn't that be easier to start with the target tool
in the first place?

Tomas



Re: FDW API / flow charts for the docs?

From
Tomas Vondra
Date:
On 18.10.2013 23:52, Peter Eisentraut wrote:
> On 10/18/13 5:35 PM, Stephen Frost wrote:
>> I can't see it being a major effort to get it from the wiki into
>> the docs, though perhaps I'm being a bit over-optomistic wrt that.
> 
> Hah!
> 
> Consider that an image would have to work with the following 
> toolchains/output formats:
> 
> - standalone HTML output
> - web site HTML output
> - reasonable mobile/small-screen experience
> - HTML XSLT
> - jadetex
> - pdfjadetex
> - FOP
> - Texinfo (at least not break it)
> - man (at least not break it)
> - EPUB
> 
> In my mind, it's not worth the effort unless we have, say, at least
> a dozen really useful images to add. I don't want to go through this 
> entire pain for one image on a pretty minor topic.

AFAIK graphviz can give you at least .ps .svg .fig .png .gif .dia
formats (and some other). I believe that covers most (if not all) of the
cases you've mentioned.



Re: FDW API / flow charts for the docs?

From
Christopher Browne
Date:
<p dir="ltr">I would be more inclined to let GraphViz into the process than Dia; the former fits *much* better into a
Make-basedprocess.<p dir="ltr">It is worth observing that there are schema diagramming systems (SchemaSpy is mighty
likable)that build diagrams using GraphViz.  We have integrated this into internal deployments at Afilias.<p
dir="ltr">RodTaylor's autodoc tool has some minimal Dia support, but it lacks the ability to do layout; if you use Dia
output,you'll have to manually drag things around to have the tables *not* all atop one another.  I'm a little
surprisedDia hasn't integrated in GraphViz to help with that; note that the MacOS diagram tool OmniGraffle
(proprietary,considered quite good, competitive with Visio) uses GraphViz to help do automatic object layout.  That
thisis integrated inside an interactive GUI app is pretty wild/cool.<p dir="ltr">I'd think it a fine idea to add rules
for.dot files (the usual GraphViz language/format to describe graphs) to the PG Make rules. 

Re: FDW API / flow charts for the docs?

From
Peter Eisentraut
Date:
On Fri, 2013-10-18 at 18:46 -0400, Stephen Frost wrote:
> As to your point about not wanting to do it for a single image- it seems
> we could potentially say that for every individual image proposed, but
> if we don't keep track of those images anywhere then we may not realize
> that 5 or 10 have actually been done and proposed but never integrated.
> If they're kept on the wiki then perhaps we would both keep track of the
> ones proposed and realize when it's worthwhile to add support for them
> to the doc build system.

Right.

> As to Tom's point on the previous thread, that we would need to actually
> maintain these images, that is helped by using dia, imv, since it's a
> pretty simple tool to use and understand and is available on many
> platforms.

I'd also take a look at ditaa, which might make this even simpler.




Re: FDW API / flow charts for the docs?

From
Peter Eisentraut
Date:
On Sat, 2013-10-19 at 00:52 +0200, Tomas Vondra wrote:
> AFAIK graphviz can give you at least .ps .svg .fig .png .gif .dia
> formats (and some other). I believe that covers most (if not all) of
> the cases you've mentioned.

I'm not concerned about that.  It's the code that would need to be
written to implement those cases.





Re: FDW API / flow charts for the docs?

From
Vik Fearing
Date:
On 10/19/2013 02:22 AM, Christopher Browne wrote:
> I would be more inclined to let GraphViz into the process than Dia;
> the former fits *much* better into a Make-based process.

I also cast my vote for Graphviz.

-- 
Vik




Re: FDW API / flow charts for the docs?

From
David Fetter
Date:
On Sat, Oct 19, 2013 at 01:56:23PM -0400, Peter Eisentraut wrote:
> On Fri, 2013-10-18 at 18:46 -0400, Stephen Frost wrote:
> > As to your point about not wanting to do it for a single image- it seems
> > we could potentially say that for every individual image proposed, but
> > if we don't keep track of those images anywhere then we may not realize
> > that 5 or 10 have actually been done and proposed but never integrated.
> > If they're kept on the wiki then perhaps we would both keep track of the
> > ones proposed and realize when it's worthwhile to add support for them
> > to the doc build system.
> 
> Right.
> 
> > As to Tom's point on the previous thread, that we would need to actually
> > maintain these images, that is helped by using dia, imv, since it's a
> > pretty simple tool to use and understand and is available on many
> > platforms.
> 
> I'd also take a look at ditaa, which might make this even simpler.

If we manage to keep Java out of our build toolchain, we will be much
the better for it.

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate



Re: FDW API / flow charts for the docs?

From
Vik Fearing
Date:
On 10/21/2013 06:24 AM, David Fetter wrote:
> On Sat, Oct 19, 2013 at 01:56:23PM -0400, Peter Eisentraut wrote:
>> I'd also take a look at ditaa, which might make this even simpler.
> If we manage to keep Java out of our build toolchain, we will be much
> the better for it.

Amen.

-- 
Vik