Thread: a patch for row-at-a-time execution for table functions

The primary change we made to Postgres in order to support our own
version of foreign data wrappers was a row-at-a-time execution for
table functions. In standard Postgres, when you execute a table
function, it gathers all of the rows at once and stuffs them into a
buffer in order to support cursors, even if it is just a vanilla
forward scan. We modified the code so that when you do vanilla forward
scans it executes the function one row at a time. This wasn't a big
change since the support for executing functions that way was always
in there but it was bypassed in the plan execution code. Probably
someone always intended to do this, but never got around to it.

We also encountered what I think is a bug in the way that the cleanup
callback functions are called at the end of a table function. I'm not
sure I remember the details correctly, but I believe it was freeing
the memory region used for the private state before calling the
cleanup function. If that was the bug though, why didn't I just use
malloc for the private state? I'll have to review my notes on that
one...

Anyway, my company has agreed to let me post a patch for these
changes. Before I go to the work of getting the patches ready, I'd
like to take the temperature of the commiters and find out if there is
any likelihood of getting these patches accepted.

Anyone?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/23/2013 06:09 PM, David Gudeman wrote:
> The primary change we made to Postgres in order to support our own 
> version of foreign data wrappers was a row-at-a-time execution for 
> table functions. In standard Postgres, when you execute a table 
> function, it gathers all of the rows at once and stuffs them into
> a buffer in order to support cursors, even if it is just a vanilla 
> forward scan. We modified the code so that when you do vanilla
> forward scans it executes the function one row at a time. This
> wasn't a big change since the support for executing functions that
> way was always in there but it was bypassed in the plan execution
> code. Probably someone always intended to do this, but never got
> around to it.

Guilty as charged :-(

> We also encountered what I think is a bug in the way that the
> cleanup callback functions are called at the end of a table
> function. I'm not sure I remember the details correctly, but I
> believe it was freeing the memory region used for the private state
> before calling the cleanup function. If that was the bug though,
> why didn't I just use malloc for the private state? I'll have to
> review my notes on that one...
> 
> Anyway, my company has agreed to let me post a patch for these 
> changes. Before I go to the work of getting the patches ready, I'd 
> like to take the temperature of the commiters and find out if there
> is any likelihood of getting these patches accepted.
> 
> Anyone?

+1

I have always been in favor of having this option -- in fact that was
the way it was originally coded IIRC. Something caused us to switch to
materialized mode and we decided we would wait a while to see how that
went before worrying again about value-per-call mode.

At this point you're looking at 9.4 for this, but I'll try to help see
it through since it has long been on my TODO list.

Joe



- -- 
Joe Conway
credativ LLC: http://www.credativ.us
Linux, PostgreSQL, and general Open Source
Training, Service, Consulting, & 24x7 Support
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJRdzOOAAoJEDfy90M199hlUeQQAI6c3H1rvsqtk7DmGFY+LfhS
LKOvEfYPQ2+BtTtgs7g0qjARfZg/fb5KPlUPMDk/3asT5BUnrOjCMOtDWQMIn5G8
hEjSvkzEjaTpCYQzZisrMWyIoD3YYtNZigND5eAsKKx/JXPqvd7xqgX+6FFEh6ui
ykpLXcasMZbRDDEBZZp5Yf6GV9ZPpgtvHEp2pHsUFN6d65XwHwHGMxNYAosmzqpS
0hV9d/Tjs/1P1Anw4LSXWnJTwp0U4SyerO/afYRHMk8PpzHxMCx5IGUyBMXybU04
IY3IcwIdTWdlhwuvLEGOpeCP+J8rAY5TenXumnrLvfBuw+heu/FClQndR6szzwdj
jwI/wthDRocifPbiEWBfd+rzoTp0F6dMdPleABZbkDROwsEib+cUqn0S1fPiE7zS
vsiWpm6THAPxXhC/PDzCC062b2HO2gN4oLBg2bGc0YS7QKjoUs6QFs3JJwDb06RK
MZRTJoxmMkUpwfIUY5R28kONOcjKSHNRBTfjc9ChBSqc2heXVwjTwQ/KBeJjwVKG
MRkIx/ZoQtURH0XQUSkJcqavRhEW52pWWLvl+/H7dPtGT9CV2qv5LMGuyZ1KBz0G
98LPk9bArBWxiymlYJ8Bx1WT8zjSpIMsl21NPh/d1MDYs4gr1yiyuqR8z3Es8ylB
aOoRUhiMEiOySggQsGZQ
=d5TY
-----END PGP SIGNATURE-----