Thread: PL/Python SD dict wiped?

PL/Python SD dict wiped?

From
Ken Huffman
Date:

Is there a known issue with the PL/Python extension clearing the SD dict under some circumstances? As suggested by the documentation, my plpythonu uses the SD dict for one-time (prepare and ConfigParser) logic:

 

CREATE OR REPLACE FUNCTION change_notify() RETURN TRIGGER AS $$

If len(SD) == 0:

    # one time logic

    config = ConfigParser.RawConfigParser()

    config.read('…')

    SD['x'] = config.get('…','…')

    SD['plan'] = plpy.prepare('…')

x = SD['x']

plan = SD['plan']

 

To test this trigger after creation, I use psql and force a db update to cause it to fire. If I do multiple updates within psql, the SD dict is only empty for the first update. Yeah!

 

But if I quit and restart psql and do a subsequent db update, the SD dict is empty again and the one-time logic has to run again. What is clearing the dict? Is that expected?

 

Ken



Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more Click Here.

Re: PL/Python SD dict wiped?

From
Tom Lane
Date:
Ken Huffman <KHuffman@westell.com> writes:
> But if I quit and restart psql and do a subsequent db update, the SD dict is empty again and the one-time logic has
torun again. What is clearing the dict? Is that expected? 

You've got a new backend process.  That dict (or any other Python state)
is only kept within a session.

            regards, tom lane


RE: PL/Python SD dict wiped?

From
Ken Huffman
Date:
I'm fine with per-session initializing of prepared statements, but is there PL/Python mechanism that spans sessions to minimize ConfigParser calls?

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Monday, January 08, 2018 10:23 AM
To: Ken Huffman <KHuffman@westell.com>
Cc: Pg Hackers <pgsql-hackers@postgresql.org>
Subject: Re: PL/Python SD dict wiped?

Ken Huffman <KHuffman@westell.com> writes:
> But if I quit and restart psql and do a subsequent db update, the SD dict is empty again and the one-time logic has to run again. What is clearing the dict? Is that expected?

You've got a new backend process. That dict (or any other Python state) is only kept within a session.

regards, tom lane


Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more Click Here.

Re: PL/Python SD dict wiped?

From
Peter Eisentraut
Date:
On 1/8/18 11:59, Ken Huffman wrote:
> I'm fine with per-session initializing of prepared statements, but is
> there PL/Python mechanism that spans sessions to minimize ConfigParser
> calls?

Nothing built-in.  You would have to use a separate external storage
mechanism of some kind.  It could be a file, separately managed shared
memory, or perhaps something like memcached.

However, if this is the problem, then you might want to figure out a way
to keep your connections open longer, such as by using a connection pool.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


Re: PL/Python SD dict wiped?

From
David Gould
Date:
On Mon, 8 Jan 2018 14:24:42 -0500
Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote:

> On 1/8/18 11:59, Ken Huffman wrote:
> > I'm fine with per-session initializing of prepared statements, but is
> > there PL/Python mechanism that spans sessions to minimize ConfigParser
> > calls?  
> 
> Nothing built-in.  You would have to use a separate external storage
> mechanism of some kind.  It could be a file, separately managed shared
> memory, or perhaps something like memcached.

Or, given that you have a postgres connection, you could use a table.

-dg

-- 
David Gould                                    daveg@sonic.net
If simplicity worked, the world would be overrun with insects.