Thread: get_database_name() from background worker

get_database_name() from background worker

From
Koichi Suzuki
Date:
Hello PG hackers;

I'm writing an extension running on background workers and found get_database_name() causes SEGV and found internally resource owner was wet to NULL.   Could anybody let me know how it happens and how I can use this function.   Argument to get_database_name() looks correct.

Regards;
---
Koichi Suzuki

RE: get_database_name() from background worker

From
"tsunakawa.takay@fujitsu.com"
Date:
From: Koichi Suzuki <koichi@2ndquadrant.com>
> I'm writing an extension running on background workers and found
> get_database_name() causes SEGV and found internally resource owner was
> wet to NULL.   Could anybody let me know how it happens and how I can use
> this function.   Argument to get_database_name() looks correct.

Did you specify BGWORKER_BACKGROUND_DATABASE_CONNECTION when registering the background worker?
Did you start transaction by calling StartTransactionCommand()?


Regards
Takayuki Tsunakawa



RE: get_database_name() from background worker

From
ROS Didier
Date:
Hi
    I would like to know : Are you using pg_background extension to work with backgroud workers ?

    Thanks in advance

Best Regards

Didier ROS
Expertise SGBD
EDF - DTEO - DSIT - IT DMA


-----Message d'origine-----
De : tsunakawa.takay@fujitsu.com [mailto:tsunakawa.takay@fujitsu.com] 
Envoyé : mercredi 11 décembre 2019 08:21
À : 'Koichi Suzuki' <koichi@2ndquadrant.com>
Cc : pgsql-hackers@lists.postgresql.org
Objet : RE: get_database_name() from background worker

From: Koichi Suzuki <koichi@2ndquadrant.com>
> I'm writing an extension running on background workers and found
> get_database_name() causes SEGV and found internally resource owner was
> wet to NULL.   Could anybody let me know how it happens and how I can use
> this function.   Argument to get_database_name() looks correct.

Did you specify BGWORKER_BACKGROUND_DATABASE_CONNECTION when registering the background worker?
Did you start transaction by calling StartTransactionCommand()?


Regards
Takayuki Tsunakawa





Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires
etles informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa
destination,toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. 

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le
divulguerou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre
système,ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous
remercionségalement d'en avertir immédiatement l'expéditeur par retour du message. 

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont
sécuriséesou dénuées de toute erreur ou virus. 
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in
thisMessage is confidential. Any use of information contained in this Message not in accord with its purpose, any
disseminationor disclosure, either whole or partial, is prohibited except formal approval. 

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this
messagein error, please delete it and all copies from your system and notify the sender immediately by return message. 

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

Re: get_database_name() from background worker

From
Koichi Suzuki
Date:
I'm not using this.   Is this the must to use get_database_name()?
---
Koichi Suzuki


2019年12月11日(水) 16:26 ROS Didier <didier.ros@edf.fr>:
Hi
        I would like to know : Are you using pg_background extension to work with backgroud workers ?

        Thanks in advance

Best Regards

Didier ROS
Expertise SGBD
EDF - DTEO - DSIT - IT DMA


-----Message d'origine-----
De : tsunakawa.takay@fujitsu.com [mailto:tsunakawa.takay@fujitsu.com]
Envoyé : mercredi 11 décembre 2019 08:21
À : 'Koichi Suzuki' <koichi@2ndquadrant.com>
Cc : pgsql-hackers@lists.postgresql.org
Objet : RE: get_database_name() from background worker

From: Koichi Suzuki <koichi@2ndquadrant.com>
> I'm writing an extension running on background workers and found
> get_database_name() causes SEGV and found internally resource owner was
> wet to NULL.   Could anybody let me know how it happens and how I can use
> this function.   Argument to get_database_name() looks correct.

Did you specify BGWORKER_BACKGROUND_DATABASE_CONNECTION when registering the background worker?
Did you start transaction by calling StartTransactionCommand()?


Regards
Takayuki Tsunakawa





Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

Re: get_database_name() from background worker

From
Koichi Suzuki
Date:
Not using this extension, sorry.
---
Koichi Suzuki


2019年12月11日(水) 16:26 ROS Didier <didier.ros@edf.fr>:
Hi
        I would like to know : Are you using pg_background extension to work with backgroud workers ?

        Thanks in advance

Best Regards

Didier ROS
Expertise SGBD
EDF - DTEO - DSIT - IT DMA


-----Message d'origine-----
De : tsunakawa.takay@fujitsu.com [mailto:tsunakawa.takay@fujitsu.com]
Envoyé : mercredi 11 décembre 2019 08:21
À : 'Koichi Suzuki' <koichi@2ndquadrant.com>
Cc : pgsql-hackers@lists.postgresql.org
Objet : RE: get_database_name() from background worker

From: Koichi Suzuki <koichi@2ndquadrant.com>
> I'm writing an extension running on background workers and found
> get_database_name() causes SEGV and found internally resource owner was
> wet to NULL.   Could anybody let me know how it happens and how I can use
> this function.   Argument to get_database_name() looks correct.

Did you specify BGWORKER_BACKGROUND_DATABASE_CONNECTION when registering the background worker?
Did you start transaction by calling StartTransactionCommand()?


Regards
Takayuki Tsunakawa





Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

RE: get_database_name() from background worker

From
ROS Didier
Date:

So, What are you using to create background workers ? Can you show me an extract of your code ?

TIA

Best Regards

Didier

 

De : koichi@2ndquadrant.com [mailto:koichi@2ndquadrant.com]
Envoyé : mercredi 11 décembre 2019 09:16
À : ROS Didier <didier.ros@edf.fr>
Cc : tsunakawa.takay@fujitsu.com; pgsql-hackers@lists.postgresql.org
Objet : Re: get_database_name() from background worker

 

I'm not using this.   Is this the must to use get_database_name()?

---

Koichi Suzuki

 

 

20191211() 16:26 ROS Didier <didier.ros@edf.fr>:

Hi
        I would like to know : Are you using pg_background extension to work with backgroud workers ?

        Thanks in advance

Best Regards

Didier ROS
Expertise SGBD
EDF - DTEO - DSIT - IT DMA


-----Message d'origine-----
De : tsunakawa.takay@fujitsu.com [mailto:tsunakawa.takay@fujitsu.com]
Envoyé : mercredi 11 décembre 2019 08:21
À : 'Koichi Suzuki' <koichi@2ndquadrant.com>
Cc : pgsql-hackers@lists.postgresql.org
Objet : RE: get_database_name() from background worker

From: Koichi Suzuki <koichi@2ndquadrant.com>
> I'm writing an extension running on background workers and found
> get_database_name() causes SEGV and found internally resource owner was
> wet to NULL.   Could anybody let me know how it happens and how I can use
> this function.   Argument to get_database_name() looks correct.

Did you specify BGWORKER_BACKGROUND_DATABASE_CONNECTION when registering the background worker?
Did you start transaction by calling StartTransactionCommand()?


Regards
Takayuki Tsunakawa





Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.


Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

Re: get_database_name() from background worker

From
Michael Paquier
Date:
On Wed, Dec 11, 2019 at 05:17:00PM +0900, Koichi Suzuki wrote:
> Not using this extension, sorry.

I have no idea what you are trying to do, but get_database_name()
accesses the system cache, which means two things:
- The access needs to be done in the context of a transaction.  That's
a trick we use in a couple of places in core, see for example
IdentifySystem() in walsender.c which looks for the database name.  In
this case, you need to do the call in-between StartTransactionCommand
and CommitTransactionCommand, and you should make sure that the memory
context does not point to the one of the transaction to have an access
to the syscache data after committing the inner transaction used for
the lookup.
- Your background worker needs a database access, so bgw_flags needs
to be BGWORKER_SHMEM_ACCESS | BGWORKER_BACKEND_DATABASE_CONNECTION,
and also make sure to use BackgroundWorkerInitializeConnection at the
beginning of the main loop of your worker.

Hope that helps.
--
Michael

Attachment

RE: get_database_name() from background worker

From
"tsunakawa.takay@fujitsu.com"
Date:
From: Koichi Suzuki <koichi@2ndquadrant.com>
> I'm not using this.   Is this the must to use get_database_name()?

I don't think pg_background is a must, but the system catalog access by get_database_name() should require database
connectionand transaction.  See src/test/modules/worker_spi/worker_spi.c for an example of background worker.  That
usesboth of them.
 


Regards
Takayuki Tsunakawa



Re: get_database_name() from background worker

From
Craig Ringer
Date:
On Wed, 11 Dec 2019 at 14:38, Koichi Suzuki <koichi@2ndquadrant.com> wrote:
Hello PG hackers;

I'm writing an extension running on background workers and found get_database_name() causes SEGV and found internally resource owner was wet to NULL.   Could anybody let me know how it happens and how I can use this function.   Argument to get_database_name() looks correct.


I think the main question is answered; if the advice given does not help please supply your code and a backtrace from the crash obtained from a core file.

However, this reminds me of something. I'd like to make our syscache/relcache/catcache and all snapshot access functions Assert(IsTransactionState()); directly or at key locations. That'd make these mistakes much more obvious - and as bgworkers become a more popular way to write code for PostgreSQL that's going to be important.

Similarly, it might make sense to assert that we have a valid snapshot in the SPI, which we don't presently do for read-only SPI calls. I recall that one biting me repeatedly when I was learning this stuff. 

--
 Craig Ringer                   http://www.2ndQuadrant.com/
 2ndQuadrant - PostgreSQL Solutions for the Enterprise

RE: get_database_name() from background worker

From
ROS Didier
Date:
Hi
   With pg_background extension ,it is possible to make  "autonomous transaction" which means possibility to commit in
atransaction.
 
 It is like a client which connects to a postgresql instance. So you can execute any sql orders .

Best Regards
Didier ROS
-----Message d'origine-----
De : tsunakawa.takay@fujitsu.com [mailto:tsunakawa.takay@fujitsu.com] 
Envoyé : jeudi 12 décembre 2019 02:04
À : 'Koichi Suzuki' <koichi@2ndquadrant.com>
Cc : ROS Didier <didier.ros@edf.fr>; pgsql-hackers@lists.postgresql.org
Objet : RE: get_database_name() from background worker

From: Koichi Suzuki <koichi@2ndquadrant.com>
> I'm not using this.   Is this the must to use get_database_name()?

I don't think pg_background is a must, but the system catalog access by get_database_name() should require database
connectionand transaction.  See src/test/modules/worker_spi/worker_spi.c for an example of background worker.  That
usesboth of them.
 


Regards
Takayuki Tsunakawa





Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires
etles informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa
destination,toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. 

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le
divulguerou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre
système,ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous
remercionségalement d'en avertir immédiatement l'expéditeur par retour du message. 

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont
sécuriséesou dénuées de toute erreur ou virus. 
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in
thisMessage is confidential. Any use of information contained in this Message not in accord with its purpose, any
disseminationor disclosure, either whole or partial, is prohibited except formal approval. 

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this
messagein error, please delete it and all copies from your system and notify the sender immediately by return message. 

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

Re: get_database_name() from background worker

From
Craig Ringer
Date:
On Thu, 12 Dec 2019 at 16:21, ROS Didier <didier.ros@edf.fr> wrote:
Hi
   With pg_background extension ,it is possible to make  "autonomous transaction" which means possibility to commit in a transaction.
 It is like a client which connects to a postgresql instance. So you can execute any sql orders .


Yes, that's possible. It's not easy though and I strongly suggest you look into existing approaches like using dblink instead.

Please start a new thread rather than following an unrelated existing one. 


--
 Craig Ringer                   http://www.2ndQuadrant.com/
 2ndQuadrant - PostgreSQL Solutions for the Enterprise