Thread: Threads With Libpq
Respected All,
This is my first request/post in PG-Generals. If it is not the place for these kind of queries, then please guide me where i need to be.
I have a quick question regarding "pthread" with PostgreSQL 9.0 Libpq. I'm facing a problem with "Pthread" and libpq. Please find the below program behavoiur.
Connection_To_PG()
{
/* Making a connection to PG 9.0 */
}
void* Independent_Thread1()
{
while(1)
{
sleep(5);
/* Doing 1 Insert Operation on Table A*/
}
}
void* Independent_Thread2()
{
while(1)
{
sleep(5);
/*Doing 1 Insert Operation on Table B*/
}
main()
{
pthread Ind1,Ind2;
Connection_TO_PG();
pthread_create(&Ind1,NULL,&Independent_Thread1,NULL);
pthread_create(&Ind2,NULL,&Independent_Thread2,NULL);
if(pthread_join(Ind1,NULL)<0)
{
printf("Ind1 is completed");
}
if(pthread_join(Ind2,NULL)<0)
{
printf("Ind2 is completed");
}
}
Problem Description:
====================
When i ran the above program, it's running(i.e inserting 2 to 10 records) some time and going to hang state. Some times, it's running more than 15 minutes and some times only 2 to 3 minutes. I enabled the postgresql log level to DEBUG5, and it's also stopped to showing the progress after 2 to 3 minutes span of time.
Can some advise me, where i'm doing mistake in the above source code.. And would like to know to why it's not functioning properlly .. And i'm also getting the following message when the above libpq program runs..
******************* Message ********************
message type 0x31 arrived from server while idle
message type 0x32 arrived from server while idle
message type 0x6e arrived from server while idle
message type 0x43 arrived from server while idle
message type 0x5a arrived from server while idle
However, the below sample thread(i.e without libpq) programg is working fine witout any issues.
#include <stdio.h>
#include <pthread.h>
#include <stdlib.h>
void* thread1()
{
while(1){
printf("Thread1\n");
}
}
void* thread2()
{
while(1){
printf("Thread2\n");
}
}
int main()
{
int status;
pthread_t tid1,tid2;
pthread_create(&tid1,NULL,&thread1,NULL);
pthread_create(&tid2,NULL,&thread2,NULL);
pthread_join(tid1,NULL);
pthread_join(tid2,NULL);
return 0;
}
Thanks in advance ...
Best Regards,
Dinesh
On 1 Aug 2012, at 14:32, dinesh kumar wrote:
> Respected All,
>
> This is my first request/post in PG-Generals. If it is not the place for these kind of queries, then please guide me
wherei need to be.
>
> I have a quick question regarding "pthread" with PostgreSQL 9.0 Libpq. I'm facing a problem with "Pthread" and libpq.
Pleasefind the below program behavoiur.
>
> Connection_To_PG()
> {
> /* Making a connection to PG 9.0 */
> }
>
> void* Independent_Thread1()
> {
> while(1)
> {
> sleep(5);
> /* Doing 1 Insert Operation on Table A*/
> }
> }
>
> void* Independent_Thread2()
> {
> while(1)
> {
> sleep(5);
> /*Doing 1 Insert Operation on Table B*/
> }
>
> main()
> {
> pthread Ind1,Ind2;
> Connection_TO_PG();
> pthread_create(&Ind1,NULL,&Independent_Thread1,NULL);
> pthread_create(&Ind2,NULL,&Independent_Thread2,NULL);
> if(pthread_join(Ind1,NULL)<0)
> {
> printf("Ind1 is completed");
> }
> if(pthread_join(Ind2,NULL)<0)
> {
> printf("Ind2 is completed");
> }
> }
You need a separate connection per thread or you need to synchronise your queries onto the single central connection,
meaningthat other threads need to be blocked (from performing queries) while any thread is performing a query.
Alban Hertroys
--
Screwing up is an excellent way to attach something to the ceiling.
On Wed, Aug 1, 2012 at 8:09 AM, Alban Hertroys <haramrae@gmail.com> wrote:
> On 1 Aug 2012, at 14:32, dinesh kumar wrote:
>
>> Respected All,
>>
>> This is my first request/post in PG-Generals. If it is not the place for these kind of queries, then please guide me
wherei need to be.
>>
>> I have a quick question regarding "pthread" with PostgreSQL 9.0 Libpq. I'm facing a problem with "Pthread" and
libpq.Please find the below program behavoiur.
>>
>> Connection_To_PG()
>> {
>> /* Making a connection to PG 9.0 */
>> }
>>
>> void* Independent_Thread1()
>> {
>> while(1)
>> {
>> sleep(5);
>> /* Doing 1 Insert Operation on Table A*/
>> }
>> }
>>
>> void* Independent_Thread2()
>> {
>> while(1)
>> {
>> sleep(5);
>> /*Doing 1 Insert Operation on Table B*/
>> }
>>
>> main()
>> {
>> pthread Ind1,Ind2;
>> Connection_TO_PG();
>> pthread_create(&Ind1,NULL,&Independent_Thread1,NULL);
>> pthread_create(&Ind2,NULL,&Independent_Thread2,NULL);
>> if(pthread_join(Ind1,NULL)<0)
>> {
>> printf("Ind1 is completed");
>> }
>> if(pthread_join(Ind2,NULL)<0)
>> {
>> printf("Ind2 is completed");
>> }
>> }
>
> You need a separate connection per thread or you need to synchronise your queries onto the single central connection,
meaningthat other threads need to be blocked (from performing queries) while any thread is performing a query.
>
> Alban Hertroys
>
> --
> Screwing up is an excellent way to attach something to the ceiling.
Yeah. Also, OP left out the most important detail, namely where and
how the connection object stored. If the objective is to try and make
two concurrent actions on the database, I'd consider giving
asynchronous queries a whirl:
http://www.postgresql.org/docs/8.1/static/libpq-async.html. Basically
you pair a PQsendQuery with a PQgetResult. It's a lot easier to code
than multi-threaded libpq and tends to be more robust in my
experience. If you must use threads, you'll want to keep a connection
with each thread instance -- I'd avoid any temptation to use a client
side connection pool.
merlin
Hi Merlin/Alban,
Thank you very much for your guidance .. Sure will use Async Llibpq Functionalities or Will use Thread Mutex for making the the queries to be synchronize ...
Best Regards,
Dinesh
On Wed, Aug 1, 2012 at 7:33 PM, Merlin Moncure <mmoncure@gmail.com> wrote:
Yeah. Also, OP left out the most important detail, namely where andOn Wed, Aug 1, 2012 at 8:09 AM, Alban Hertroys <haramrae@gmail.com> wrote:
> On 1 Aug 2012, at 14:32, dinesh kumar wrote:
>
>> Respected All,
>>
>> This is my first request/post in PG-Generals. If it is not the place for these kind of queries, then please guide me where i need to be.
>>
>> I have a quick question regarding "pthread" with PostgreSQL 9.0 Libpq. I'm facing a problem with "Pthread" and libpq. Please find the below program behavoiur.
>>
>> Connection_To_PG()
>> {
>> /* Making a connection to PG 9.0 */
>> }
>>
>> void* Independent_Thread1()
>> {
>> while(1)
>> {
>> sleep(5);
>> /* Doing 1 Insert Operation on Table A*/
>> }
>> }
>>
>> void* Independent_Thread2()
>> {
>> while(1)
>> {
>> sleep(5);
>> /*Doing 1 Insert Operation on Table B*/
>> }
>>
>> main()
>> {
>> pthread Ind1,Ind2;
>> Connection_TO_PG();
>> pthread_create(&Ind1,NULL,&Independent_Thread1,NULL);
>> pthread_create(&Ind2,NULL,&Independent_Thread2,NULL);
>> if(pthread_join(Ind1,NULL)<0)
>> {
>> printf("Ind1 is completed");
>> }
>> if(pthread_join(Ind2,NULL)<0)
>> {
>> printf("Ind2 is completed");
>> }
>> }
>
> You need a separate connection per thread or you need to synchronise your queries onto the single central connection, meaning that other threads need to be blocked (from performing queries) while any thread is performing a query.
>
> Alban Hertroys
>
> --
> Screwing up is an excellent way to attach something to the ceiling.
how the connection object stored. If the objective is to try and make
two concurrent actions on the database, I'd consider giving
asynchronous queries a whirl:
http://www.postgresql.org/docs/8.1/static/libpq-async.html. Basically
you pair a PQsendQuery with a PQgetResult. It's a lot easier to code
than multi-threaded libpq and tends to be more robust in my
experience. If you must use threads, you'll want to keep a connection
with each thread instance -- I'd avoid any temptation to use a client
side connection pool.
merlin
Hi All ..
It seems Semaphores satisfied my conditions ... Below is the modified behaviour ...
Thank you all for the wonderful guidance on this ...
Best Regards,
Dinesh
manojadinesh.blogspot.com
It seems Semaphores satisfied my conditions ... Below is the modified behaviour ...
Connection_To_PG()
{
/* Making a connection to PG 9.0 */
}
void* Independent_Thread1()
{
while(1)
{
sleep(5);
/* Doing 1 Insert Operation on Table A*/
sem_post(&Flag);
}
}
}
void* Independent_Thread2()
{
while(1)
{
sleep(5);
sleep(5);
sem_wait(&Flag);
/*Doing 1 Insert Operation on Table B*/}
main()
{
pthread Ind1,Ind2;
Connection_TO_PG();
pthread_create(&Ind1,NULL,&Independent_Thread1,NULL);
pthread_create(&Ind2,NULL,&Independent_Thread2,NULL);
if(pthread_join(Ind1,NULL)<0)
{
printf("Ind1 is completed");
}
if(pthread_join(Ind2,NULL)<0)
{
printf("Ind2 is completed");
}
}
Thank you all for the wonderful guidance on this ...
Best Regards,
Dinesh
manojadinesh.blogspot.com
On Wed, Aug 1, 2012 at 8:34 PM, dinesh kumar <dineshkumar02@gmail.com> wrote:
Hi Merlin/Alban,Thank you very much for your guidance .. Sure will use Async Llibpq Functionalities or Will use Thread Mutex for making the the queries to be synchronize ...Best Regards,DineshOn Wed, Aug 1, 2012 at 7:33 PM, Merlin Moncure <mmoncure@gmail.com> wrote:Yeah. Also, OP left out the most important detail, namely where andOn Wed, Aug 1, 2012 at 8:09 AM, Alban Hertroys <haramrae@gmail.com> wrote:
> On 1 Aug 2012, at 14:32, dinesh kumar wrote:
>
>> Respected All,
>>
>> This is my first request/post in PG-Generals. If it is not the place for these kind of queries, then please guide me where i need to be.
>>
>> I have a quick question regarding "pthread" with PostgreSQL 9.0 Libpq. I'm facing a problem with "Pthread" and libpq. Please find the below program behavoiur.
>>
>> Connection_To_PG()
>> {
>> /* Making a connection to PG 9.0 */
>> }
>>
>> void* Independent_Thread1()
>> {
>> while(1)
>> {
>> sleep(5);
>> /* Doing 1 Insert Operation on Table A*/
>> }
>> }
>>
>> void* Independent_Thread2()
>> {
>> while(1)
>> {
>> sleep(5);
>> /*Doing 1 Insert Operation on Table B*/
>> }
>>
>> main()
>> {
>> pthread Ind1,Ind2;
>> Connection_TO_PG();
>> pthread_create(&Ind1,NULL,&Independent_Thread1,NULL);
>> pthread_create(&Ind2,NULL,&Independent_Thread2,NULL);
>> if(pthread_join(Ind1,NULL)<0)
>> {
>> printf("Ind1 is completed");
>> }
>> if(pthread_join(Ind2,NULL)<0)
>> {
>> printf("Ind2 is completed");
>> }
>> }
>
> You need a separate connection per thread or you need to synchronise your queries onto the single central connection, meaning that other threads need to be blocked (from performing queries) while any thread is performing a query.
>
> Alban Hertroys
>
> --
> Screwing up is an excellent way to attach something to the ceiling.
how the connection object stored. If the objective is to try and make
two concurrent actions on the database, I'd consider giving
asynchronous queries a whirl:
http://www.postgresql.org/docs/8.1/static/libpq-async.html. Basically
you pair a PQsendQuery with a PQgetResult. It's a lot easier to code
than multi-threaded libpq and tends to be more robust in my
experience. If you must use threads, you'll want to keep a connection
with each thread instance -- I'd avoid any temptation to use a client
side connection pool.
merlin