Thread: FW: Java Memory Issue while Loading Postgres library

On 29/04/2010 7:34 PM, A.Bhattacharya@sungard.com wrote:

> We have a java exe making a call to a postgres function. This postgres
> function internally makes a call to a dll (which is written using
> Postgres extended C).

If I understand you correctly, you have a dll that uses ecpg to
communicate with postgresql. It is loaded by a JVM via JNI, and that JVM
then uses JNI calls to use your ecpg-based DLL to talk to the database.

If I've understood you right: Why this strange architecture? Usually
Java apps just talk to the database via JDBC.

As far as I know there's no particular reason you *can't* do calls to
your ecpg-using dll via JNI, though. I'd be surprised if your problems
didn't turn out to be misuse/misunderstanding of the
notoriously-hard-to-get-right JNI interface, or issues with your DLL
and/or its use of ecpg.

If I have NOT understood how you are doing things correctly, then please
be MUCH MORE SPECIFIC. Making people guess what you're doing doesn't
help you get accurate, useful answers.


I strongly recommend that you try to produce a self-contained test case.
Write a minimalist ecpg dll that does only one simple thing. Write a
minimalist, simple JNI-using Java program that loads and uses that DLL.
See if it leaks memory. If this minimalist test case demonstrates the
leak, post the source code to BOTH the dll and your simple Java program
here. Make sure both can be compiled without extra libraries and include
any project files etc required to compile them.

If your new test case doesn't leak, then you know the basic JNI + ECPG
combination isn't the issue. You need to start investigating what in
your code is causing the leak. If you just can't figure it out, then
start chopping things out and disabling things until the leak stops
happening to narrow it down.

Without a lot more detail than you have provided, it is unlikely that
anybody here can help you.

> Now the issue is that, when we make a call to this dll, it consumes a
> lot of memory and this memory is getting consumed from the heap space of
> the original java process causing an out of memory exception in Java.
>
> Is this a known issue.

I rather doubt that anybody else anywhere in the world is doing what
you're doing ;-)

Not as far as I know it's not, no. Nor have you provided any evidence
it's an issue with PostgreSQL not your own code yet.

> Do we have a way to keep these processes disjoint
> and not eat up heap space of the original process?

http://google.com/search?q=inter+process+communication

... but you should probably fix the problem that's causing the excessive
memory use/leakage rather than just working around it by running the
code out-of-process.

--
Craig Ringer

-----Original Message-----
From: Craig Ringer [mailto:craig@postnewspapers.com.au]
Sent: Thursday, April 29, 2010 6:02 PM
To: Bhattacharya, A
Cc: pgsql-general@postgresql.org
Subject: Re: FW: [GENERAL] Java Memory Issue while Loading Postgres
library

On 29/04/2010 7:34 PM, A.Bhattacharya@sungard.com wrote:

> We have a java exe making a call to a postgres function. This postgres
> function internally makes a call to a dll (which is written using
> Postgres extended C).

If I understand you correctly, you have a dll that uses ecpg to
communicate with postgresql. It is loaded by a JVM via JNI, and that JVM

then uses JNI calls to use your ecpg-based DLL to talk to the database.

If I've understood you right: Why this strange architecture? Usually
Java apps just talk to the database via JDBC.

As far as I know there's no particular reason you *can't* do calls to
your ecpg-using dll via JNI, though. I'd be surprised if your problems
didn't turn out to be misuse/misunderstanding of the
notoriously-hard-to-get-right JNI interface, or issues with your DLL
and/or its use of ecpg.

If I have NOT understood how you are doing things correctly, then please

be MUCH MORE SPECIFIC. Making people guess what you're doing doesn't
help you get accurate, useful answers.


I strongly recommend that you try to produce a self-contained test case.

Write a minimalist ecpg dll that does only one simple thing. Write a
minimalist, simple JNI-using Java program that loads and uses that DLL.
See if it leaks memory. If this minimalist test case demonstrates the
leak, post the source code to BOTH the dll and your simple Java program
here. Make sure both can be compiled without extra libraries and include

any project files etc required to compile them.

If your new test case doesn't leak, then you know the basic JNI + ECPG
combination isn't the issue. You need to start investigating what in
your code is causing the leak. If you just can't figure it out, then
start chopping things out and disabling things until the leak stops
happening to narrow it down.

Without a lot more detail than you have provided, it is unlikely that
anybody here can help you.

> Now the issue is that, when we make a call to this dll, it consumes a
> lot of memory and this memory is getting consumed from the heap space
of
> the original java process causing an out of memory exception in Java.
>
> Is this a known issue.

I rather doubt that anybody else anywhere in the world is doing what
you're doing ;-)

Not as far as I know it's not, no. Nor have you provided any evidence
it's an issue with PostgreSQL not your own code yet.

> Do we have a way to keep these processes disjoint
> and not eat up heap space of the original process?

http://google.com/search?q=inter+process+communication

... but you should probably fix the problem that's causing the excessive

memory use/leakage rather than just working around it by running the
code out-of-process.

--
Craig Ringer
------------------------------------------------------------------------
----------------

Thanks Craig for your comment.

Your understanding is slightly incorrect. Actually we required to uses a
special library from postgres. For this we write a wrapper around the
library and composed a dll out of it (which uses ECPG to communicate to
postgres). From the application front (Java) we make a JDBC call to
postgres function. This postgres function then makes call to the dll.
But the strange thing we observed that at runtime, somehow the execution
of dll is attached with java.exe. In other words, when we monitor the
performance of the application we realized that the execution of dll is
consuming java heap memory and is shown as a thread in java.exe process.
We expected that it should be linked with postgres.exe. And this
incorrect linking resulted in out of heap memory error in java. But when
we run the same postgres function using pgAdmin tool, the memory and CPU
utilization of the dll is shown attached with the postgres.exe
(corresponding to pgAdmin).

I hope I am able to explain the issue.

On 29/04/2010 8:48 PM, A.Bhattacharya@sungard.com wrote:

> Your understanding is slightly incorrect. Actually we required to uses a
> special library from postgres.

That mystery library being? From "postgres"? Do you mean a library
supplied by the PostgreSQL project its self? Libpq? If not, what?

C'mon, specifics. Please. Try to read your message after you have
written it while pretending you are somebody who does not work with your
code and does not know the things you know about it. Try to notice
assumptions and clarify them.

> For this we write a wrapper around the
> library and composed a dll out of it (which uses ECPG to communicate to
> postgres). From the application front (Java) we make a JDBC call to
> postgres function.  This postgres function then makes call to the dll.

This does not make sense. Your earlier mail said that the DLL is loaded
into the Java runtime.

How does a "postgres function" make a call to a DLL running in the JRE?
You can't make a call from the postgresql backend to code on a client
connected to ecpg.

Please be SPECIFIC. What is a "postgres function?" How does it "make a
call to the dll?" Etc. Provide links to PostgreSQL and Java
documentation for the APIs you use, use the precise terms for those
APIs, and generally be SPECIFIC and DETAILED. Post diagrams or code
samples if you have to!

> But the strange thing we observed that at runtime, somehow the execution
> of dll is attached with java.exe. In other words, when we monitor the
> performance of the application we realized that the execution of dll is
> consuming java heap memory and is shown as a thread in java.exe process.
> We expected that it should be linked with postgres.exe.

Ah, ok, so you are TRYING to load code a C extension to PostgreSQL into
the server backend?

If so, why are you using ECPG? Or are you in fact *not* using ECPG, but
the PostgreSQL backend extension interface?

How is the dll "shown as a thread in [the] java.exe process" ? Using a
Java debugger attached to the JRE? Using Process Explorer or a C-level
debugger examining the loaded DLL list of the JRE? Using the Java
monitoring APIs?

I really do not understand what you are doing, or what you are trying to
do. I doubt anybody can help you without a much more detailed and
specific explanation of what you're doing.

I think it'd be a really good idea for you to write a minimalist test
case for this and post it. Either you'll figure out what's wrong in the
process of making the test case, or you'll have something to post that
people can play with to see what you are trying to do.

--
Craig Ringer

Hi Craig,

Sorry for creating confusion. Let me (I work with Ambarish, the original
author of the mail) try to be more specific now.

We have a library (written in C) which helps us in doing phonetic based
name search. We want to use this library inside a postgres DB function.
To achieve this we wrote a small C code (we referred as wrapper class)
which uses the library. This C code is an ECPG which is bundled as a dll
and placed in postgres's lib dir. The original postgres function is
supposed to be called from a java program using JDBC. And the postgres
function should call the C function of the wrapper class. At runtime we
observed that when the postgres DB function calls the C function of the
wrapper class (in the dll), the java heap memory start increasing and
reached to the max level resulted in crashing of JVM. Then we commented
out the call to ECPG C function from postgres DB function and realized
that everything went well.

We were surprised why the loading and execution of the ECPG is taking
JVM memory. Ideally it should use the postgres.exe memory and CPU
utilization. We observed all these using windows task manager.

I hope this will help you in understanding our problem.

Thanks and Regards,

Ashish Arya

-----Original Message-----
From: pgsql-general-owner@postgresql.org
[mailto:pgsql-general-owner@postgresql.org] On Behalf Of Craig Ringer
Sent: Thursday, April 29, 2010 6:34 PM
To: Bhattacharya, A
Cc: pgsql-general@postgresql.org
Subject: Re: FW: [GENERAL] Java Memory Issue while Loading Postgres
library

On 29/04/2010 8:48 PM, A.Bhattacharya@sungard.com wrote:

> Your understanding is slightly incorrect. Actually we required to uses
a
> special library from postgres.

That mystery library being? From "postgres"? Do you mean a library
supplied by the PostgreSQL project its self? Libpq? If not, what?

C'mon, specifics. Please. Try to read your message after you have
written it while pretending you are somebody who does not work with your

code and does not know the things you know about it. Try to notice
assumptions and clarify them.

> For this we write a wrapper around the
> library and composed a dll out of it (which uses ECPG to communicate
to
> postgres). From the application front (Java) we make a JDBC call to
> postgres function.  This postgres function then makes call to the dll.

This does not make sense. Your earlier mail said that the DLL is loaded
into the Java runtime.

How does a "postgres function" make a call to a DLL running in the JRE?
You can't make a call from the postgresql backend to code on a client
connected to ecpg.

Please be SPECIFIC. What is a "postgres function?" How does it "make a
call to the dll?" Etc. Provide links to PostgreSQL and Java
documentation for the APIs you use, use the precise terms for those
APIs, and generally be SPECIFIC and DETAILED. Post diagrams or code
samples if you have to!

> But the strange thing we observed that at runtime, somehow the
execution
> of dll is attached with java.exe. In other words, when we monitor the
> performance of the application we realized that the execution of dll
is
> consuming java heap memory and is shown as a thread in java.exe
process.
> We expected that it should be linked with postgres.exe.

Ah, ok, so you are TRYING to load code a C extension to PostgreSQL into
the server backend?

If so, why are you using ECPG? Or are you in fact *not* using ECPG, but
the PostgreSQL backend extension interface?

How is the dll "shown as a thread in [the] java.exe process" ? Using a
Java debugger attached to the JRE? Using Process Explorer or a C-level
debugger examining the loaded DLL list of the JRE? Using the Java
monitoring APIs?

I really do not understand what you are doing, or what you are trying to

do. I doubt anybody can help you without a much more detailed and
specific explanation of what you're doing.

I think it'd be a really good idea for you to write a minimalist test
case for this and post it. Either you'll figure out what's wrong in the
process of making the test case, or you'll have something to post that
people can play with to see what you are trying to do.

--
Craig Ringer

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

On 29/04/2010 9:23 PM, Ashish.Arya@sungard.com wrote:
> Hi Craig,
>
> Sorry for creating confusion. Let me (I work with Ambarish, the original
> author of the mail) try to be more specific now.
>
> We have a library (written in C) which helps us in doing phonetic based
> name search. We want to use this library inside a postgres DB function.
> To achieve this we wrote a small C code (we referred as wrapper class)
> which uses the library. This C code is an ECPG which is bundled as a dll
> and placed in postgres's lib dir.

OK, that makes sense - though I'm not sure your use of ecpg in that role
does.

I haven't worked with ecpg much at all, but I didn't realise it was
capable of being used as a tool for server backend functions. Are you
using EXEC SQL CONNECT TO in the ecpg code? Or is there some other way
of using ECPG embeded in a backend that I don't know about?

I don't see anything in:
  http://www.postgresql.org/docs/8.4/static/ecpg.html
but I haven't gone through it in detail.


The usual way to write a PostgreSQL backend function in C is using the
server extension interfaces:

   http://www.postgresql.org/docs/8.4/static/extend.html
   http://www.postgresql.org/docs/8.4/static/xfunc-c.html

and, if you need to execute SQL from within your C code, the Server
Programming Interface:

   http://www.postgresql.org/docs/8.4/static/spi.html



Your code is generally compiled using pgxs.



> The original postgres function is
> supposed to be called from a java program using JDBC. And the postgres
> function should call the C function of the wrapper class.

That makes sense.

> At runtime we
> observed that when the postgres DB function calls the C function of the
> wrapper class (in the dll), the java heap memory start increasing and
> reached to the max level resulted in crashing of JVM.

OK, so the earlier statement that made it sound like you were calling a
DLL from the Java runtime:

"Now the issue is that, when we make a call to this dll, it consumes a
lot of memory and this memory is getting consumed from the heap space of
the original java process causing an out of memory exception in Java."

meant nothing of the sort, and your Java code (that's running out of
memory) is really only using JDBC?


> Then we commented
> out the call to ECPG C function from postgres DB function and realized
> that everything went well.

... but the amount of data returned from your function call changed lots
too, right?

> We were surprised why the loading and execution of the ECPG is taking
> JVM memory.

I doubt it is. I suspect you're just seeing memory used by a large
result set. Consider using a cursor. See the JDBC manual on handling
large result sets.

The JVM should *not* crash if it runs out of memory due to JDBC using
too much, though. Any crash really should be reported directly to sun.
If the JVM crashes it saves some error logs and reports the crash on the
text console so you can send them off to Sun for analysis.

If you don't actually mean that the JVM crashes at all, and actually
mean "my program throws an OutOfMemoryError" ... then yes, that's the
expected behaviour when you try to use too much memory loading a big
result set. This is NOT a jvm crash, the jvm is doing exactly what it's
supposed to do. Describing this as "crashing of JVM" is very misleading
if this is actually what's happening.

--
Craig Ringer

Hi

Did you eventually figure out what was wrong?

Was it just that you were trying to load a full result set and running
out of memory with an OutOfMemoryError?

Or was the jvm truly crashing rather than just throwing OutOfMemoryError?

--
Craig Ringer

-----Original Message-----
From: Craig Ringer [mailto:craig@postnewspapers.com.au]
Sent: Friday, April 30, 2010 12:01 PM
To: Arya, Ashish
Cc: pgsql-general@postgresql.org; Bhattacharya, A
Subject: Re: FW: [GENERAL] Java Memory Issue while Loading Postgres
library

Hi

Did you eventually figure out what was wrong?

Was it just that you were trying to load a full result set and running
out of memory with an OutOfMemoryError?

Or was the jvm truly crashing rather than just throwing
OutOfMemoryError?

--
Craig Ringer

---------------
Thanks Craig for your inputs.

However, we identified the problem as " OutOfMemoryError" and it was
thrown from Java  because of unnecessary "Raise info message"  from our
program.
Thus we have set the client_min_messages and log_min_messages to 'error'
level and switched off the messages from our program  has solved the
problem.

Please if anyone face the same issue refer to the below link.

http://archives.postgresql.org/pgsql-jdbc/2009-01/msg00068.php

Many thanks
AB