Thread: [BUGS] (Benign ?) Libpq memory leak

[BUGS] (Benign ?) Libpq memory leak

From
Michael Cress
Date:

Hello:

 

I have noticed a memory leak in libpq. Running the following code from the context of a worker thread leaks 1024 bytes:

 

              const char conninfo[] = "dbname=testdb host=localhost user=testaccount password=password";

              PGconn     *conn;

              PGresult   *res;

              int nFields, i, j;

 

              printf("Opening connection to database\n");

              conn = PQconnectdb(conninfo);

 

/* PQ statements used to be here but were removed when the leak was noticed. They executed fine and did not contribute to the leak. */

 

              printf("Closing connection to database\n");

              PQfinish(conn);

 

 

 

Results for “Valgrind –leak-check=full “

 

 

==19194== HEAP SUMMARY:

==19194==     in use at exit: 5,324 bytes in 25 blocks

==19194==   total heap usage: 33,714 allocs, 33,689 frees, 5,311,760 bytes allocated

==19194==

==19194== 40 bytes in 1 blocks are definitely lost in loss record 3 of 9

==19194==    at 0x402E23C: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)

==19194==    by 0x7DE3E42: ???

==19194==    by 0x7DE4F5E: ???

==19194==    by 0x7DDF577: ???

==19194==    by 0x7DD0698: ???

==19194==    by 0x7DD0EAA: ???

==19194==    by 0x7DD184E: ???

==19194==    by 0x42D6C18: ???

<Redacted application call stack specific to application>

==19194==

==19194== 984 bytes in 1 blocks are definitely lost in loss record 8 of 9

==19194==    at 0x402E23C: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)

==19194==    by 0x7DE4BC3: ???

==19194==    by 0x7DDF557: ???

==19194==    by 0x7DD067E: ???

==19194==    by 0x7DD0EAA: ???

==19194==    by 0x7DD184E: ???

==19194==    by 0x42D6C18: ???

<Redacted application call stack specific to application>

==19194==

==19194== LEAK SUMMARY:

==19194==    definitely lost: 1,024 bytes in 2 blocks

==19194==    indirectly lost: 0 bytes in 0 blocks

==19194==      possibly lost: 0 bytes in 0 blocks

==19194==    still reachable: 4,300 bytes in 23 blocks

==19194==         suppressed: 0 bytes in 0 blocks

==19194== Reachable blocks (those to which a pointer was found) are not shown.

==19194== To see them, rerun with: --leak-check=full --show-leak-kinds=all

==19194==

==19194== For counts of detected and suppressed errors, rerun with: -v

==19194== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)

 

 

However, running the exact same code sequence from a simple test main() application does not show the same leak.

 

 

Results for “Valgrind –leak-check=full “ for test app

 

 

Opening connection to database

Closing connection to database

==19318==

==19318== HEAP SUMMARY:

==19318==     in use at exit: 45,120 bytes in 2,861 blocks

==19318==   total heap usage: 5,782 allocs, 2,921 frees, 359,938 bytes allocated

==19318==

==19318== LEAK SUMMARY:

==19318==    definitely lost: 0 bytes in 0 blocks

==19318==    indirectly lost: 0 bytes in 0 blocks

==19318==      possibly lost: 0 bytes in 0 blocks

==19318==    still reachable: 45,120 bytes in 2,861 blocks

==19318==         suppressed: 0 bytes in 0 blocks

==19318== Reachable blocks (those to which a pointer was found) are not shown.

==19318== To see them, rerun with: --leak-check=full --show-leak-kinds=all

 

 

 

Commenting out the code sequence from the threaded application yields the following Valgrind results:

 

Results for “Valgrind –leak-check=full “ for original threaded application with code sequence commented out and repeating the same testing steps

 

==19429==

==19429== HEAP SUMMARY:

==19429==     in use at exit: 20 bytes in 1 blocks

==19429==   total heap usage: 10,850 allocs, 10,849 frees, 577,899 bytes allocated

==19429==

==19429== LEAK SUMMARY:

==19429==    definitely lost: 0 bytes in 0 blocks

==19429==    indirectly lost: 0 bytes in 0 blocks

==19429==      possibly lost: 0 bytes in 0 blocks

==19429==    still reachable: 20 bytes in 1 blocks

==19429==         suppressed: 0 bytes in 0 blocks

==19429== Reachable blocks (those to which a pointer was found) are not shown.

==19429== To see them, rerun with: --leak-check=full --show-leak-kinds=all

 

 

The system environment is x86 running an up-to-date Ubuntu 16.10. The version of Postgresql installed is “9.5.5”. I have not performed any tests on other environments. This test was conducted against a development database containing data as well as a test database containing no data.

 

This issue appears benign as I have not yet seen the 1024 byte count increase despite multiple threads executing the same code statements. However, this was a trivial test and not subjected to any sort of load conditions (which would result in concurrent thread execution of the same code) so I cannot guarantee that statement is completely true at this time.

 

Has this issue been seen before? Is this something I should be concerned about for production use?

 

 

Thank you,

 

Michael Cress

Re: [BUGS] (Benign ?) Libpq memory leak

From
Andres Freund
Date:
Hi,m

On 2017-02-12 21:40:11 +0000, Michael Cress wrote:
> I have noticed a memory leak in libpq. Running the following code from the context of a worker thread leaks 1024
bytes:
> 
>               const char conninfo[] = "dbname=testdb host=localhost user=testaccount password=password";
>               PGconn     *conn;
>               PGresult   *res;
>               int nFields, i, j;
> 
>               printf("Opening connection to database\n");
>               conn = PQconnectdb(conninfo);
> 
> /* PQ statements used to be here but were removed when the leak was noticed. They executed fine and did not
contributeto the leak. */
 
> 
>               printf("Closing connection to database\n");
>               PQfinish(conn);

> ==19194== HEAP SUMMARY:
> ==19194==     in use at exit: 5,324 bytes in 25 blocks
> ==19194==   total heap usage: 33,714 allocs, 33,689 frees, 5,311,760 bytes allocated
> ==19194==
> ==19194== 40 bytes in 1 blocks are definitely lost in loss record 3 of 9
> ==19194==    at 0x402E23C: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
> ==19194==    by 0x7DE3E42: ???
> ==19194==    by 0x7DE4F5E: ???
> ==19194==    by 0x7DDF577: ???
> ==19194==    by 0x7DD0698: ???
> ==19194==    by 0x7DD0EAA: ???
> ==19194==    by 0x7DD184E: ???
> ==19194==    by 0x42D6C18: ???
> <Redacted application call stack specific to application>
> ==19194==
> ==19194== 984 bytes in 1 blocks are definitely lost in loss record 8 of 9
> ==19194==    at 0x402E23C: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
> ==19194==    by 0x7DE4BC3: ???
> ==19194==    by 0x7DDF557: ???
> ==19194==    by 0x7DD067E: ???
> ==19194==    by 0x7DD0EAA: ???
> ==19194==    by 0x7DD184E: ???
> ==19194==    by 0x42D6C18: ???
> <Redacted application call stack specific to application>

Without libpq call-stack it's hard to make useful comments here.  Could
you recompile with debugging symbols / install debugging symbols?


> The system environment is x86 running an up-to-date Ubuntu 16.10. The version of Postgresql installed is “9.5.5”. I
havenot performed any tests on other environments. This test was conducted against a development database containing
dataas well as a test database containing no data.
 

IIRC recent ubuntu version provide dbgsym packages in a separate
repository that you need to enable.

Greetings,

Andres Freund


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Re: [BUGS] (Benign ?) Libpq memory leak

From
Michael Cress
Date:

Andres,

 

I found and installed the package that you suggested, but it didn’t seem to rectify the problem with the valgrind output. I researched why this might be happening and came across an article that mentioned something about the library being unloaded prior to valgrind being able to determine the symbol. With that in mind, I rebuilt every dependency up to and including Postgresql from source sand, after static linking to my application, my Valgrind output is now showing no memory leaks. I will spend some time exploring this issue when I get an opportunity but at this time I am going to consider this issue as resolved. Thank you very much for your suggestion and quick response; it is greatly appreciated.

 

Regards,

 

Michael Cress

 

 

From: Andres Freund
Sent: Sunday, February 12, 2017 3:54 PM
To: Michael Cress
Cc: pgsql-bugs@postgresql.org
Subject: Re: [BUGS] (Benign ?) Libpq memory leak

 

Hi,m

On 2017-02-12 21:40:11 +0000, Michael Cress wrote:
> I have noticed a memory leak in libpq. Running the following code from the context of a worker thread leaks 1024 bytes:
>
>               const char conninfo[] = "dbname=testdb host=localhost user=testaccount password=password";
>               PGconn     *conn;
>               PGresult   *res;
>               int nFields, i, j;
>
>               printf("Opening connection to database\n");
>               conn = PQconnectdb(conninfo);
>
> /* PQ statements used to be here but were removed when the leak was noticed. They executed fine and did not contribute to the leak. */
>
>               printf("Closing connection to database\n");
>               PQfinish(conn);

> ==19194== HEAP SUMMARY:
> ==19194==     in use at exit: 5,324 bytes in 25 blocks
> ==19194==   total heap usage: 33,714 allocs, 33,689 frees, 5,311,760 bytes allocated
> ==19194==
> ==19194== 40 bytes in 1 blocks are definitely lost in loss record 3 of 9
> ==19194==    at 0x402E23C: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
> ==19194==    by 0x7DE3E42: ???
> ==19194==    by 0x7DE4F5E: ???
> ==19194==    by 0x7DDF577: ???
> ==19194==    by 0x7DD0698: ???
> ==19194==    by 0x7DD0EAA: ???
> ==19194==    by 0x7DD184E: ???
> ==19194==    by 0x42D6C18: ???
> <Redacted application call stack specific to application>
> ==19194==
> ==19194== 984 bytes in 1 blocks are definitely lost in loss record 8 of 9
> ==19194==    at 0x402E23C: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
> ==19194==    by 0x7DE4BC3: ???
> ==19194==    by 0x7DDF557: ???
> ==19194==    by 0x7DD067E: ???
> ==19194==    by 0x7DD0EAA: ???
> ==19194==    by 0x7DD184E: ???
> ==19194==    by 0x42D6C18: ???
> <Redacted application call stack specific to application>

Without libpq call-stack it's hard to make useful comments here.  Could
you recompile with debugging symbols / install debugging symbols?


> The system environment is x86 running an up-to-date Ubuntu 16.10. The version of Postgresql installed is “9.5.5”. I have not performed any tests on other environments. This test was conducted against a development database containing data as well as a test database containing no data.

IIRC recent ubuntu version provide dbgsym packages in a separate
repository that you need to enable.

Greetings,

Andres Freund