Thread: Memory Management in pqlib, Garbage Collection support
Hi, since I got no answer so far I searched through the docu again. I searched for GC as well as Garbage, and all garbage refersto is with regard to vacuuming a database. But my question refers to wether or not memory management is with garbagecollection supported or not. When I try to link against pqlib, it seems I need to have garbage collection enabledin order for it to link. But the documentation mentions it nowhere. Please, can someone confirm this or is this the wrong list? Thanks Alex
On Thu, May 03, 2012 at 09:08:53AM +0200, Alexander Reichstadt wrote: > > Hi, > > since I got no answer so far I searched through the docu again. I searched for GC as well as Garbage, and all garbage refersto is with regard to vacuuming a database. But my question refers to wether or not memory management is with garbagecollection supported or not. When I try to link against pqlib, it seems I need to have garbage collection enabledin order for it to link. But the documentation mentions it nowhere. What kind of garbage collection do you need to have enabled in order to link? C is based on manual memory-management, and you can't generally have garbage-collection in it. Hence, libpq is *not* garbage-collected and your statement that it needs "garbage collection enabled" doesn't make much sense to me. Are you talking about libpq bindings to some other language, perhaps? > Please, can someone confirm this or is this the wrong list? This is the right list. Cheers, Peter -- http://sjamaan.ath.cx -- "The process of preparing programs for a digital computer is especially attractive, not only because it can be economically and scientifically rewarding, but also because it can be an aesthetic experience much like composing poetry or music." -- Donald Knuth
On 05/03/12 12:08 AM, Alexander Reichstadt wrote: > since I got no answer so far I searched through the docu again. I searched for GC as well as Garbage, and all garbage refersto is with regard to vacuuming a database. But my question refers to wether or not memory management is with garbagecollection supported or not. When I try to link against pqlib, it seems I need to have garbage collection enabledin order for it to link. But the documentation mentions it nowhere. what is this garbage collection you are talking about having to enable? I'm not aware of any such options in C, and this is a C API. not Java. not Python. not C++. -- john r pierce N 37, W 122 santa cruz ca mid-left coast
Thanks, that's answering my question. In Objective-C as well as many other languages there is the feature to turn on Garbage Collection. It's a separate thread that scans memory for strong pointers, their source and origin and "vacuums" memory so to not have any leaks. Anything unreferenced and no longer needed is cleaned up automatically. There are some border cases where GC can fail, but for most it works.
There are three values for the compiler: off, supported and required. One cannot mix subprojects unlike in e.g. ARC (Automated Reference Counting). Either it's on or off. Supported means that it covers both sides, "manual" memory management and automated management.
As GC is an evolutionary stage across languages, objects formerly deallocated or freed or released (or whatever other term is used across different languages) through a unique call or method name where one would normally also host any steps to release associated resources not necessarily memory related (closing sockets, file handles etc.), such call is replaced with the language specific call that the garbage collector calls to still provide for closing these resources while still being able to handle memory management in an automated fashion. All directly memory related calls become no-ops to provide for backward compatibility. In the end one can compile with GC on, or GC off and the compiler refers to the calls it needs to deal with the two aspects accordingly, either resource-only management or resource-and-memory management.
In general there are libs that provide garbage collection for C as well, like here:
For example, it'd help avoid leaks like those caused by a result not being PQclear'ed.
Alex
Am 03.05.2012 um 09:16 schrieb Peter Bex:
On Thu, May 03, 2012 at 09:08:53AM +0200, Alexander Reichstadt wrote:Hi,since I got no answer so far I searched through the docu again. I searched for GC as well as Garbage, and all garbage refers to is with regard to vacuuming a database. But my question refers to wether or not memory management is with garbage collection supported or not. When I try to link against pqlib, it seems I need to have garbage collection enabled in order for it to link. But the documentation mentions it nowhere.
What kind of garbage collection do you need to have enabled in order to link?
C is based on manual memory-management, and you can't generally have
garbage-collection in it. Hence, libpq is *not* garbage-collected and
your statement that it needs "garbage collection enabled" doesn't make
much sense to me.
Are you talking about libpq bindings to some other language, perhaps?Please, can someone confirm this or is this the wrong list?
This is the right list.
Cheers,
Peter
--
http://sjamaan.ath.cx
--
"The process of preparing programs for a digital computer
is especially attractive, not only because it can be economically
and scientifically rewarding, but also because it can be an aesthetic
experience much like composing poetry or music."
-- Donald Knuth
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
On Thu, May 3, 2012 at 9:39 AM, Alexander Reichstadt <lxr@mac.com> wrote: > Thanks, that's answering my question. In Objective-C as well as many other > languages there is the feature to turn on Garbage Collection. It's a > separate thread that scans memory for strong pointers, their source and > origin and "vacuums" memory so to not have any leaks. Anything unreferenced > and no longer needed is cleaned up automatically. There are some border > cases where GC can fail, but for most it works. libpq is a C library, not an objective-C library. So it clearly doesn't use an objective-c construct. If you are using some wrapper on top of libpq it might change things, but libpq itself has no knowledge of GC at all. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
On Thu, May 03, 2012 at 09:39:29AM +0200, Alexander Reichstadt wrote: > Thanks, that's answering my question. In Objective-C as well as many other languages there is the feature to turn on GarbageCollection. It's a separate thread that scans memory for strong pointers, their source and origin and "vacuums" memoryso to not have any leaks. Anything unreferenced and no longer needed is cleaned up automatically. There are some bordercases where GC can fail, but for most it works. OK, so you're specifically talking about Objective C. I don't know much about that language, but unless there are specific ObjC-bindings for libpq (like libpq++ for C++), you'll need to perform manual memory management and call PQClear yourself. > As GC is an evolutionary stage across languages [...] Thank you for your explanation of GC but that really wasn't neccessary. The confusion was mostly due to your mail omitting the fact that you were using ObjC rather than C, which would be the default assumption when talking about libpq. Actually, my language of choice (Chicken Scheme) is also GCed and the postgresql bindings I wrote for it integrate with its GC in such a way as to automatically call PQClear when an object gets collected. This is easy to do in any language that supports "finalizers", and with a little more effort it could even be integrated more tightly with the GC. The user can also manually decide to clear up the memory used by a result set when it's known in advance that this will no longer be needed and the memory use is prohibitive. I think if you want something similar you'll either need to write your own ObjC class to wrap the C functions or look for something pre-existing. Perhaps you can use libpq++. I've also found a "PostgreSQL Cocoa Framework" project (but it appears to be abandoned): http://pgsqlcocoa.sourceforge.net/ > In general there are libs that provide garbage collection for C as well, like here: > <http://www.hpl.hp.com/personal/Hans_Boehm/gc/> The Boehm GC is a good prototyping solution but not very suitable for general use. It's an especially bad idea to force use of such a garbage collector on the user whenever using libpq. This would also prohibit the integration of libpq with other languages and their GCs. AFAIK libpq currently does not offer specifying callback functions to use for allocating and freeing objects (it uses malloc/free directly). If this was available, it would be even easier to integrate deeply with a custom GC. > For example, it'd help avoid leaks like those caused by a result not being PQclear'ed. C programmers are generally comfortable with manual memory management, or they'd use another language. Cheers, Peter -- http://sjamaan.ath.cx -- "The process of preparing programs for a digital computer is especially attractive, not only because it can be economically and scientifically rewarding, but also because it can be an aesthetic experience much like composing poetry or music." -- Donald Knuth
On Thu, May 3, 2012 at 8:56 AM, Magnus Hagander <magnus@hagander.net> wrote: > On Thu, May 3, 2012 at 9:39 AM, Alexander Reichstadt <lxr@mac.com> wrote: >> Thanks, that's answering my question. In Objective-C as well as many other >> languages there is the feature to turn on Garbage Collection. It's a >> separate thread that scans memory for strong pointers, their source and >> origin and "vacuums" memory so to not have any leaks. Anything unreferenced >> and no longer needed is cleaned up automatically. There are some border >> cases where GC can fail, but for most it works. > > libpq is a C library, not an objective-C library. So it clearly > doesn't use an objective-c construct. If you are using some wrapper on > top of libpq it might change things, but libpq itself has no knowledge > of GC at all. It would be nice to be able to garbage collect libpq programs automatically though. I think every time I read some libpq code I see an error. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On 3 May 2012 09:39, Alexander Reichstadt <lxr@mac.com> wrote: > Thanks, that's answering my question. In Objective-C as well as many other I notice that you're talking about pqlib instead of libpq. Perhaps pqlib is an Obj-C wrapper around libpq that most of us just don't know about? Obj-C is not a particularly common programming language, after all ;) -- If you can't see the forest for the trees, Cut the trees and you'll see there is no forest.
(Sorry, I meant libpq). Actually it's finalize in Objective-C as well. PGSQLKit is the ObjC wrapper framework for libpq.I was confused by what I had learnt about GC, being it can't mix with non-GC. What the docu didn't mention in the placesI read --or at least not so that it stuck-- was that it does link against C-libs regardless of GC and without throwingany kind of warning or error. So, if I understand this correctly, if I want to use GC in my frontend app, which links against PGSQLKit which links againstlibpq, I need to enhance PGSQLKit to support GC by adding finalizers for management of libpq's memory use to makeGC available in all projects wanting to link against PGSQLKit, yes? Am 03.05.2012 um 10:32 schrieb Simon Riggs: Am 03.05.2012 um 10:13 schrieb Peter Bex: > > OK, so you're specifically talking about Objective C. I don't know much > about that language, but unless there are specific ObjC-bindings for > libpq (like libpq++ for C++), you'll need to perform manual memory > management and call PQClear yourself. > >> As GC is an evolutionary stage across languages [...] > > […] > I think if you want something similar you'll either need to write your > own ObjC class to wrap the C functions or look for something pre-existing. > Perhaps you can use libpq++. I've also found a "PostgreSQL Cocoa Framework" > project (but it appears to be abandoned): http://pgsqlcocoa.sourceforge.net/ Thanks for the pointer, but it's distributed under GPL, I can't use that. PGSQLKit is like PG, a BSD-style license. That'swhat brought me from MySQL to PG. > >> In general there are libs that provide garbage collection for C as well, like here: >> <http://www.hpl.hp.com/personal/Hans_Boehm/gc/> > > The Boehm GC is a good prototyping solution but not very suitable for > general use. It's an especially bad idea to force use of such a garbage > collector on the user whenever using libpq. This would also prohibit > the integration of libpq with other languages and their GCs. I didn't look inside, I really just looked up Wikipedia on the subject and if there are ways to get GC into C somehow andseemed to have gotten lucky. > AFAIK libpq currently does not offer specifying callback functions to > use for allocating and freeing objects (it uses malloc/free directly). > If this was available, it would be even easier to integrate deeply with > a custom GC. > >> For example, it'd help avoid leaks like those caused by a result not being PQclear'ed. > > C programmers are generally comfortable with manual memory management, or > they'd use another language. It's ok, I like Cocoa's GC though.