Thread: psql & query string length

psql & query string length

From
"Ansley, Michael"
Date:
Hi,

Well, I got psql to do it's thing, eventually.  I've tested it for pretty
much everything, including \e, \g, \r, \i.  
The one problem that I have had is that after about the third '\i long.sql',
I get a core dump, because sprintf moaned about string size complications.
The way I have structured it, memory is reallocated (re- malloc'd, not
realloc'd) every time the query is extended.  I suspect that this is very
inefficient, and probably causing the system to hooch after loading long.sql
three times.  The main thought that I have had is to extend the query buffer
in blocks of about 8k or 16k.  I presume that once working with set memory
sizes, the memory usage will be substantially more efficient.  Ideas?

Also, what's the deal with realloc?  I tried it a couple of times, but it
really screwed me around (hence the re- malloc'ing).  Or is it just a Bad
Move to use realloc?

MikeA



Re: [HACKERS] psql & query string length

From
Tom Lane
Date:
"Ansley, Michael" <Michael.Ansley@intec.co.za> writes:
> The way I have structured it, memory is reallocated (re- malloc'd, not
> realloc'd) every time the query is extended.  I suspect that this is very
> inefficient,

Probably.  You should normally expand by a significant amount each time
you reallocate an expansible buffer, just to avoid making too many
demands on malloc.  The method I favor is to double the buffer size at
each realloc step.

> and probably causing the system to hooch after loading long.sql
> three times.

... but not doing so shouldn't cause a coredump.  I bet a plain old
bug is involved here, like writing past the end of the space you do
have allocated.

> Also, what's the deal with realloc?  I tried it a couple of times, but it
> really screwed me around (hence the re- malloc'ing).  Or is it just a Bad
> Move to use realloc?

realloc is perfectly fine ... see above for more likely theory.

On some old pre-ANSI-standard machines, realloc(NULL, ...) does not
work, so for portability's sake you ought to only use realloc to
increase the size of an existing buffer.
        regards, tom lane