Thread: Re: [HACKERS] ODBC and palloc ...

I think this is not my case.
See attachment log for details.

Hi

----
Davide Libenzi at :
Maticad s.r.l.
Via Della Giustizia n.9 Fano (PS) 61032 Italy
Tel.: +39-721-808308 (ra)  Fax: +39-721-808309
Email: <davidel@maticad.it>
WWW: <http://www.maticad.it>

-----Original Message-----
From: Byron Nikolaidis <byronn@insightdist.com>
To: Davide Libenzi <dlibenzi@maticad.it>
Cc: pgsql-hackers@postgreSQL.org <pgsql-hackers@postgreSQL.org>;
pgsql-interfaces@postgreSQL.org <pgsql-interfaces@postgreSQL.org>; David
Hartwig <daveh@insightdist.com>
Date: Thursday, July 23, 1998 6:59 PM
Subject: Re: [HACKERS] ODBC and palloc ...


>Davide Libenzi wrote:
>>
>> After a lot of changes I've compiled,linked and tested (regression) my
>> PostgreSQL installation no HPUX 9.*.
>>
>> I've also built and installed the ODBC driver and I get Ms Access error
>> which the PostgresSQL server log in "palloc failure : memory exausted".
>>
>> Is this a server bug or ODBC driver bug ?
>>
>
>I am assuming you have a fairly new odbc driver (6.30.0248 is the
>latest) and not the old postodbc.  BTW, on our website
>(www.insightdist.com/psqlodbc) we have the DLL and a full install EXE
>for win32 so you wouldn't have to build it yourself from the source code
>if you didn't want to.
>
>The palloc failure usually occurs because Access uses the multiple OR
>query (select ... where a=1 OR a=2 OR a=3...) to access the recordset.
>The backend does not handle this very well and it is already well known
>on the TODO list.
>
>There are several possibilities to get past this:
>1. Use a non-updateable table (by setting the driver readonly option, or
>by not specifying any unique identifiers).
>2. For a query, use a snapshot recordset in the query properties.
>3. Show the OID column in the drivers advanced datasource options and
>use that alone to index on.  You should create an index on it too.  This
>is still slow, but at least shouldn't crash.
>
>Other possibilities:
>
>In house, Dave made a patch to postgres which rewrites the multiple OR
>query into a UNION query, which works great and its fast!  We may make
>this patch available evntually on our website.
>
>Byron
>

Davide Libenzi wrote:
>
> I think this is not my case.
> See attachment log for details.
>

From looking at the log that is *exactly* your case.    I pulled the
offending query out and cleaned it up a bit.

You have a two-part key (padre & figlio) and you can see the multiple
OR's between them.  The MS Jet db engine typically uses a rowset size of
10 (so you see 10 keys below) and a keyset size of a couple of hundred
or so.  In other words, it first read in 200 keys (the "keyset") and
then uses these keys to access a "rowset" of size 10 out of the entire
"resultset" (how ever many records you have total).  This is called a
Mixed (Keyset/Dynamic) cursor or a "Dynaset".   Like I said in my last
email, if you change the datasource to be read-only, then re-link your
table in Access, it will not use this style of retrieval and you should
get some results.  OR, you can try the other options I mentioned.

SELECT "padre","figlio","qta"  FROM "distinta"
WHERE "padre" = 'PPPA' AND "figlio" = 'AAA'
OR "padre" = 'KKKL' AND "figlio" = 'LLLA'
OR "padre" = 'AAAAA' AND "figlio" = 'ASDWDWD'
OR "padre" = 'AAAAA' AND "figlio" = 'ASDWDWD'
OR "padre" = 'AAAAA' AND "figlio" = 'ASDWDWD'
OR "padre" = 'AAAAA' AND "figlio" = 'ASDWDWD'
OR "padre" = 'AAAAA' AND "figlio" = 'ASDWDWD'
OR "padre" = 'AAAAA' AND "figlio" = 'ASDWDWD'
OR "padre" = 'AAAAA' AND "figlio" = 'ASDWDWD'
OR "padre" = 'AAAAA' AND "figlio" = 'ASDWDWD'



The only problem with this style of retrieving records is that the
Postgres backend can not handle it.  It results in exponential memory
usage as it tries to optimize it.  You could type in the above query by
hand to the monitor and see the same result.

Then for fun try rewriting the query to use UNIONS instead of OR's and
you will see how fast it is (assuming you have an index).  See below.

SELECT "padre","figlio","qta"  FROM "distinta"
WHERE "padre" = 'PPPA' AND "figlio" = 'AAA'
UNION
SELECT "padre","figlio","qta"  FROM "distinta"
WHERE "padre" = 'KKKL' AND "figlio" = 'LLLA'
UNION
SELECT "padre","figlio","qta"  FROM "distinta"
WHERE "padre" = 'AAAAA' AND "figlio" = 'ASDWDWD'
UNION
SELECT "padre","figlio","qta"  FROM "distinta"
WHERE "padre" = 'AAAAA' AND "figlio" = 'ASDWDWD'
....



Byron