Re: Oom on temp (un-analyzed table caused by JIT) V16.1 [Fixed Already] - Mailing list pgsql-hackers

From Kirk Wolak
Subject Re: Oom on temp (un-analyzed table caused by JIT) V16.1 [Fixed Already]
Date
Msg-id CACLU5mT30u+6EyJ1RP-r5YB52B_tz94Kwa-H_4dTpkQ7Eqc0KQ@mail.gmail.com
Whole thread Raw
In response to Re: Oom on temp (un-analyzed table caused by JIT) V16.1 [Fixed Already]  (Daniel Gustafsson <daniel@yesql.se>)
Responses Re: Oom on temp (un-analyzed table caused by JIT) V16.1 [ NOT Fixed ]
List pgsql-hackers
On Fri, Jan 19, 2024 at 7:03 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> On 19 Jan 2024, at 23:09, Kirk Wolak <wolakk@gmail.com> wrote:

>  From a FUTURE email, I noticed pg_jit_available() and it's set to f??

Right, then this installation does not contain the necessary library to JIT
compile the query.

> Okay, so does this require a special BUILD command?

Yes, it requires that you compile with --with-llvm.  If you don't have llvm in
the PATH you might need to set LLVM_CONFIG to point to your llvm-config binary.
With autotools that would be something like:

    ./configure <other params> --with-llvm LLVM_CONFIG=/path/to/llvm-config

--
Daniel Gustafsson

Thank you, that made it possible to build and run...
UNFORTUNATELY this has a CLEAR memory leak (visible in htop)
I am watching it already consuming 6% of my system memory.

I am re-attaching my script.  WHICH includes the settings to FORCE JIT.
It also does an EXPLAIN so you can verify that JIT is on (this is what I added/noticed!)
And it takes over 20 minutes to get this far.  It's still running.
I am re-attaching the script. (as I tweaked it).

This creates 90 million rows of data, so it takes a while.
I BELIEVE that it consumes far less memory if you do not fetch any rows (I had problems reproducing it if no rows were fetched).
So, this may be beyond the planning stages.

Thanks,

Kirk Out!
Attachment

pgsql-hackers by date:

Previous
From: Peter Smith
Date:
Subject: Re: WIP Incremental JSON Parser
Next
From: Michael Paquier
Date:
Subject: Re: Sequence Access Methods, round two