Re: Using per-transaction memory contexts for storing decoded tuples - Mailing list pgsql-hackers

On Thu, Sep 19, 2024 at 10:46 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Fri, Sep 20, 2024 at 5:13 AM David Rowley <dgrowleyml@gmail.com> wrote:
> >
> > On Fri, 20 Sept 2024 at 05:03, Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > > I've done other benchmarking tests while changing the memory block
> > > sizes from 8kB to 8MB. I measured the execution time of logical
> > > decoding of one transaction that inserted 10M rows. I set
> > > logical_decoding_work_mem large enough to avoid spilling behavior. In
> > > this scenario, we allocate many memory chunks while decoding the
> > > transaction and resulting in calling more malloc() in smaller memory
> > > block sizes. Here are results (an average of 3 executions):
> >
> > I was interested in seeing the memory consumption with the test that
> > was causing an OOM due to the GenerationBlock fragmentation.
> >
>
> +1. That test will be helpful.

Sure. Here are results of peak memory usage in bytes reported by
MemoryContextMemAllocated() (when rb->size shows 43MB):

8kB:       52,371,328
16kB:     52,887,424
32kB:     53,428,096
64kB:     55,099,264
128kB:   86,163,328
256kB: 149,340,032
512kB: 273,334,144
1MB:    523,419,520
2MB: 1,021,493,120
4MB: 1,984,085,888
8MB: 2,130,886,528

Probably we can increase the size to 64kB?

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com