Re: PATCH: logical_work_mem and logical streaming of largein-progress transactions - Mailing list pgsql-hackers

From Alvaro Herrera
Subject Re: PATCH: logical_work_mem and logical streaming of largein-progress transactions
Date
Msg-id 20190929173044.GA32051@alvherre.pgsql
Whole thread Raw
In response to Re: PATCH: logical_work_mem and logical streaming of largein-progress transactions  (Amit Kapila <amit.kapila16@gmail.com>)
Responses Re: PATCH: logical_work_mem and logical streaming of largein-progress transactions
List pgsql-hackers
On 2019-Sep-29, Amit Kapila wrote:

> On Sun, Sep 29, 2019 at 12:39 AM Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:

> > So that's what I did in the attached patch - I've renamed the GUC to
> > logical_decoding_work_mem, detached it from m_w_m and set the default to
> > 64MB (i.e. the same default as m_w_m).
> 
> Fair enough, let's not argue more on this unless someone else wants to
> share his opinion.

I just read this part of the conversation and I agree that having a
separate GUC with its own value independent from other GUCs is a good
solution.  Tying it to m_w_m seemed reasonable, but it's true that
people frequently set m_w_m very high, and it would be undesirable to
propagate that value to logical decoding memory usage.


I wonder what would constitute good advice on how to set this value, I
mean what is the metric that the user needs to be thinking about.   Is
it the total of memory required to keep all concurrent write transactions 
in memory?  (Quick example: if you do 2048 wTPS and each transaction
lasts 1s, and each transaction does 1kB of logically-decoded changes,
then ~2MB are sufficient for the average case.  Is that correct?  I
*think* that full-page images do not count, correct?  With these things
in mind users could go through pg_waldump output and figure out what to
set the value to.)

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



pgsql-hackers by date:

Previous
From: Fujii Masao
Date:
Subject: Re: Standby accepts recovery_target_timeline setting?
Next
From: Tomas Vondra
Date:
Subject: Re: PATCH: logical_work_mem and logical streaming of largein-progress transactions