Re: prevent immature WAL streaming - Mailing list pgsql-hackers

From Alvaro Herrera
Subject Re: prevent immature WAL streaming
Date
Msg-id 202110131852.rrxabqmxxoiw@alvherre.pgsql
Whole thread Raw
In response to Re: prevent immature WAL streaming  (Andres Freund <andres@anarazel.de>)
Responses Re: prevent immature WAL streaming  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
On 2021-Oct-13, Andres Freund wrote:

> I added LSNs to the error message:
> not ok 1 - 000000010000000000000002 (0/2002350) differs from 000000010000000000000002 (0/2099600)
> 
> It appears that the problem is that inbetween the determination of
> rows_walsize the insert LSN moved to the next segment separately from the
> insertion, presumably due to autovacuum/analayze or such.
> <retries, with log_autovacuum_min_duration_statement=0, log_min_duration_statement=0>

Oh, of course.

> Hm. I guess we can disable autovac. But that's not a great solution, there
> might be WAL files due to catalog access etc too.

Well, we don't expect anything else to happen -- the cluster is
otherwise idle.  I think we should do it regardless of any other
changes, just to keep things steadier.

> Seems like it might be worth doing the "filling" of the segment with a loop in
> a DO block instead, where the end condition is to be within some distance to
> the end of the segment? With plenty headroom?

Eh, good idea ... didn't think of that, but it should keep things more
stable under strange conditions.

> Another thing: filling a segment by inserting lots of very tiny rows is pretty
> expensive. Can't we use something a bit wider? Perhaps even emit_message?

I think I realized partway through writing the test that I could use
emit_message instead of using a batched row insert ... so, yeah, we
can use it here also.

-- 
Álvaro Herrera              Valdivia, Chile  —  https://www.EnterpriseDB.com/
"La verdad no siempre es bonita, pero el hambre de ella sí"



pgsql-hackers by date:

Previous
From: John Naylor
Date:
Subject: Re: [RFC] building postgres with meson
Next
From: Stephen Frost
Date:
Subject: Re: pg14 psql broke \d datname.nspname.relname