Re: pg_restore causing ENOSPACE on the WAL partition. Fundamental issue? - Mailing list pgsql-hackers

From David G. Johnston
Subject Re: pg_restore causing ENOSPACE on the WAL partition. Fundamental issue?
Date
Msg-id CAKFQuwZCAoasGyBdmVg8GQdT3P-5tFundHpDoxJaXR8tSkdkbQ@mail.gmail.com
Whole thread Raw
In response to pg_restore causing ENOSPACE on the WAL partition. Fundamental issue?  (Dimitrios Apostolou <jimis@gmx.net>)
List pgsql-hackers
On Tuesday, June 10, 2025, Dimitrios Apostolou <jimis@gmx.net> wrote:
Hello list,

I have previously raised an issue on pgsql-general, where PostgreSQL is logging without any boundaries to the WAL directory, even if other writer processes can't catch up with it. It ends up with WAL partition becoming full and a bad crash. Read more at the thread at:

https://www.postgresql.org/message-id/flat/076464ad-3d70-dd25-9e8f-e84f27decfba%40gmx.net

If it's true, I consider this a critical and under-documented issue, thus I'm bringing it up here. Otherwise please let me know if any of my assumptions are wrong.

In short, this is how it can easily happen, but in principle it's much more generic:

+ The WAL is on a dedicated high-perf drive

+ pg_restore is writing to a slow tablespace on a network drive

+ data is logged to the WAL faster than what can be written to that
  tablespace

+ it eventually ends up with the WAL filling up, regardless of
  max_wal_size...


In other words, I'm surprised that there is no mechanism for the backends to block while the WAL is overflowing. Am I wrong here?


Probably not.  max_wal_size is documented as being a soft limit that can be exceeded due to this very thing.  If the performance of your data disk is so much worse than your WAL disk that you cannot checkpoint fast enough to prevent WAL from overflowing its disk during that period then you do indeed have an issue that PostgreSQL doesn’t provide protections for - it’s being optimistic and helpful to the point of letting you hang yourself with the rope it’s given you.

David J.

pgsql-hackers by date:

Previous
From: Álvaro Herrera
Date:
Subject: Re: queryId constant squashing does not support prepared statements
Next
From: Andres Freund
Date:
Subject: Re: Non-reproducible AIO failure