pg_restore causing ENOSPACE on the WAL partition. Fundamental issue? - Mailing list pgsql-hackers

From Dimitrios Apostolou
Subject pg_restore causing ENOSPACE on the WAL partition. Fundamental issue?
Date
Msg-id 769dc83a-8279-7217-a84a-998aba2c4a62@gmx.net
Whole thread Raw
Responses Re: pg_restore causing ENOSPACE on the WAL partition. Fundamental issue?
List pgsql-hackers
Hello list,

I have previously raised an issue on pgsql-general, where PostgreSQL is
logging without any boundaries to the WAL directory, even if other writer
processes can't catch up with it. It ends up with WAL partition becoming
full and a bad crash. Read more at the thread at:

https://www.postgresql.org/message-id/flat/076464ad-3d70-dd25-9e8f-e84f27decfba%40gmx.net

If it's true, I consider this a critical and under-documented issue, thus
I'm bringing it up here. Otherwise please let me know if any of my
assumptions are wrong.

In short, this is how it can easily happen, but in principle it's much
more generic:

+ The WAL is on a dedicated high-perf drive

+ pg_restore is writing to a slow tablespace on a network drive

+ data is logged to the WAL faster than what can be written to that
   tablespace

+ it eventually ends up with the WAL filling up, regardless of
   max_wal_size...


In other words, I'm surprised that there is no mechanism for the backends
to block while the WAL is overflowing. Am I wrong here?


Thank in advance,
Dimitris



pgsql-hackers by date:

Previous
From: Junwang Zhao
Date:
Subject: Use RELATION_IS_OTHER_TEMP where possible
Next
From: Andrew Johnson
Date:
Subject: Re: [PATCH v1] Add pg_stat_multixact view for multixact membership usage monitoring