On Tue, Sep 5, 2017 at 9:40 PM, Vladimir Borodin <root@simply.name> wrote:
We do compress WALs and send them over network. Doing it via archive_command in single thread is sometimes slower than new WALs are written under heavy load.
Ah, yeah, true. I do use pg_receivexlog --compress for that locally and do a bulk copy of only the compressed WALs needed, when needed... So there is a guarantee that completed segments are durable locally, which is very useful.
It seems that option --compress appeared only in postgres 10 which is not ready for production yet. BTW I assume that pg_receivexlog is single-threaded too? So it still may be the bottleneck when 3-5 WALs per second are written.
You should definitely avoid putting that in PGDATA though, the same counts for tablespaces within PGDATA for example.
I would love to but there might be some problems with archiving and in many cases the only partition with enough space to accumulate WALs is partition for PGDATA.
