Thread: Delay in converting logs from ready state to done state
Team,
we are seeing delay in converting logs from ready state to done state in pg_xlog archive status.
we have seen pg_xlog generated 2000 per hour and it is getting archived 1894. So the speed at which the archiving is done is too slow as compare to the pg_xlog generation
So our pg_xlog directory keeps filling regularly. What should be the real cause here?
We cannot see any specific error on pg_log except no space left on device.
current setting:
wal_level = archive
archive_mode = on
max_wal_senders = 3
archive_command = 'gzip < %p > /pgarchive/%f'
checkpoint_segments = 3
checkpoint_timeout = 5min
log_checkpoints = on
archive_timeout = 60
Thanks,
Samir Magar
On 12/10/2016 07:26, Samir Magar wrote: > Team, > > we are seeing delay in converting logs from ready state to done state in > pg_xlog archive status. > > we have seen pg_xlog generated 2000 per hour and it is getting archived > 1894. So the speed at which the archiving is done is too slow as compare > to the pg_xlog generation > > So our pg_xlog directory keeps filling regularly. What should be the > real cause here? > > We cannot see any specific error on pg_log except no space left on device. > > > > current setting: > > wal_level = archive > > archive_mode = on > > max_wal_senders = 3 > > archive_command = 'gzip < %p > /pgarchive/%f' > You could use pigz which is parallel, that could speed up compression. > checkpoint_segments = 3 > this is way to low. If you generate 2000 WAL per hour, you should configure it to something like 170 (or 5 min average if 2000 is a spike). It'll perform less checkpoint and also generate less WALs. -- Julien Rouhaud http://dalibo.com - http://dalibo.org
Hello Julien,
Thank you for your prompt response!
we have changed the checkpoint_segment to 170 and use pigz for the pg_xlog compress.
It is working very well now.
Thanks again!!
Regards,
Samir Magar
On Wed, Oct 12, 2016 at 11:33 AM, Julien Rouhaud <julien.rouhaud@dalibo.com> wrote:
On 12/10/2016 07:26, Samir Magar wrote:
> Team,
>
> we are seeing delay in converting logs from ready state to done state in
> pg_xlog archive status.
>
> we have seen pg_xlog generated 2000 per hour and it is getting archived
> 1894. So the speed at which the archiving is done is too slow as compare
> to the pg_xlog generation
>
> So our pg_xlog directory keeps filling regularly. What should be the
> real cause here?
>
> We cannot see any specific error on pg_log except no space left on device.
>
>
>
> current setting:
>
> wal_level = archive
>
> archive_mode = on
>
> max_wal_senders = 3
>
> archive_command = 'gzip < %p > /pgarchive/%f'
>
You could use pigz which is parallel, that could speed up compression.
> checkpoint_segments = 3
>
this is way to low. If you generate 2000 WAL per hour, you should
configure it to something like 170 (or 5 min average if 2000 is a
spike). It'll perform less checkpoint and also generate less WALs.
--
Julien Rouhaud
http://dalibo.com - http://dalibo.org