Re: posix_fadvise() and pg_receivexlog - Mailing list pgsql-hackers

From Robert Haas
Subject Re: posix_fadvise() and pg_receivexlog
Date
Msg-id CA+TgmobuMcGta+M_Y5qxLCGZWUEFN2Zb4cfuUGs6mnGBQsmR3Q@mail.gmail.com
Whole thread Raw
In response to Re: posix_fadvise() and pg_receivexlog  (didier <did447@gmail.com>)
List pgsql-hackers
On Tue, Sep 9, 2014 at 8:07 AM, didier <did447@gmail.com> wrote:
>> Well, I'd like to hear someone from the field complaining that
>> pg_receivexlog is thrashing the cache and thus reducing the performance of
>> some other process. Or a least a synthetic test case that demonstrates that
>> happening.
> It's not with pg_receivexlog but it's related.
>
> On a small box without replication server connected perfs were good
> enough but not so with a replication server connected, there was 1GB
> worth of WAL sitting in RAM vs next to nothing without slave!
> setup:
> 8GB RAM
> 2GB shared_buffers (smaller has other issues)
> checkpoint_segments 40 (smaller value trigger too much xlog checkpoint)
> checkpoints spread over 10 mn and write 30 to 50% of shared buffers.
> live data set fit in RAM.
> constant load.
>
> On startup (1 or 2/hour) applications were running requests on cold
> data which were now saturating IO.
> I'm not sure it's an OS bug as the WAL were 'hotter' than the cold data.
>
> A cron task every minute with vmtouch -e for evicting old WAL files
> from memory has solved the issue.

That seems like pretty good evidence that it might be worth doing
something here.  But I still think maybe it should be optional,
because if the user plans to reread those files and, say, copy them
somewhere else, then they won't want this behavior.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: Spinlocks and compiler/memory barriers
Next
From: Tom Lane
Date:
Subject: Re: Built-in binning functions