Re: Parallel Select query performance and shared buffers - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: Parallel Select query performance and shared buffers
Date
Msg-id CAA4eK1L8aZxSOAfJVat4-0iCAFatAjnpp1qbE9L_CtvRZDKj9Q@mail.gmail.com
Whole thread Raw
In response to Re: Parallel Select query performance and shared buffers  (Claudio Freire <klaussfreire@gmail.com>)
List pgsql-hackers
On Wed, Dec 4, 2013 at 10:40 AM, Claudio Freire <klaussfreire@gmail.com> wrote:
> On Wed, Dec 4, 2013 at 12:57 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:
>>> As a quick side, we also repeated the same experiment on an EC2 instance
>>> with 16 CPU cores, and found that the scale out behavior became worse there.
>>> (We also tried increasing the shared_buffers to 30 GB. This change
>>> completely solved the scaling out problem on this instance type, but hurt
>>> our performance on the hi1.4xlarge instances.)
>>
>> Instead of 30GB, you can try with lesser value, but it should be close
>> to your data size.
>
> The OS cache should have provided a similar function.
  The performance cannot be same when those pages are in shared buffers as   a. OS can flush those pages   b. anyway
loadingit again in shared buffers will have some overhead.
 

> In fact, larger shared buffers shouldn't have made a difference if the
> main I/O pattern are sequential scans, because they use a ring buffer.
  Yeah, this is right, but then why he is able to see scaling when he
increased shared buffer's  to larger value.


With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com



pgsql-hackers by date:

Previous
From: Amit Kapila
Date:
Subject: Re: Proof of concept: standalone backend with full FE/BE protocol
Next
From: Ian Lawrence Barwick
Date:
Subject: Re: FDW: possible resjunk columns in AddForeignUpdateTargets