Re: Linux/PostgreSQL scalability issue - problem with 8 cores - Mailing list pgsql-performance

From Bruce Momjian
Subject Re: Linux/PostgreSQL scalability issue - problem with 8 cores
Date
Msg-id 200803220144.m2M1iTe16630@momjian.us
Whole thread Raw
In response to Re: Linux/PostgreSQL scalability issue - problem with 8 cores  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-performance
Added to TODO:

* Improve performance of shared invalidation queue for multiple CPUs

  http://archives.postgresql.org/pgsql-performance/2008-01/msg00023.php


---------------------------------------------------------------------------

Tom Lane wrote:
> Alvaro Herrera <alvherre@commandprompt.com> writes:
> > Perhaps it would make sense to try to take the "fast path" in
> > SIDelExpiredDataEntries with only a shared lock rather than exclusive.
>
> I think the real problem here is that sinval catchup processing is well
> designed to create contention :-(.  Once we've decided that the message
> queue is getting too full, we SIGUSR1 all the backends at once (or as
> fast as the postmaster can do it anyway), then they all go off and try
> to touch the sinval queue.  Backends that haven't awoken even once
> since the last time will have to process the entire queue contents,
> and they're all trying to do that at the same time.  What's worse, they
> take and release the SInvalLock once for each message they take off the
> queue.  This isn't so horrid for one-core machines (since each process
> will monopolize the CPU for probably less than one timeslice while it's
> catching up) but it's pretty obvious where all the contention is coming
> from on an 8-core.
>
> Some ideas for improving matters:
>
> 1. Try to avoid having all the backends hit the queue at once.  Instead
> of SIGUSR1'ing everybody at the same time, maybe hit only the process
> with the oldest message pointer, and have him hit the next oldest after
> he's done reading the queue.
>
> 2. Try to take more than one message off the queue per SInvalLock cycle.
> (There is a tuning tradeoff here, since it would mean holding the lock
> for longer at a time.)
>
> 3. Try to avoid having every backend run SIDelExpiredDataEntries every
> time through ReceiveSharedInvalidMessages.  It's not critical to delete
> entries until the queue starts getting full --- maybe we could rejigger
> the logic so it only happens once when somebody notices the queue is
> getting full, or so that only the guy(s) who had nextMsgNum == minMsgNum
> do it, or something like that?
>
>             regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 7: You can help support the PostgreSQL project by donating at
>
>                 http://www.postgresql.org/about/donate

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

pgsql-performance by date:

Previous
From: Bruce Momjian
Date:
Subject: Re: PostgreSQL NetApp and NFS
Next
From: "sathiya psql"
Date:
Subject: Having MANY MANY empty columns in database