Re: reducing statistics write overhead - Mailing list pgsql-hackers

From Martin Pihlak
Subject Re: reducing statistics write overhead
Date
Msg-id 48C6761E.60900@gmail.com
Whole thread Raw
In response to Re: reducing statistics write overhead  (Alvaro Herrera <alvherre@commandprompt.com>)
List pgsql-hackers
Alvaro Herrera wrote:
>> Attached is a patch that implements the described signalling.
> 
> Are we keeping the idea that the reader sends a stat message when it
> needs to read stats?  What about the lossiness of the transport?
> 

As the message is resent in the wait loop, the collector should receive
it sooner or later. And initial testing shows that its not really easy to
make the collector lose messages.

I used a modified version of the patch to run a simple load test on a 4 core
amd64 Linux box:

- patch modified so that pgstat_send_inquiry() is sent only once - before wait loop, so it times out if message is
lost.
- stats file bloated to ~ 5MB by creating 40k tables.
- 4 pgbench instances running: -c 500 -t 500
- 2 clients constantly pulling stats
- all cores near 100% busy, tx traffic over loopback ~ 200kB/sec.

Most of the stats requests required 1 or 2 file wait iterations (5ms sleep each).
Occasionally 3, but no timeouts (ie. no lost messages). Maybe other platforms are
more sensitive, but I have none available for testing at the moment.

regards,
Martin


pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: [gsmith@gregsmith.com: Re: [patch] GUC source file and line number]
Next
From: Markus Wanner
Date:
Subject: Re: Synchronous Log Shipping Replication