Thread: Posgresql 14 and CarbonBlack on RHEL8?
(CarbonBlack is cross-platform AV software sold by VMware.)
Both old and new VMs are 32 CPU with 128GB RAM.
Nothing but PG, CB and itsm software runs on these systems.
When running stress tests on the systems (in prod, during the maintenance window), 171K events/second are generated on the RHEL8 servers, and CB needs (according to top(1)) 325% of CPU to handle that, and still dropping 92% of them.
The RHEL6 system doesn't bat an eye at running the exact same test (36 cron jobs running psql executing SELECT statements).
The small RHEL8/PG14 non-prod systems show similar load when lots of SELECT statements run.
Has anyone else seen this? If so, how did you resolve it?
Ron Johnson <ronljohnsonjr@gmail.com> writes:
> When running stress tests on the systems (in prod, during the maintenance
> window), 171K events/second are generated on the RHEL8 servers, and CB
> needs (according to top(1)) 325% of CPU to handle that, and still dropping
> 92% of them.
> The RHEL6 system doesn't bat an eye at running the exact same test (36 cron
> jobs running psql executing SELECT statements).
Is JIT enabled on the newer system? If so try turning it off, or else
raise the associated cost settings. We've seen lots of reports of
workloads where, by default, the planner is too aggressive about
applying JIT.
regards, tom lane
On Tue, Apr 30, 2024 at 10:07 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Ron Johnson <ronljohnsonjr@gmail.com> writes:
> When running stress tests on the systems (in prod, during the maintenance
> window), 171K events/second are generated on the RHEL8 servers, and CB
> needs (according to top(1)) 325% of CPU to handle that, and still dropping
> 92% of them.
> The RHEL6 system doesn't bat an eye at running the exact same test (36 cron
> jobs running psql executing SELECT statements).
Is JIT enabled on the newer system? If so try turning it off, or else
raise the associated cost settings. We've seen lots of reports of
workloads where, by default, the planner is too aggressive about
applying JIT.
A puzzling suggestion. Why should it impact AV software?
At one point, I disabled JIT to test its impact on PG, performance was a bit of a wash (some queries were a bit faster, some were a bit slower), but I didn't monitor CB.
Just now, I did ALTER SYSTEM SET jit='off'; and re-ran the stress test. No impact to CarbonBlack.