Weird CPU utilization patterns with Postgres - Mailing list pgsql-general

From István
Subject Weird CPU utilization patterns with Postgres
Date
Msg-id CAB25XEXNONdRmC1_cy3jvmB0TMyDm38eF9q2D7xLa0rbnCJ5pQ@mail.gmail.com
Whole thread Raw
Responses Re: Weird CPU utilization patterns with Postgres
List pgsql-general
Hi,

We are having a really interesting problem with our Postgres 9.3 instance in our infrastructure.

Few days ago our box started to show huge CPU spikes while the IO Wait is negligible on the box. After a while I have installed perf and started to monitor the Postgres master process and here is what I have found:

Samples: 372K of event 'cycles', Event count (approx.): 110095222173, ThreaSamples: 372K of event 'cycles', Event count (approx.): 1100 93.65%  libc-2.12.so  [.] __strcoll_l
  0.97%  libc-2.12.so  [.] memcpy
  0.90%  postgres      [.] slot_getattr
  0.88%  postgres      [.] nocachegetattr
  0.64%  postgres      [.] varstr_cmp
  0.52%  libc-2.12.so  [.] __strcmp_sse42
  0.43%  postgres      [.] hash_any
  0.32%  postgres      [.] pg_detoast_datum_packed
  0.31%  libc-2.12.so  [.] __strlen_sse2
  0.22%  postgres      [.] bttextcmp
  0.18%  postgres      [.] ExecStoreTuple
  0.14%  postgres      [.] MemoryContextReset
  0.09%  postgres      [.] pgstat_end_function_usage
  0.08%  libc-2.12.so  [.] strcoll
  0.08%  postgres      [.] heap_hot_search_buffer
  0.07%  postgres      [.] lc_collate_is_c
  0.06%  [kernel]      [k] sys_semtimedop
  0.06%  postgres      [.] heap_page_prune_opt
  0.05%  postgres      [.] slot_getsomeattrs
  0.05%  postgres      [.] heap_fill_tuple
  0.04%  postgres      [.] hash_search
  0.03%  postgres      [.] GetMemoryChunkSpace
  0.03%  postgres      [.] heap_form_minimal_tuple
  0.03%  [kernel]      [k] update_queue
  0.02%  postgres      [.] ReadBufferExtended
  0.02%  postgres      [.] memcpy@plt

It seems that the box is using __strcoll a lot. The query performance is down, while previously the box was able to sustain with ~20 clients right now it is hardly able to keep up with 5.

I am wondering why the root cause might be here.

Let me know if anybody has seen this before.

Regards,
Istvan





--
the sun shines for all


pgsql-general by date:

Previous
From: AJ Welch
Date:
Subject: Re: Use cases for lateral that do not involve a set returning function
Next
From: Peter Geoghegan
Date:
Subject: Re: Weird CPU utilization patterns with Postgres