spinlock->pthread_mutex : first results with Jeff's pgbench+plsql - Mailing list pgsql-hackers

Hi Jeff and all,

apparently, Jeff has indeed been very right on how to provoke spinlock
contention with his patched pgbench.

The following table (view attached pdf if you can) summarizes the results of the
pgbench runs detailed at the bottom of this mail. For some pgbench
configurations, I have tested the original against the patched code:

    - 8/16/32/64/128 threads with one pgbench database
    - 2x128 threads to each of two pgbench databases

The colums are:

tr:        sum of "number of transactions actually processed" output from
        pgbench

sps_ex        sum of "selects per second (excluding connections
        establishing)" from pgbench

u:        usr time as reported by /usr/bin/time

s:        sys time as reported by /usr/bin/time

(u+s)/tr    resource consumption normalized by the number of
        transactions processed


The p/o row gives the patched-to-unpatched ratio of the respective column in percent


TEST    tr    sps_ex        u        s        (u+s)/tr

8 threads 1 DB
o    1080    178,768.20      461.65       43.40      0.47
p     328     53,967.07      345.73      132.51      1.46
p/o             30.19%       74.89%      305.32%    311.79%

16 threads 1 DB
o     764    124,476.93      899.84      103.89      1.31
p     178     29,142.80      402.85      333.87      4.14
p/o             23.41%       44.77%      321.37%    315.04%

32 threads 1 DB
o     228     36,434.82    1,647.15      175.14      7.99
p     190     27,193.83      428.31      585.52      5.34
p/o             74.64%       26.00%      334.32%     66.76%


64 threads 1 DB
o     195     29,724.96    2,268.39      439.51     13.89
p     192     24,127.50      458.46      886.33      7.00
p/o             81.17%       20.21%      201.66%     50.44%

128 threads 1 DB
o     128     16,638.27    2,780.02      322.67     24.24
p     256     22,756.89      592.34    1,114.39      6.67
p/o            136.77%       21.31%      345.37%     27.50%

128 threads 2 DBs
o     256     10,444.96    9,050.27      523.88     37.40
p     256     21,265.00      667.79    1,052.84      6.72
p/o            203.59%        7.38%      200.97%     17.97%


How I read this under the assumption that the test was correct and valid _and_
can be reproduced independently:

* for very low concurrency, the existing spinlock implementation is ideal -
  we can't do any better both in terms of resulting sps and resource
  consumption.

  One path to explore here would be PTHREAD_MUTEX_ADAPTIVE_NP, which essentially
  is the same as a spinlock for contended case with very low lock aquisition
  time. The code which I have tested uses PTHREAD_MUTEX_NORMAL, which, on Linux,
  will always syscall for the contended case.

  Quite clearly the overhead is with futexes syscalling, because kernel
  resource consumption is 3x higher with the patch than without.

* With this benchmark, for "half" concurrency in the order of 0.5 x #cores,
  spinlocks still yield better tps, but resource overhead for spinlocks starts
  to take off and futexes are already 40% more efficient, despite the fact that
  spinlocks still have a 25% advantage in terms of sps.

* At "full" concurrency (64 threads on 64 cores), resource consumption of
  the spinlocks leads to almost doubled overall resource consumption and
  the increased efficiency starts to pay off in terms of sps

* and for the "quadruple overloaded" case (2x128 threads on 64 cores), spinlock
  contention really brings the system down and sps drops to half.

Please note that this is with 3.1.7. I understand that many scalability
improvements have been implemented in later versions and it would have to be
expected that using less synchronization points will imply that spinlock
overhead is less. In other words, the results _should_ look less drastic with
later versions.

Please do also take note of the custom minimum time quantum scheduler
parameters, which could have a significant impact on the result (and I have not
yet repeated this benchmark with the defaults). Regarding the actual production
issue, though, I had started with default parameters and only changed them
later. The adjusted values appeared to have positive effects.


Having said this, I take the benchmark results as a clear indication that
postgresql scalability issues at least in 9.1 are highly related to spinlock
contention on large systems.

My personal opinion is that a system which behaves in a stable and predictable
manner also in high load situations should be preferred over one which has
optimal results for the best case, but I could imagine that with a custom futex
implementation we could have a more adaptive approach and maybe even get the
benefits without much of the drawbacks. Maybe one could have stats on individual
spinlocks and adjust the amount of "adaptive spinning" before deciding to syscall?

A simpler approach could be a global config variable for the type of lock to use
for traditional spinlocks. At this point, I could imagine:

    - traditional s_lock
    - PTHREAD_MUTEX_ADAPTIVE_NP
    - PTHREAD_MUTEX_NORMAL

== SETUP DETAILS ==

=== Initialization ===

cd /usr/pgsql-9.1noslock/bin
./initdb -D /var/tmp/data_jeff_bench
for i in {1..6} ; do \
    echo "create database bench_scale100_$i;" ; done \
        | ./psql -p 55432 postgres
for i in {1..6} ; do \
    ./pgbench -p 55432 -i -s 100 bench_scale100_$i &
done

# stop postgres
mv /var/tmp/data_jeff_bench /var/tmp/data_jeff_bench_template

=== pgsql config ===

$ egrep '^[a-z]' /var/tmp/data_jeff_bench_template/postgresql.conf
max_connections = 1800            # (change requires restart)
shared_buffers = 10GB            # min 128kB
temp_buffers = 64MB            # min 800kB
work_mem = 256MB            # min 64kB,d efault 1MB
maintenance_work_mem = 2GB        # min 1MB, default 16MB
bgwriter_delay = 10ms            # 10-10000ms between rounds
bgwriter_lru_maxpages = 1000        # 0-1000 max buffers written/round
bgwriter_lru_multiplier = 10.0        # 0-10.0 multipler on buffers scanned/round
wal_level = hot_standby            # minimal, archive, or hot_standby
wal_buffers = 64MB            # min 32kB, -1 sets based on shared_buffers
fsync = off                # turns forced synchronization on or off
commit_delay = 10000            # range 0-100000, in microseconds
datestyle = 'iso, mdy'
lc_messages = 'en_US.UTF-8'            # locale for system error message
lc_monetary = 'en_US.UTF-8'            # locale for monetary formatting
lc_numeric = 'en_US.UTF-8'            # locale for number formatting
lc_time = 'en_US.UTF-8'                # locale for time formatting
default_text_search_config = 'pg_catalog.english'
seq_page_cost = 1.0            # measured on an arbitrary scale
random_page_cost = 1.5            # same scale as above (default: 4.0)
cpu_tuple_cost = 0.005
cpu_index_tuple_cost = 0.0025
cpu_operator_cost = 0.0001
effective_cache_size = 192GB

== RUN DETAILS ==

used attached "bench.sh"
the pwd output indicates the postgresql server binaries used
original binaries are

running all processes with nice 19

scheduler settings:

#                                         s ms us ns
kernel.sched_min_granularity_ns         =  100000000
kernel.sched_wakeup_granularity_ns      =  100000000
kernel.sched_latency_ns                 = 1000000000
# can migrate to all CPUs
kernel.sched_nr_migrate                 = 64

=== 8 threads 1 DB original code ===

++ pwd
/usr/pgsql-9.1/bin
++ ppid=6955
++ /usr/bin/time ./postgres -D /mnt/db1/ssd/tables/.t/data_jeff_bench/ -p 55432
+++ pgrep -P 6955
++ pid=6957
++ sleep 15
LOG:  database system was shut down at 2012-07-02 11:30:16 CEST
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
++ /usr/pgsql-9.1noslock/bin/pgbench -c8 -j8 -P -T60 -s100 -p 55432 bench_scale100_1
Scale option ignored, using pgbench_branches table count = 100
plgsql function created.
starting vacuum...end.
transaction type: SELECT only via plpgsql
scaling factor: 100
query mode: simple
number of clients: 8
number of threads: 8
duration: 60 s
number of transactions actually processed: 1080
tps = 17.869689 (including connections establishing)
tps = 17.876820 (excluding connections establishing)
selects per second = 178696.891635 (including connections establishing)
selects per second = 178768.198473 (excluding connections establishing)
++ kill 6957
++ wait
LOG:  received smart shutdown request
LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
461.65user 43.40system 1:19.95elapsed 631%CPU (0avgtext+0avgdata
6326144maxresident)k
0inputs+4248outputs (0major+3287417minor)pagefaults 0swaps

=== 8 threads 1 DB patched ===

++ pwd
/usr/pgsql-9.1noslock/bin
++ ppid=10346
++ /usr/bin/time ./postgres -D /mnt/db1/ssd/tables/.t/data_jeff_bench/ -p 55432
+++ pgrep -P 10346
++ pid=10348
++ sleep 15
LOG:  database system was shut down at 2012-07-02 11:30:16 CEST
LOG:  autovacuum launcher started
LOG:  database system is ready to accept connections
++ /usr/pgsql-9.1noslock/bin/pgbench -c8 -j8 -P -T60 -s100 -p 55432 bench_scale100_1
Scale option ignored, using pgbench_branches table count = 100
plgsql function created.
starting vacuum...end.
transaction type: SELECT only via plpgsql
scaling factor: 100
query mode: simple
number of clients: 8
number of threads: 8
duration: 60 s
number of transactions actually processed: 328
tps = 5.394723 (including connections establishing)
tps = 5.396707 (excluding connections establishing)
selects per second = 53947.230891 (including connections establishing)
selects per second = 53967.065956 (excluding connections establishing)
++ kill 10348
++ wait
LOG:  received smart shutdown request
LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
345.73user 132.51system 1:21.42elapsed 587%CPU (0avgtext+0avgdata
5798640maxresident)k
0inputs+4240outputs (0major+3051296minor)pagefaults 0swaps


=== 16 threads 1 DB original code ===

++ pwd
/usr/pgsql-9.1/bin
++ ppid=54955
++ /usr/bin/time ./postgres -D /mnt/db1/ssd/tables/.t/data_jeff_bench/ -p 55432
+++ pgrep -P 54955
++ pid=54957
++ sleep 15
LOG:  database system was shut down at 2012-07-02 11:30:16 CEST
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
++ /usr/pgsql-9.1noslock/bin/pgbench -c16 -j16 -P -T60 -s100 -p 55432
bench_scale100_1
Scale option ignored, using pgbench_branches table count = 100
plgsql function created.
starting vacuum...end.
transaction type: SELECT only via plpgsql
scaling factor: 100
query mode: simple
number of clients: 16
number of threads: 16
duration: 60 s
number of transactions actually processed: 764
tps = 12.441953 (including connections establishing)
tps = 12.447693 (excluding connections establishing)
selects per second = 124419.525508 (including connections establishing)
selects per second = 124476.932381 (excluding connections establishing)
++ kill 54957
++ wait
LOG:  received smart shutdown request
LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
899.84user 103.89system 1:22.65elapsed 1214%CPU (0avgtext+0avgdata
6040912maxresident)k
0inputs+4224outputs (0major+5718494minor)pagefaults 0swaps


=== 16 threads 1 DB patched ===

++ pwd
/usr/pgsql-9.1noslock/bin
++ ppid=57669
++ /usr/bin/time ./postgres -D /mnt/db1/ssd/tables/.t/data_jeff_bench/ -p 55432
+++ pgrep -P 57669
++ pid=57671
++ sleep 15
LOG:  database system was shut down at 2012-07-02 11:30:16 CEST
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
++ /usr/pgsql-9.1noslock/bin/pgbench -c16 -j16 -P -T60 -s100 -p 55432
bench_scale100_1
Scale option ignored, using pgbench_branches table count = 100
plgsql function created.
starting vacuum...end.
transaction type: SELECT only via plpgsql
scaling factor: 100
query mode: simple
number of clients: 16
number of threads: 16
duration: 60 s
number of transactions actually processed: 178
tps = 2.912747 (including connections establishing)
tps = 2.914280 (excluding connections establishing)
selects per second = 29127.471536 (including connections establishing)
selects per second = 29142.795859 (excluding connections establishing)
++ kill 57671
++ wait
LOG:  received smart shutdown request
LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
402.85user 333.87system 1:19.92elapsed 921%CPU (0avgtext+0avgdata
3896960maxresident)k
0inputs+4248outputs (0major+3968573minor)pagefaults 0swaps


=== 32 threads 1 DB original code ===

++ pwd
/usr/pgsql-9.1/bin
++ ppid=61921
++ /usr/bin/time ./postgres -D /mnt/db1/ssd/tables/.t/data_jeff_bench/ -p 55432
+++ pgrep -P 61921
++ pid=61923
++ sleep 15
LOG:  database system was shut down at 2012-07-02 11:30:16 CEST
LOG:  autovacuum launcher started
LOG:  database system is ready to accept connections
++ /usr/pgsql-9.1noslock/bin/pgbench -c32 -j32 -P -T60 -s100 -p 55432
bench_scale100_1
Scale option ignored, using pgbench_branches table count = 100
plgsql function created.
starting vacuum...end.
transaction type: SELECT only via plpgsql
scaling factor: 100
query mode: simple
number of clients: 32
number of threads: 32
duration: 60 s
number of transactions actually processed: 228
tps = 3.641156 (including connections establishing)
tps = 3.643482 (excluding connections establishing)
selects per second = 36411.559667 (including connections establishing)
selects per second = 36434.817110 (excluding connections establishing)
++ kill 61923
++ wait
LOG:  received smart shutdown request
LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
1647.15user 175.14system 1:23.49elapsed 2182%CPU (0avgtext+0avgdata
3103952maxresident)k
0inputs+4224outputs (0major+5683435minor)pagefaults 0swaps


=== 32 threads 1 DB patched ===

++ pwd
/usr/pgsql-9.1noslock/bin
++ ppid=1051
++ /usr/bin/time ./postgres -D /mnt/db1/ssd/tables/.t/data_jeff_bench/ -p 55432
+++ pgrep -P 1051
++ pid=1053
++ sleep 15
LOG:  database system was shut down at 2012-07-02 11:30:16 CEST
LOG:  autovacuum launcher started
LOG:  database system is ready to accept connections
++ /usr/pgsql-9.1noslock/bin/pgbench -c32 -j32 -P -T60 -s100 -p 55432
bench_scale100_1
Scale option ignored, using pgbench_branches table count = 100
plgsql function created.
starting vacuum...end.
transaction type: SELECT only via plpgsql
scaling factor: 100
query mode: simple
number of clients: 32
number of threads: 32
duration: 60 s
number of transactions actually processed: 190
tps = 2.717758 (including connections establishing)
tps = 2.719383 (excluding connections establishing)
selects per second = 27177.577274 (including connections establishing)
selects per second = 27193.834024 (excluding connections establishing)
++ kill 1053
++ wait
LOG:  received smart shutdown request
LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
428.31user 585.52system 1:32.30elapsed 1098%CPU (0avgtext+0avgdata
2836128maxresident)k
0inputs+4448outputs (0major+5835601minor)pagefaults 0swaps


=== 64 threads 1 DB original code ===

++ pwd
/usr/pgsql-9.1/bin
++ ppid=8220
++ /usr/bin/time ./postgres -D /mnt/db1/ssd/tables/.t/data_jeff_bench/ -p 55432
+++ pgrep -P 8220
++ pid=8222
++ sleep 15
LOG:  database system was shut down at 2012-07-02 11:30:16 CEST
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
++ /usr/pgsql-9.1noslock/bin/pgbench -c64 -j64 -P -T60 -s100 -p 55432
bench_scale100_1
Scale option ignored, using pgbench_branches table count = 100
plgsql function created.
starting vacuum...end.

transaction type: SELECT only via plpgsql
scaling factor: 100
query mode: simple
number of clients: 64
number of threads: 64
duration: 60 s
number of transactions actually processed: 195
tps = 2.969237 (including connections establishing)
tps = 2.972496 (excluding connections establishing)
selects per second = 29692.373760 (including connections establishing)
selects per second = 29724.961208 (excluding connections establishing)
++ kill 8222
++ wait
LOG:  received smart shutdown request
LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
2268.39user 439.51system 1:30.66elapsed 2986%CPU (0avgtext+0avgdata
2050464maxresident)k
0inputs+4432outputs (0major+7134606minor)pagefaults 0swaps

=== 64 threads 1 DB patched ===

++ pwd
/usr/pgsql-9.1noslock/bin
++ ppid=12542
++ /usr/bin/time ./postgres -D /mnt/db1/ssd/tables/.t/data_jeff_bench/ -p 55432
+++ pgrep -P 12542
++ pid=12544
++ sleep 15
LOG:  database system was shut down at 2012-07-02 11:30:16 CEST
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
++ /usr/pgsql-9.1noslock/bin/pgbench -c64 -j64 -P -T60 -s100 -p 55432
bench_scale100_1
Scale option ignored, using pgbench_branches table count = 100
plgsql function created.
starting vacuum...end.
transaction type: SELECT only via plpgsql
scaling factor: 100
query mode: simple
number of clients: 64
number of threads: 64
duration: 60 s
number of transactions actually processed: 192
tps = 2.410056 (including connections establishing)
tps = 2.412750 (excluding connections establishing)
selects per second = 24100.559887 (including connections establishing)
selects per second = 24127.504716 (excluding connections establishing)
++ kill 12544
++ wait
LOG:  received smart shutdown request
LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
458.46user 886.33system 1:45.37elapsed 1276%CPU (0avgtext+0avgdata
2003472maxresident)k
0inputs+3856outputs (0major+7998883minor)pagefaults 0swaps


=== 128 threads 1 DB original code ===

++ pwd
/usr/pgsql-9.1/bin
++ ppid=46191
++ /usr/bin/time ./postgres -D /mnt/db1/ssd/tables/.t/data_jeff_bench/ -p 55432
+++ pgrep -P 46191
++ pid=46193
++ sleep 15
LOG:  database system was shut down at 2012-07-02 11:30:16 CEST
LOG:  autovacuum launcher started
LOG:  database system is ready to accept connections
++ /usr/pgsql-9.1noslock/bin/pgbench -c128 -j128 -P -T60 -s100 -p 55432
bench_scale100_1
Scale option ignored, using pgbench_branches table count = 100
plgsql function created.
starting vacuum...end.
transaction type: SELECT only via plpgsql
scaling factor: 100
query mode: simple
number of clients: 128
number of threads: 128
duration: 60 s
number of transactions actually processed: 128
tps = 1.661937 (including connections establishing)
tps = 1.663827 (excluding connections establishing)
selects per second = 16619.373297 (including connections establishing)
selects per second = 16638.268091 (excluding connections establishing)
++ kill 46193
++ wait
LOG:  received smart shutdown request
LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
2780.02user 322.67system 1:35.79elapsed 3238%CPU (0avgtext+0avgdata
1368576maxresident)k
0inputs+4400outputs (0major+7112920minor)pagefaults 0swaps


=== 128 threads 1 DB patched ===

++ pwd
/usr/pgsql-9.1noslock/bin
++ ppid=50278
++ /usr/bin/time ./postgres -D /mnt/db1/ssd/tables/.t/data_jeff_bench/ -p 55432
+++ pgrep -P 50278
++ pid=50280
++ sleep 15
LOG:  database system was shut down at 2012-07-02 11:30:16 CEST
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
++ /usr/pgsql-9.1noslock/bin/pgbench -c128 -j128 -P -T60 -s100 -p 55432
bench_scale100_1
Scale option ignored, using pgbench_branches table count = 100
plgsql function created.
starting vacuum...end.
transaction type: SELECT only via plpgsql
scaling factor: 100
query mode: simple
number of clients: 128
number of threads: 128
duration: 60 s
number of transactions actually processed: 256
tps = 2.273227 (including connections establishing)
tps = 2.275689 (excluding connections establishing)
selects per second = 22732.267812 (including connections establishing)
selects per second = 22756.889258 (excluding connections establishing)
++ kill 50280
++ wait
LOG:  received smart shutdown request
LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
592.34user 1114.39system 2:12.96elapsed 1283%CPU (0avgtext+0avgdata
2003472maxresident)k
0inputs+3744outputs (0major+12355329minor)pagefaults 0swaps


=== 2x128 threads 2 DB original code ===

++ pwd
/usr/pgsql-9.1/bin
++ ppid=9841
++ /usr/bin/time ./postgres -D /mnt/db1/ssd/tables/.t/data_jeff_bench/ -p 55432
+++ pgrep -P 9841
++ pid=9843
++ sleep 15
LOG:  database system was shut down at 2012-07-02 11:30:16 CEST
LOG:  autovacuum launcher started
LOG:  database system is ready to accept connections
++ typeset -a benchpids
++ for db in 'bench_scale100_{1..2}'
++ benchpids[${#benchpids[@]}]=10123
++ for db in 'bench_scale100_{1..2}'
++ /usr/pgsql-9.1noslock/bin/pgbench -c128 -j128 -P -T60 -s100 -p 55432
bench_scale100_1
++ benchpids[${#benchpids[@]}]=10124
++ wait 10123 10124
++ /usr/pgsql-9.1noslock/bin/pgbench -c128 -j128 -P -T60 -s100 -p 55432
bench_scale100_2
Scale option ignored, using pgbench_branches table count = 100
plgsql function created.
starting vacuum...Scale option ignored, using pgbench_branches table count = 100
plgsql function created.
starting vacuum...end.
end.
transaction type: SELECT only via plpgsql
scaling factor: 100
query mode: simple
number of clients: 128
number of threads: 128
duration: 60 s
number of transactions actually processed: 128
tps = 0.522012 (including connections establishing)
tps = 0.522310 (excluding connections establishing)
selects per second = 5220.116915 (including connections establishing)
selects per second = 5223.104689 (excluding connections establishing)
transaction type: SELECT only via plpgsql
scaling factor: 100
query mode: simple
number of clients: 128
number of threads: 128
duration: 60 s
number of transactions actually processed: 128
tps = 0.521770 (including connections establishing)
tps = 0.522186 (excluding connections establishing)
selects per second = 5217.696100 (including connections establishing)
selects per second = 5221.859622 (excluding connections establishing)
++ kill 9843
++ wait
LOG:  received smart shutdown request
LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
9050.27user 523.88system 4:25.05elapsed 3612%CPU (0avgtext+0avgdata
1368560maxresident)k
0inputs+4304outputs (0major+15846016minor)pagefaults 0swaps


=== 2x128 threads 2 DB patched ===

++ pwd
/usr/pgsql-9.1noslock/bin
++ ppid=1494
++ /usr/bin/time ./postgres -D /mnt/db1/ssd/tables/.t/data_jeff_bench/ -p 55432
+++ pgrep -P 1494
++ pid=1496
++ sleep 15
LOG:  database system was shut down at 2012-07-02 11:30:16 CEST
LOG:  autovacuum launcher started
LOG:  database system is ready to accept connections
++ typeset -a benchpids
++ for db in 'bench_scale100_{1..2}'
++ benchpids[${#benchpids[@]}]=2044
++ for db in 'bench_scale100_{1..2}'
++ /usr/pgsql-9.1noslock/bin/pgbench -c128 -j128 -P -T60 -s100 -p 55432
bench_scale100_1
++ benchpids[${#benchpids[@]}]=2045
++ wait 2044 2045
++ /usr/pgsql-9.1noslock/bin/pgbench -c128 -j128 -P -T60 -s100 -p 55432
bench_scale100_2
Scale option ignored, using pgbench_branches table count = 100
Scale option ignored, using pgbench_branches table count = 100
plgsql function created.
starting vacuum...plgsql function created.
starting vacuum...end.
end.
transaction type: SELECT only via plpgsql
scaling factor: 100
query mode: simple
number of clients: 128
number of threads: 128
duration: 60 s
number of transactions actually processed: 128
tps = 1.086408 (including connections establishing)
tps = 1.088396 (excluding connections establishing)
selects per second = 10864.079761 (including connections establishing)
selects per second = 10883.964930 (excluding connections establishing)
transaction type: SELECT only via plpgsql
scaling factor: 100
query mode: simple
number of clients: 128
number of threads: 128
duration: 60 s
number of transactions actually processed: 128
tps = 1.035884 (including connections establishing)
tps = 1.038104 (excluding connections establishing)
selects per second = 10358.836908 (including connections establishing)
selects per second = 10381.039715 (excluding connections establishing)
++ kill 1496
++ wait
LOG:  received smart shutdown request
LOG:  autovacuum launcher shutting down
FATAL:  terminating autovacuum process due to administrator command
LOG:  shutting down
LOG:  database system is shut down
667.79user 1052.84system 2:24.75elapsed 1188%CPU (0avgtext+0avgdata
2003456maxresident)k
0inputs+3968outputs (0major+18333314minor)pagefaults 0swaps