[HACKERS] [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking inserializable transactions - Mailing list pgsql-hackers

From Mengxing Liu
Subject [HACKERS] [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking inserializable transactions
Date
Msg-id 7b6cd04e.1ea9.15becbd5a37.Coremail.liu-mx15@mails.tsinghua.edu.cn
Whole thread Raw
Responses [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking inserializable transactions  (Alvaro Herrera <alvherre@2ndquadrant.com>)
List pgsql-hackers

Hi, Alvaro and Kevin. I'm  Mengxing.  

This is a “synchronization” email to  tell you what I've done and my next plan. I'm looking forward to your advice. 


According to my proposal, I want to prepare the experimental environment during the community bonding period. 

As this is the first time I discuss with Alvaro, here I will introduce the environment again. 

My lab have a Lenovo System x3950 X6 machine. 

https://www.lenovo.com/images/products/system-x/pdfs/datasheets/x3950_x6_ds.pdf

More specifically, there are 8 sockets, each has 15 Intel(R) Xeon(R) CPU E7-8870 v2 @ 2.30GHz. 

Thus we have 120 cores in total. The storage is a 1 TB SSD, with SAS interface, 500MB write bandwidth. 

OS is ubuntu 14.04. 


I've compiled & installed PostgreSQL and have tested it by using pgbench. I tuned parameter for database according to

http://www.dbdude.com/postgresql/postgresqlperformance.php#BESTPRACTICES

For pgbench, I set scale factor (-s) 512. Other configurations are default values. 


Because we have too many cores, SSD becomes the bottleneck. In my test, pgbench can scale to 36 connections. ( 18 KTPS throughput). CPU utilization is smaller than 30%.  p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 19.0px; font: 13.0px 'Helvetica Neue'} p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 19.0px; font: 13.0px 'Helvetica Neue'; min-height: 15.0px}

Therefore:

1. Is there anything wrong in my tuning parameters?For example, should I close "fsync"? Because we don't care recovery here. 

2. Can we use just two sockets (30 physical cores) to run database server? Then the CPU can be the bottleneck so that it  shows the problem we try to solve.


p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 19.0px; font: 13.0px 'Helvetica Neue'}


p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 19.0px; font: 13.0px 'Helvetica Neue'} p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 19.0px; font: 13.0px 'Helvetica Neue'; min-height: 15.0px}

BTW. Here is the question of Stephen. 

>  What method of communication will be used among the mentors and with Mengxing.

What method do you prefer?


>  Frequency of status updates (must be at least once a week and more often is encouraged).

How about reporting my status once a week?


>  What steps will be taken next during the community bonding period.

As I wrote in the proposal, I want to establish the environment and read the related source code. Do you have any suggestions for me?


--
Mengxing Liu






pgsql-hackers by date:

Previous
From: Ashutosh Bapat
Date:
Subject: Re: [HACKERS] Bug in pg_dump --table and --exclude-table fordeclarative partition table handling.
Next
From: Thomas Munro
Date:
Subject: Re: [HACKERS] transition table behavior with inheritance appearsbroken (was: Declarative partitioning - another take)