Postgres Pro
Downloads
Home   >   mailing lists

Re: zheap: a new storage format for PostgreSQL - Mailing list pgsql-hackers

From Alexander Korotkov
Subject Re: zheap: a new storage format for PostgreSQL
Date
Msg-id CAPpHfds2hZoJAkkS_+=CsJ8XfsMJws9F2d5f06_2pFa7YaBp0w@mail.gmail.com
Whole thread Raw
In response to zheap: a new storage format for PostgreSQL  (Amit Kapila <amit.kapila16@gmail.com>)
Responses Re: zheap: a new storage format for PostgreSQL  (Amit Kapila <amit.kapila16@gmail.com>)
Re: zheap: a new storage format for PostgreSQL  (Kuntal Ghosh <kuntalghosh.2007@gmail.com>)
List pgsql-hackers
Tree view
On Thu, Mar 1, 2018 at 5:09 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:
Preliminary performance results
-------------------------------------------

We’ve shown the performance improvement of zheap over heap in a few different pgbench scenarios.  All of these tests were run with data that fits in shared_buffers (32GB), and 16 transaction slots per zheap page. Scenario-1 and Scenario-2 has used synchronous_commit = off and Scenario-3 and Scenario-4 has used synchronous_commit = on


What hardware did you use for benchmarks?
Also, I note that you have 4 transaction slots per zheap page in github code while you use 16 in benchmarks.

#define MAX_PAGE_TRANS_INFO_SLOTS 4

I would also note that in the code you preserve only 3 bits for transaction slot number.  So, one have to redefine 3 macros to change transaction slot number to the value you used in the benchmarks.

#define ZHEAP_XACT_SLOT 0x3800 /* 3 bits (12, 13 and 14) for transaction slot */
#define ZHEAP_XACT_SLOT_MASK 0x000B /* 11 - mask to retrieve transaction slot */

I'm only starting reviewing this, but it makes me think that we need transaction slots number to be tunable (or even auto-tunable).

BTW, last two macros don't look properly named for me.  I would rather rename them in a following way:
ZHEAP_XACT_SLOT_MASK => ZHEAP_XACT_SLOT_OFFSET
ZHEAP_XACT_SLOT => ZHEAP_XACT_SLOT_MASK

Scenario 1: A 15 minutes simple-update pgbench test with scale factor 100 shows 5.13% TPS improvement with 64 clients. The performance improvement increases as we increase the scale factor; at scale factor 1000, it reaches11.5% with 64 clients.


Scale Factor

HEAP

ZHEAP (tables)*

Improvement

Before test

100

1281 MB

1149 MB

-10.3%

1000

13 GB

11 GB

-15.38%

After test

100

4.08 GB

3 GB

-26.47%

1000

15 GB

12.6 GB

-16%

* The size of zheap tables increase because of the insertions in pgbench_history table.


I think results representation should be improved.  You show total size of the database, but it's hard to understand how bloat degree was really decreased, assuming that there are both update and append-only tables.  So, I propose to show the results in per table manner.

What is total number of transactions processed in both cases?  It would be also more fair to compare sizes for the same number of processed transactions.

Also, what are index sizes?  What are undo log sizes for zheap?
 
I also suggest to use Zipfian distribution in testing.  It's more close to real world workloads.  And it would be a good stress test for both HOT and transaction slots.

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: WIP Patch: Precalculate stable functions, infrastructure v1
Next
From: David Rowley
Date:
Subject: Re: [HACKERS] Removing LEFT JOINs in more cases