Re: zheap: a new storage format for PostgreSQL - Mailing list pgsql-hackers
| From | Amit Kapila |
|---|---|
| Subject | Re: zheap: a new storage format for PostgreSQL |
| Date | |
| Msg-id | CAA4eK1LhZHnXKg7Ji0-zDKFiMVwj+0r63hbSSmcXHZy2TSj-7g@mail.gmail.com Whole thread Raw |
| In response to | Re: zheap: a new storage format for PostgreSQL (Alexander Korotkov <a.korotkov@postgrespro.ru>) |
| List | pgsql-hackers |
On Fri, Mar 2, 2018 at 2:42 AM, Alexander Korotkov <a.korotkov@postgrespro.ru> wrote:
On Thu, Mar 1, 2018 at 5:09 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:Preliminary performance results------------------------------------------- We’ve shown the performance improvement of zheap over heap in a few different pgbench scenarios. All of these tests were run with data that fits in shared_buffers (32GB), and 16 transaction slots per zheap page. Scenario-1 and Scenario-2 has used synchronous_commit = off and Scenario-3 and Scenario-4 has used synchronous_commit = on
What hardware did you use for benchmarks?
Also, I note that you have 4 transaction slots per zheap page in github code while you use 16 in benchmarks.#define MAX_PAGE_TRANS_INFO_SLOTS 4I would also note that in the code you preserve only 3 bits for transaction slot number. So, one have to redefine 3 macros to change transaction slot number to the value you used in the benchmarks.#define ZHEAP_XACT_SLOT 0x3800 /* 3 bits (12, 13 and 14) for transaction slot */#define ZHEAP_XACT_SLOT_MASK 0x000B /* 11 - mask to retrieve transaction slot */I'm only starting reviewing this, but it makes me think that we need transaction slots number to be tunable (or even auto-tunable).
Yeah, that is the plan. So, the idea is that for now we will give compile time option to configure the number of slots (the patch for the same is ready, currently we are testing it), later we can even give the option to user at relation level or whatever we decides. Why I think it makes sense to give an option at relation level is that for larger relations, we can do with very few transaction slots considering that the chances of many transactions operating on the same page are less, it is only for smaller relations that we need more number of slots. OTOH, there could be workloads where we can expect many concurrent transactions on the same page. However, for now if you want to test, the patch to increase transaction slots is attached, you need to change the value of few macros according to the number of slots you want.
BTW, last two macros don't look properly named for me. I would rather rename them in a following way:ZHEAP_XACT_SLOT_MASK => ZHEAP_XACT_SLOT_OFFSET
How about ZHEAP_XACT_SLOT_SHIFT? I see similar things named with *_SHIFT suffix in code .
ZHEAP_XACT_SLOT => ZHEAP_XACT_SLOT_MASK
makes sense. I will change it.
Scenario 1: A 15 minutes simple-update pgbench test with scale factor 100 shows 5.13% TPS improvement with 64 clients. The performance improvement increases as we increase the scale factor; at scale factor 1000, it reaches11.5% with 64 clients.
Scale Factor
HEAP
ZHEAP (tables)*
Improvement
Before test
100
1281 MB
1149 MB
-10.3%
1000
13 GB
11 GB
-15.38%
After test
100
4.08 GB
3 GB
-26.47%
1000
15 GB
12.6 GB
-16%
* The size of zheap tables increase because of the insertions in pgbench_history table.
I think results representation should be improved. You show total size of the database, but it's hard to understand how bloat degree was really decreased, assuming that there are both update and append-only tables. So, I propose to show the results in per table manner.
Fair enough, Kuntal has done this testing. He will share the results as you have requested.
What is total number of transactions processed in both cases? It would be also more fair to compare sizes for the same number of processed transactions.Also, what are index sizes? What are undo log sizes for zheap?
There shouldn't be any change in the index sizes and by the end of tests undo is completely discarded. I think to see the impact of undo size, we need some different tests where in we can keep the transaction open till end of test or some such.
I also suggest to use Zipfian distribution in testing. It's more close to real world workloads. And it would be a good stress test for both HOT and transaction slots.
Yeah, we can do such tests, but keep in mid this code is still a work in progress and lot of things are going to change and till now we have not done much optimization in the code to improve the performance numbers.
Thanks a lot for showing interest in this work!
--
Attachment
pgsql-hackers by date: