Thread: [pgsql-advocacy] PostgreSQL 10: Call for Quotes
Hi!
The PostgreSQL 10 release is rapidly approaching and we are starting the process of putting together all the necessary materials to ensure we can make this the most successful Postgres release to date! One feature that we like to include in Postgres release announcements are quotes from Postgres users that highlight the new features in the upcoming release as well as the overall quality of the PostgreSQL platform.
If you are interested in submitting a quote, please include the following:
* Your Name
* Your Organization
* A 1-3 sentence quote that covers on at least one of the following criteria:
* A new major feature in PostgreSQL 10, such as but not limited to:
* Logical Replication
* Native Table Partitioning
* Improved Query Parallelism
* Quorum Commit for Synchronous Replication
* More ideas here: https://www.postgresql.org/about/news/1749/
* The overall benefits PostgreSQL provides to your organization and how it has helped you scale / save / develop more rapidly etc.
Please send your quote submissions to press@postgresql.org no later than Friday, September 8, 2017 . I do encourage you to submit early and often.
Thanks in advance for your contributions!
Jonathan
On 26 July 2017 at 20:16, Jonathan S. Katz <jkatz@postgresql.org> wrote: > The PostgreSQL 10 release is rapidly approaching and we are starting the > process of putting together all the necessary materials to ensure we can > make this the most successful Postgres release to date! ... > * A new major feature in PostgreSQL 10, such as but not limited to: > * Logical Replication > * Native Table Partitioning > * Improved Query Parallelism > * Quorum Commit for Synchronous Replication Can we agree that is the list of Major Features for PG10? (It sure looks like it) We still have nothing here... https://www.postgresql.org/docs/10/static/release-10.html it just says Major enhancements in PostgreSQL 10 include: (to be written) Given this info also earlier noted this post https://www.infoq.com/news/2017/04/postgresql-10-features I propose the feature list as this * Logical Replication * Native Table Partitioning * Improved Query Parallelism * Quorum Commit for Synchronous Replication * SCRAM-SHA-256 strong authentication Any objection to me updating the docs to say that now, subject to further discussion? It's better to have something in place soon, even if it is updated later. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 08/31/2017 08:32 AM, Simon Riggs wrote: > On 26 July 2017 at 20:16, Jonathan S. Katz <jkatz@postgresql.org> wrote: > Given this info also earlier noted this post > https://www.infoq.com/news/2017/04/postgresql-10-features > > I propose the feature list as this > > * Logical Replication > * Native Table Partitioning > * Improved Query Parallelism > * Quorum Commit for Synchronous Replication > * SCRAM-SHA-256 strong authentication > > Any objection to me updating the docs to say that now, subject to > further discussion? I would argue that Traceable commit is just as interesting as Quorum Commit (and likely applies to a larger audience). I would also argue that SCRAM-SHA-256 isn't as "big feature" as we would like it to be as we have supported strong authentication through external mechanisms for a very long time. Other than that, I have no comments and yes we should update the page. JD > > It's better to have something in place soon, even if it is updated later. > -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL Centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://pgconf.us ***** Unless otherwise stated, opinions are my own. *****
On Thu, Aug 31, 2017 at 11:38 AM, Joshua D. Drake <jd@commandprompt.com> wrote: >> * Logical Replication >> * Native Table Partitioning >> * Improved Query Parallelism >> * Quorum Commit for Synchronous Replication >> * SCRAM-SHA-256 strong authentication >> >> Any objection to me updating the docs to say that now, subject to >> further discussion? > > I would argue that Traceable commit is just as interesting as Quorum Commit > (and likely applies to a larger audience). I would also argue that > SCRAM-SHA-256 isn't as "big feature" as we would like it to be as we have > supported strong authentication through external mechanisms for a very long > time. > > Other than that, I have no comments and yes we should update the page. I think SCRAM authentication is a major feature. I would argue for adding Andres's executor speedups and Amit's hash index work, too, but I realize we can't list everything. I'm slightly surprised that quorum commit for synchronous replication ended up ahead of either of those; surely that's a pretty niche feature compared to making the executor faster? But I just work here. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
> On Aug 31, 2017, at 09:48, Robert Haas <robertmhaas@gmail.com> wrote: > I think SCRAM authentication is a major feature. +1. MD5-only is a major negative talking point against PostgreSQL, and as this is primarily a marketing document, highlightingit is important.
* Robert Haas (robertmhaas@gmail.com) wrote: > On Thu, Aug 31, 2017 at 11:38 AM, Joshua D. Drake <jd@commandprompt.com> wrote: > >> * Logical Replication > >> * Native Table Partitioning > >> * Improved Query Parallelism > >> * Quorum Commit for Synchronous Replication > >> * SCRAM-SHA-256 strong authentication > >> > >> Any objection to me updating the docs to say that now, subject to > >> further discussion? > > > > I would argue that Traceable commit is just as interesting as Quorum Commit > > (and likely applies to a larger audience). I would also argue that > > SCRAM-SHA-256 isn't as "big feature" as we would like it to be as we have > > supported strong authentication through external mechanisms for a very long > > time. > > > > Other than that, I have no comments and yes we should update the page. > > I think SCRAM authentication is a major feature. I would argue for > adding Andres's executor speedups and Amit's hash index work, too, but > I realize we can't list everything. I'm slightly surprised that > quorum commit for synchronous replication ended up ahead of either of > those; surely that's a pretty niche feature compared to making the > executor faster? But I just work here. I definitely think SCRAM is a major feature and it'll impact a large portion of our userbase too. Hash indexes seems like a bigger feature than quorum commit for sync rep to me too. Performance improvements seems a bit more difficult to justify as a 'new major feature' to me. Traceable commit is definitely interesting, but I don't see it as anywhere near the level of SCRAM or Quorum commit. Thanks! Stephen
Attachment
On 31 August 2017 at 17:48, Robert Haas <robertmhaas@gmail.com> wrote: > I would argue for > adding Andres's executor speedups and Amit's hash index work, too Do we have accepted/verified perf results we can quote? -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Thu, Aug 31, 2017 at 1:16 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > On 31 August 2017 at 17:48, Robert Haas <robertmhaas@gmail.com> wrote: >> I would argue for >> adding Andres's executor speedups and Amit's hash index work, too > > Do we have accepted/verified perf results we can quote? Andres had a chart showing mammoth speedups on some of the TPC-H queries which he presented at PGCon. See, e.g. slide 17 at https://www.pgcon.org/2017/schedule/attachments/462_jit-pgcon-2017-05-25.pdf I don't know that that's quite formal enough for PR, but it - Q1 in particular - sure shows a nice gain. For hash indexes, the benefit is mostly that now they don't go kablooey after a crash, rather than performance, although there was performance work done as part of the project too. Not sure what the best way to showcase this would be. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 31 Aug 2017, at 18:34, Robert Haas <robertmhaas@gmail.com> wrote: > On Thu, Aug 31, 2017 at 1:16 PM, Simon Riggs <simon@2ndquadrant.com> wrote: >> On 31 August 2017 at 17:48, Robert Haas <robertmhaas@gmail.com> wrote: >>> I would argue for >>> adding Andres's executor speedups and Amit's hash index work, too >> >> Do we have accepted/verified perf results we can quote? > > Andres had a chart showing mammoth speedups on some of the TPC-H > queries which he presented at PGCon. See, e.g. slide 17 at > https://www.pgcon.org/2017/schedule/attachments/462_jit-pgcon-2017-05-25.pdf Hmmm, that PDF renders fine when downloaded (eg curl) and viewed on OSX using the OSX in-built "Preview" application. It's all kinds of busted (mostly unreadable) when viewed online with Chrome/Opera's in-built viewer. Andres, any idea if that can be fixed so it's viewable for everyone? :) Regards and best wishes, Justin Clift -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi
On 2017-08-31 18:52:04 +0100, Justin Clift wrote: > On 31 Aug 2017, at 18:34, Robert Haas <robertmhaas@gmail.com> wrote: > > On Thu, Aug 31, 2017 at 1:16 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > >> On 31 August 2017 at 17:48, Robert Haas <robertmhaas@gmail.com> wrote: > >>> I would argue for > >>> adding Andres's executor speedups and Amit's hash index work, too > >> > >> Do we have accepted/verified perf results we can quote? > > > > Andres had a chart showing mammoth speedups on some of the TPC-H > > queries which he presented at PGCon. See, e.g. slide 17 at > > https://www.pgcon.org/2017/schedule/attachments/462_jit-pgcon-2017-05-25.pdf > > Hmmm, that PDF renders fine when downloaded (eg curl) and viewed on OSX > using the OSX in-built "Preview" application. > > It's all kinds of busted (mostly unreadable) when viewed online with > Chrome/Opera's in-built viewer. > > Andres, any idea if that can be fixed so it's viewable for everyone? :) Hm - no idea. Works in my version of chromium (60.0.3112.78). It's a bit hard to fix something if you have no idea what's the cause for the problem :(. It's just a libreoffice export... - Andres
On 31 Aug 2017, at 19:00, Andres Freund <andres@anarazel.de> wrote: > On 2017-08-31 18:52:04 +0100, Justin Clift wrote: >> On 31 Aug 2017, at 18:34, Robert Haas <robertmhaas@gmail.com> wrote: >>> On Thu, Aug 31, 2017 at 1:16 PM, Simon Riggs <simon@2ndquadrant.com> wrote: >>>> On 31 August 2017 at 17:48, Robert Haas <robertmhaas@gmail.com> wrote: >>>>> I would argue for >>>>> adding Andres's executor speedups and Amit's hash index work, too >>>> >>>> Do we have accepted/verified perf results we can quote? >>> >>> Andres had a chart showing mammoth speedups on some of the TPC-H >>> queries which he presented at PGCon. See, e.g. slide 17 at >>> https://www.pgcon.org/2017/schedule/attachments/462_jit-pgcon-2017-05-25.pdf >> >> Hmmm, that PDF renders fine when downloaded (eg curl) and viewed on OSX >> using the OSX in-built "Preview" application. >> >> It's all kinds of busted (mostly unreadable) when viewed online with >> Chrome/Opera's in-built viewer. >> >> Andres, any idea if that can be fixed so it's viewable for everyone? :) > > Hm - no idea. Works in my version of chromium (60.0.3112.78). It's a bit > hard to fix something if you have no idea what's the cause for the > problem :(. It's just a libreoffice export... Yeah, figured that might be the case. Hmmm, if you have the source doc handy, I can try exporting it with a few different releases of LibreOffice and see if any of them don't bust it. :) + Justin -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi
On Thu, Aug 31, 2017 at 10:46 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > On 31 August 2017 at 17:48, Robert Haas <robertmhaas@gmail.com> wrote: > >> I would argue for >> adding Andres's executor speedups and Amit's hash index work, too > > Do we have accepted/verified perf results we can quote? > The main benefit (as mentioned by Robert) of hash indexes is that now they are durable, but they do better than btree indexes in some use cases (in terms of size and raw TPS) as has been demonstrated with the help of real world workload and by doing performance testing using benchmark pgbench. Just to give some references, please see the posts of some of the users on hackers. Reference about size: https://www.postgresql.org/message-id/20170704105728.mwb72jebfmok2nm2%40zip.com.au Reference about the performance of Select: https://www.postgresql.org/message-id/4575a870-1315-18ac-0516-c21a83a7afdf%40redhat.com Apart from above, I and some of my colleagues have also done performance testing, the results of which are summarized in my PGCon presentation: https://www.pgcon.org/2017/schedule/events/1039.en.html Now, I don't want to say that hash indexes became better in terms of size and performance only because of work in PG-10. However, there has been substantial work done in PG-10 in both areas and the numbers have been shared on hackers (in case you or others want to see, I can dig down those emails and share it here). I am not sure whether the work done on hash indexes in PG-10 makes it a candidate for the major feature, but I think it is worth considering. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
On 1 September 2017 at 05:08, Amit Kapila <amit.kapila16@gmail.com> wrote: > On Thu, Aug 31, 2017 at 10:46 PM, Simon Riggs <simon@2ndquadrant.com> wrote: >> On 31 August 2017 at 17:48, Robert Haas <robertmhaas@gmail.com> wrote: >> >>> I would argue for >>> adding Andres's executor speedups and Amit's hash index work, too >> >> Do we have accepted/verified perf results we can quote? >> > > The main benefit (as mentioned by Robert) of hash indexes is that now > they are durable, but they do better than btree indexes in some use > cases (in terms of size and raw TPS) as has been demonstrated with the > help of real world workload and by doing performance testing using > benchmark pgbench. Just to give some references, please see the posts > of some of the users on hackers. > > Reference about size: > https://www.postgresql.org/message-id/20170704105728.mwb72jebfmok2nm2%40zip.com.au > > Reference about the performance of Select: > https://www.postgresql.org/message-id/4575a870-1315-18ac-0516-c21a83a7afdf%40redhat.com > > Apart from above, I and some of my colleagues have also done > performance testing, the results of which are summarized in my PGCon > presentation: > https://www.pgcon.org/2017/schedule/events/1039.en.html > > Now, I don't want to say that hash indexes became better in terms of > size and performance only because of work in PG-10. However, there > has been substantial work done in PG-10 in both areas and the numbers > have been shared on hackers (in case you or others want to see, I can > dig down those emails and share it here). > > I am not sure whether the work done on hash indexes in PG-10 makes it > a candidate for the major feature, but I think it is worth > considering. This is very good work, well done. We've discussed my misgivings about hash indexes face to face, so forgive me if I repeat some of them here. Hash indexes work well for equality lookups on unique data, yet do not yet themselves enforce uniqueness, so you are forced to have a btree anyway. Expanding the hash index gives operational issues and we have no measurements of the effects of that - not something we should be letting people discover in production. Some concern over write performance, especially since no published measurements. BRIN suffered from people misunderstanding its use case, so perhaps we can avoid a repeat of that. Are we safe to draw attention to these indexes, for a particular use case? Can we get a clear statement of what that is? If we can, I would incline towards adding them to the major items list. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Fri, Sep 1, 2017 at 7:35 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
This is very good work, well done.
We've discussed my misgivings about hash indexes face to face, so
forgive me if I repeat some of them here.
Hash indexes work well for equality lookups on unique data, yet do not
yet themselves enforce uniqueness, so you are forced to have a btree
anyway. Expanding the hash index gives operational issues and we have
no measurements of the effects of that - not something we should be
letting people discover in production. Some concern over write
performance, especially since no published measurements.
BRIN suffered from people misunderstanding its use case, so perhaps we
can avoid a repeat of that.
Are we safe to draw attention to these indexes, for a particular use
case? Can we get a clear statement of what that is? If we can, I would
incline towards adding them to the major items list.
I would like to second this and add a note.
I ran a small benchmark myself on tables inserting large numbers of uuids (5 million). These went first into a holding table. Then in the benchmark I did an insert .... select....;
Three tables:
1. Unindexed (control)
2. Btree
3. Hash
What I found was that in my tests, hash indexes were marginally faster for lookups.
Btrees handled inserts far better (20% improvement *worst case* and 300% improvement *best case*)
So from this I concluded that this was not the use case for hash indexes.
But I would be very interested in where the use cases are.
Best Wishes,
Chris Travers
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services--
Sent via pgsql-advocacy mailing list (pgsql-advocacy@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-advocacy
Best Wishes,
Chris Travers
Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in.
On Fri, Sep 1, 2017 at 11:05 AM, Simon Riggs <simon@2ndquadrant.com> wrote: > On 1 September 2017 at 05:08, Amit Kapila <amit.kapila16@gmail.com> wrote: >> On Thu, Aug 31, 2017 at 10:46 PM, Simon Riggs <simon@2ndquadrant.com> wrote: >>> On 31 August 2017 at 17:48, Robert Haas <robertmhaas@gmail.com> wrote: >>> >>>> I would argue for >>>> adding Andres's executor speedups and Amit's hash index work, too >>> >>> Do we have accepted/verified perf results we can quote? >>> >> >> The main benefit (as mentioned by Robert) of hash indexes is that now >> they are durable, but they do better than btree indexes in some use >> cases (in terms of size and raw TPS) as has been demonstrated with the >> help of real world workload and by doing performance testing using >> benchmark pgbench. Just to give some references, please see the posts >> of some of the users on hackers. >> >> Reference about size: >> https://www.postgresql.org/message-id/20170704105728.mwb72jebfmok2nm2%40zip.com.au >> >> Reference about the performance of Select: >> https://www.postgresql.org/message-id/4575a870-1315-18ac-0516-c21a83a7afdf%40redhat.com >> >> Apart from above, I and some of my colleagues have also done >> performance testing, the results of which are summarized in my PGCon >> presentation: >> https://www.pgcon.org/2017/schedule/events/1039.en.html >> >> Now, I don't want to say that hash indexes became better in terms of >> size and performance only because of work in PG-10. However, there >> has been substantial work done in PG-10 in both areas and the numbers >> have been shared on hackers (in case you or others want to see, I can >> dig down those emails and share it here). >> >> I am not sure whether the work done on hash indexes in PG-10 makes it >> a candidate for the major feature, but I think it is worth >> considering. > > This is very good work, well done. > > We've discussed my misgivings about hash indexes face to face, so > forgive me if I repeat some of them here. > > Hash indexes work well for equality lookups on unique data, yet do not > yet themselves enforce uniqueness, so you are forced to have a btree > anyway. Expanding the hash index gives operational issues and we have > no measurements of the effects of that - not something we should be > letting people discover in production. > Yes, expansion of hash indexes was a problem and we have done some work to make them expand more gradually. See commit ea69a0dead5128c421140dc53fac165ba4af8520 Author: Robert Haas <rhaas@postgresql.org> Date: Mon Apr 3 23:46:33 2017 -0400 Expand hash indexes more gradually. > Some concern over write > performance, especially since no published measurements. > Yes, writes/updates on hash indexes are not good and we have done some work to improve it like in commit 6977b8b7, but I think that needs more work. commit 6977b8b7f4dfb40896ff5e2175cad7fdbda862eb Author: Robert Haas <rhaas@postgresql.org> Date: Wed Mar 15 22:18:56 2017 -0400 Port single-page btree vacuum logic to hash indexes. > BRIN suffered from people misunderstanding its use case, so perhaps we > can avoid a repeat of that. > > Are we safe to draw attention to these indexes, for a particular use > case? Can we get a clear statement of what that is? > It will be good for cases where the size of index matters and the index is on columns like UUID, bytea, varchar (basically, the key length is somewhat bigger say 32 or 64 bytes) and the index data is not updated frequently. It should perform well on Read workloads especially at higher client counts. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
On Fri, Sep 1, 2017 at 11:13 AM, Chris Travers <chris.travers@gmail.com> wrote: > > > On Fri, Sep 1, 2017 at 7:35 AM, Simon Riggs <simon@2ndquadrant.com> wrote: >> >> >> This is very good work, well done. >> >> We've discussed my misgivings about hash indexes face to face, so >> forgive me if I repeat some of them here. >> >> Hash indexes work well for equality lookups on unique data, yet do not >> yet themselves enforce uniqueness, so you are forced to have a btree >> anyway. Expanding the hash index gives operational issues and we have >> no measurements of the effects of that - not something we should be >> letting people discover in production. Some concern over write >> performance, especially since no published measurements. >> >> BRIN suffered from people misunderstanding its use case, so perhaps we >> can avoid a repeat of that. >> >> Are we safe to draw attention to these indexes, for a particular use >> case? Can we get a clear statement of what that is? If we can, I would >> incline towards adding them to the major items list. > > > I would like to second this and add a note. > > I ran a small benchmark myself on tables inserting large numbers of uuids (5 > million). These went first into a holding table. Then in the benchmark I > did an insert .... select....; > > Three tables: > 1. Unindexed (control) > 2. Btree > 3. Hash > Did you notice the size of indexes and what happens if you double the data? > What I found was that in my tests, hash indexes were marginally faster for > lookups. > At what client count (how many simultaneous clients were accessing hash index), did you tried this on higher client counts? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
On 31 August 2017 at 18:34, Robert Haas <robertmhaas@gmail.com> wrote: > On Thu, Aug 31, 2017 at 1:16 PM, Simon Riggs <simon@2ndquadrant.com> wrote: >> On 31 August 2017 at 17:48, Robert Haas <robertmhaas@gmail.com> wrote: >>> I would argue for >>> adding Andres's executor speedups and Amit's hash index work, too >> >> Do we have accepted/verified perf results we can quote? > > Andres had a chart showing mammoth speedups on some of the TPC-H > queries which he presented at PGCon. See, e.g. slide 17 at > https://www.pgcon.org/2017/schedule/attachments/462_jit-pgcon-2017-05-25.pdf > > I don't know that that's quite formal enough for PR, but it - Q1 in > particular - sure shows a nice gain. That's good enough to include on the major items certainly. > For hash indexes, the benefit is mostly that now they don't go > kablooey after a crash, rather than performance, although there was > performance work done as part of the project too. Not sure what the > best way to showcase this would be. Discussion continues on that aspect. I'm going to commit this patch in next hour, barring objections, so we have something there as a stopgap while we discuss what the final list will be. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachment
On Fri, Sep 1, 2017 at 1:35 AM, Simon Riggs <simon@2ndquadrant.com> wrote: > Are we safe to draw attention to these indexes, for a particular use > case? Can we get a clear statement of what that is? If we can, I would > incline towards adding them to the major items list. I think it's cases where the keys are very wide, only equality lookups are needed, and the data isn't changing a ton. Write performance for hash indexes is still not fantastic, although I believe that, due to the single-page vacuuming logic in particular, it's a lot better than formerly. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Hi,
On Jul 26, 2017, at 3:16 PM, Jonathan S. Katz <jkatz@postgresql.org> wrote:Hi!The PostgreSQL 10 release is rapidly approaching and we are starting the process of putting together all the necessary materials to ensure we can make this the most successful Postgres release to date! One feature that we like to include in Postgres release announcements are quotes from Postgres users that highlight the new features in the upcoming release as well as the overall quality of the PostgreSQL platform.If you are interested in submitting a quote, please include the following:* Your Name* Your Organization* A 1-3 sentence quote that covers on at least one of the following criteria:* A new major feature in PostgreSQL 10, such as but not limited to:* Logical Replication* Native Table Partitioning* Improved Query Parallelism* Quorum Commit for Synchronous Replication* More ideas here: https://www.postgresql.org/about/news/1749/* The overall benefits PostgreSQL provides to your organization and how it has helped you scale / save / develop more rapidly etc.Please send your quote submissions to press@postgresql.org no later than Friday, September 8, 2017 . I do encourage you to submit early and often.
To date we have received 2 quotes that were submitted seriously. We could use a few more. If you would like to submit a quote or know someone who is using PostgreSQL in a way you believe is worth sharing, please submit the quotes to press@postgresql.org by Friday, September 15, 2017.
Thanks!
Jonathan