Thread: max_parallel_degree > 0 for 9.6 beta
Hi, max_parallel_degree currently defaults to 0. I think we should enable it by default for at least the beta period. Otherwise we're primarily going to get reports back after the release. Then, at the end of beta, we can decide what the default should be. - Andres
På onsdag 20. april 2016 kl. 19:46:31, skrev Andres Freund <andres@anarazel.de>:
Hi,
max_parallel_degree currently defaults to 0. I think we should enable
it by default for at least the beta period. Otherwise we're primarily
going to get reports back after the release.
Then, at the end of beta, we can decide what the default should be.
+1
Not enabling it per default gives the signal "It's not safe".
--
Andreas Joseph Krogh
CTO / Partner - Visena AS
Mobile: +47 909 56 963
Attachment
Andres Freund <andres@anarazel.de> writes: > max_parallel_degree currently defaults to 0. I think we should enable > it by default for at least the beta period. Otherwise we're primarily > going to get reports back after the release. > Then, at the end of beta, we can decide what the default should be. +1, but let's put an entry on the 9.6 open-items page to remind us to make that decision at the right time. regards, tom lane
On Wed, Apr 20, 2016 at 2:28 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Andres Freund <andres@anarazel.de> writes: >> max_parallel_degree currently defaults to 0. I think we should enable >> it by default for at least the beta period. Otherwise we're primarily >> going to get reports back after the release. > >> Then, at the end of beta, we can decide what the default should be. > > +1, but let's put an entry on the 9.6 open-items page to remind us to > make that decision at the right time. So, I suggest that the only sensible non-zero values here are probably "1" or "2", given a default pool of 8 worker processes system-wide. Andres told me yesterday he'd vote for "2". Any other opinions? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Robert Haas <robertmhaas@gmail.com> writes: > On Wed, Apr 20, 2016 at 2:28 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> Andres Freund <andres@anarazel.de> writes: >>> max_parallel_degree currently defaults to 0. I think we should enable >>> it by default for at least the beta period. Otherwise we're primarily >>> going to get reports back after the release. > So, I suggest that the only sensible non-zero values here are probably > "1" or "2", given a default pool of 8 worker processes system-wide. > Andres told me yesterday he'd vote for "2". Any other opinions? It has to be at least 2 for beta purposes, else you are not testing situations with more than one worker process at all, which would be rather a large omission no? regards, tom lane
On Thu, Apr 21, 2016 at 1:48 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Robert Haas <robertmhaas@gmail.com> writes: >> On Wed, Apr 20, 2016 at 2:28 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>> Andres Freund <andres@anarazel.de> writes: >>>> max_parallel_degree currently defaults to 0. I think we should enable >>>> it by default for at least the beta period. Otherwise we're primarily >>>> going to get reports back after the release. > >> So, I suggest that the only sensible non-zero values here are probably >> "1" or "2", given a default pool of 8 worker processes system-wide. >> Andres told me yesterday he'd vote for "2". Any other opinions? > > It has to be at least 2 for beta purposes, else you are not testing > situations with more than one worker process at all, which would be > rather a large omission no? That's what Andres, thought, too. From my point of view, the big thing is to be using workers at all. It is of course possible that there could be some bugs where a single worker is not enough, but there's a lot of types of bug where even one worker would probably find the problem. But I'm OK with changing the default to 2. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 22/04/16 06:07, Robert Haas wrote: > On Thu, Apr 21, 2016 at 1:48 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> Robert Haas <robertmhaas@gmail.com> writes: >>> On Wed, Apr 20, 2016 at 2:28 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>>> Andres Freund <andres@anarazel.de> writes: >>>>> max_parallel_degree currently defaults to 0. I think we should enable >>>>> it by default for at least the beta period. Otherwise we're primarily >>>>> going to get reports back after the release. >>> So, I suggest that the only sensible non-zero values here are probably >>> "1" or "2", given a default pool of 8 worker processes system-wide. >>> Andres told me yesterday he'd vote for "2". Any other opinions? >> It has to be at least 2 for beta purposes, else you are not testing >> situations with more than one worker process at all, which would be >> rather a large omission no? > That's what Andres, thought, too. From my point of view, the big > thing is to be using workers at all. It is of course possible that > there could be some bugs where a single worker is not enough, but > there's a lot of types of bug where even one worker would probably > find the problem. But I'm OK with changing the default to 2. > I'm curious. Why not 4? As most processors now have at least 4 physical cores, & surely it be more likely to flush out race conditions. Cheers, Gavin
On Thu, Apr 21, 2016 at 4:01 PM, Gavin Flower <GavinFlower@archidevsys.co.nz> wrote: > On 22/04/16 06:07, Robert Haas wrote: >> >> On Thu, Apr 21, 2016 at 1:48 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>> >>> Robert Haas <robertmhaas@gmail.com> writes: >>>> >>>> On Wed, Apr 20, 2016 at 2:28 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>>>> >>>>> Andres Freund <andres@anarazel.de> writes: >>>>>> >>>>>> max_parallel_degree currently defaults to 0. I think we should enable >>>>>> it by default for at least the beta period. Otherwise we're primarily >>>>>> going to get reports back after the release. >>>> >>>> So, I suggest that the only sensible non-zero values here are probably >>>> "1" or "2", given a default pool of 8 worker processes system-wide. >>>> Andres told me yesterday he'd vote for "2". Any other opinions? >>> >>> It has to be at least 2 for beta purposes, else you are not testing >>> situations with more than one worker process at all, which would be >>> rather a large omission no? >> >> That's what Andres, thought, too. From my point of view, the big >> thing is to be using workers at all. It is of course possible that >> there could be some bugs where a single worker is not enough, but >> there's a lot of types of bug where even one worker would probably >> find the problem. But I'm OK with changing the default to 2. >> > I'm curious. > > Why not 4? As most processors now have at least 4 physical cores, & surely > it be more likely to flush out race conditions. Because if we did that, then it's extremely likely that people would end up writing queries that are faster only if workers are present, and then not get any workers. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Robert Haas <robertmhaas@gmail.com> writes: > On Thu, Apr 21, 2016 at 4:01 PM, Gavin Flower > <GavinFlower@archidevsys.co.nz> wrote: >> Why not 4? As most processors now have at least 4 physical cores, & surely >> it be more likely to flush out race conditions. > Because if we did that, then it's extremely likely that people would > end up writing queries that are faster only if workers are present, > and then not get any workers. Is that because max_worker_processes is only 8 by default? Maybe we need to raise that, at least for beta purposes? regards, tom lane
On Fri, Apr 22, 2016 at 1:31 AM, Gavin Flower <GavinFlower@archidevsys.co.nz> wrote:
IIUC, the idea to change max_parallel_degree for beta is to catch any bugs in parallelism code, not to do any performance testing of same. So, I think either 1 or 2 should be sufficient to hit the bugs if there are any. Do you have any reason to think that we might miss some category of bugs if we don't use higher max_parallel_degree?
With Regards,
Amit Kapila.
On 22/04/16 06:07, Robert Haas wrote:On Thu, Apr 21, 2016 at 1:48 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:I'm curious.Robert Haas <robertmhaas@gmail.com> writes:That's what Andres, thought, too. From my point of view, the bigOn Wed, Apr 20, 2016 at 2:28 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:It has to be at least 2 for beta purposes, else you are not testingAndres Freund <andres@anarazel.de> writes:So, I suggest that the only sensible non-zero values here are probablymax_parallel_degree currently defaults to 0. I think we should enable
it by default for at least the beta period. Otherwise we're primarily
going to get reports back after the release.
"1" or "2", given a default pool of 8 worker processes system-wide.
Andres told me yesterday he'd vote for "2". Any other opinions?
situations with more than one worker process at all, which would be
rather a large omission no?
thing is to be using workers at all. It is of course possible that
there could be some bugs where a single worker is not enough, but
there's a lot of types of bug where even one worker would probably
find the problem. But I'm OK with changing the default to 2.
Why not 4?
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
On Thu, Apr 21, 2016 at 7:20 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Robert Haas <robertmhaas@gmail.com> writes: >> On Thu, Apr 21, 2016 at 4:01 PM, Gavin Flower >> <GavinFlower@archidevsys.co.nz> wrote: >>> Why not 4? As most processors now have at least 4 physical cores, & surely >>> it be more likely to flush out race conditions. > >> Because if we did that, then it's extremely likely that people would >> end up writing queries that are faster only if workers are present, >> and then not get any workers. > > Is that because max_worker_processes is only 8 by default? Maybe we > need to raise that, at least for beta purposes? I'm not really in favor of that. I mean, almost all of our default settings are optimized for running PostgreSQL on, for example, a Raspberry Pi 2, so it would seem odd to suddenly swing the other direction and assume that there are more than 8 unused CPU cores. It doesn't make sense to me to roll out settings in beta that we wouldn't be willing to release with if they work out. That's why, honestly, I would prefer max_parallel_degree=1, which I think would be practical for many real-world deployments. max_parallel_degree=2 is OK. Beyond that, we're just setting people up to fail, I think. Higher settings should probably only be used on substantial hardware, and not everybody has that. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
På fredag 22. april 2016 kl. 14:56:33, skrev Robert Haas <robertmhaas@gmail.com>:
On Thu, Apr 21, 2016 at 7:20 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>> On Thu, Apr 21, 2016 at 4:01 PM, Gavin Flower
>> <GavinFlower@archidevsys.co.nz> wrote:
>>> Why not 4? As most processors now have at least 4 physical cores, & surely
>>> it be more likely to flush out race conditions.
>
>> Because if we did that, then it's extremely likely that people would
>> end up writing queries that are faster only if workers are present,
>> and then not get any workers.
>
> Is that because max_worker_processes is only 8 by default? Maybe we
> need to raise that, at least for beta purposes?
I'm not really in favor of that. I mean, almost all of our default
settings are optimized for running PostgreSQL on, for example, a
Raspberry Pi 2, so it would seem odd to suddenly swing the other
direction and assume that there are more than 8 unused CPU cores. It
doesn't make sense to me to roll out settings in beta that we wouldn't
be willing to release with if they work out. That's why, honestly, I
would prefer max_parallel_degree=1, which I think would be practical
for many real-world deployments. max_parallel_degree=2 is OK. Beyond
that, we're just setting people up to fail, I think. Higher settings
should probably only be used on substantial hardware, and not
everybody has that.
Maybe it's time to ask the question if the settings should be optimized more for high-end HW and not som matchstick-box? I mean, most of the people I know who are responsible for databases run them on HW colser to high-end than low-end. I'm not sure why optimizing for low-end is such a great choice.
--
Andreas Joseph Krog
Robert Haas <robertmhaas@gmail.com> writes: > On Thu, Apr 21, 2016 at 7:20 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> Is that because max_worker_processes is only 8 by default? Maybe we >> need to raise that, at least for beta purposes? > I'm not really in favor of that. I mean, almost all of our default > settings are optimized for running PostgreSQL on, for example, a > Raspberry Pi 2, so it would seem odd to suddenly swing the other > direction and assume that there are more than 8 unused CPU cores. I'm not following why you think that max_worker_processes cannot be set higher than the number of cores. By that argument, it's insane that we ship with max_connections = 100. In practice it's generally fine, and people can get away with oversubscribing their core count even more than that, because it's seldom that all those processes are actually contending for CPU at the same time. There are enough inefficiencies in our parallel-query design that the same will most certainly be true for parallel workers. So what I'm concerned about for beta purposes is that we have a setup that can exercise cases like, say, varying orders in which different workers return tuples, or potential deadlocks between sibling workers. We'd get no coverage of that behavioral space at max_parallel_degree=1. I'm not really convinced that we'll get adequate coverage at max_parallel_degree=2. regards, tom lane
On Fri, Apr 22, 2016 at 9:33 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Robert Haas <robertmhaas@gmail.com> writes: >> On Thu, Apr 21, 2016 at 7:20 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>> Is that because max_worker_processes is only 8 by default? Maybe we >>> need to raise that, at least for beta purposes? > >> I'm not really in favor of that. I mean, almost all of our default >> settings are optimized for running PostgreSQL on, for example, a >> Raspberry Pi 2, so it would seem odd to suddenly swing the other >> direction and assume that there are more than 8 unused CPU cores. > > I'm not following why you think that max_worker_processes cannot be > set higher than the number of cores. By that argument, it's insane > that we ship with max_connections = 100. In practice it's generally > fine, and people can get away with oversubscribing their core count > even more than that, because it's seldom that all those processes > are actually contending for CPU at the same time. There are enough > inefficiencies in our parallel-query design that the same will most > certainly be true for parallel workers. It is much less likely to be true for parallel workers. The reason why those processes aren't contending for the CPU at the same time is generally that most of the connections are in fact idle. But a parallel worker is never idle. It is launched when it is needed to run a query and exits immediately afterward. If it's not contending for the CPU, it will be contending for I/O bandwidth, or a lock. > So what I'm concerned about for beta purposes is that we have a setup that > can exercise cases like, say, varying orders in which different workers > return tuples, or potential deadlocks between sibling workers. We'd get > no coverage of that behavioral space at max_parallel_degree=1. I'm not > really convinced that we'll get adequate coverage at > max_parallel_degree=2. The right solution to that is for people who have the right hardware to raise the settings, not to unleash a ridiculous set of defaults on everyone. I really hope that some people do serious destruction testing of parallel query and try to break it. For example, you could use the parallel_degree reloption to force 100 parallel workers to scan the same relation. That's likely to be dog slow, but it might well turn up some bugs. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 04/22/2016 06:47 AM, Robert Haas wrote: > On Fri, Apr 22, 2016 at 9:33 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> Robert Haas <robertmhaas@gmail.com> writes: >>> On Thu, Apr 21, 2016 at 7:20 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>>> Is that because max_worker_processes is only 8 by default? Maybe we >>>> need to raise that, at least for beta purposes? >> >>> I'm not really in favor of that. I mean, almost all of our default >>> settings are optimized for running PostgreSQL on, for example, a >>> Raspberry Pi 2, so it would seem odd to suddenly swing the other >>> direction and assume that there are more than 8 unused CPU cores. This is the problem right here. We should be shipping for a reasonable production configuration. It is not reasonable to assume that someone is going to be running on a Rasberry Pi 2. Yes, we can effectively run on that platform that doesn't mean it should be our default target configuration. Consider that a 5.00/mo Digital Ocean VM is going to outperform a Rasberry Pi. > > It is much less likely to be true for parallel workers. The reason > why those processes aren't contending for the CPU at the same time is > generally that most of the connections are in fact idle. But a > parallel worker is never idle. It is launched when it is needed to > run a query and exits immediately afterward. If it's not contending > for the CPU, it will be contending for I/O bandwidth, or a lock. > True, but isn't that also what context switching and (possibly) hyperthreading are for? >> So what I'm concerned about for beta purposes is that we have a setup that >> can exercise cases like, say, varying orders in which different workers >> return tuples, or potential deadlocks between sibling workers. We'd get >> no coverage of that behavioral space at max_parallel_degree=1. I'm not >> really convinced that we'll get adequate coverage at >> max_parallel_degree=2. > > The right solution to that is for people who have the right hardware > to raise the settings, not to unleash a ridiculous set of defaults on > everyone. I really hope that some people do serious destruction > testing of parallel query and try to break it. For example, you could > use the parallel_degree reloption to force 100 parallel workers to > scan the same relation. That's likely to be dog slow, but it might > well turn up some bugs. I think your argument sounds more like a production solution, not a Beta solution. We should be pushing it a little bit in Beta. JD > -- Command Prompt, Inc. http://the.postgres.company/ +1-503-667-4564 PostgreSQL Centered full stack support, consulting and development. Everyone appreciates your honesty, until you are honest with them.
On Fri, Apr 22, 2016 at 10:07 AM, Joshua D. Drake <jd@commandprompt.com> wrote: > This is the problem right here. > > We should be shipping for a reasonable production configuration. It is not > reasonable to assume that someone is going to be running on a Rasberry Pi 2. > Yes, we can effectively run on that platform that doesn't mean it should be > our default target configuration. Consider that a 5.00/mo Digital Ocean VM > is going to outperform a Rasberry Pi. I don't disagree with that, and I think there is a considerable amount of work that could be done to create a saner "out of the box" configuration. But I don't think that the two weeks before beta is the right time to start building a consensus around what that might look like. > True, but isn't that also what context switching and (possibly) > hyperthreading are for? Sure. What you should expect, though, is that overall system throughput will be higher if the system is not oversubscribed. You can use parallel query selectively to speed up certain queries even if that takes you above the number of CPUs you have; if those queries are on a deadline, finishing them sooner may be worth whatever you lose in overall throughput. > I think your argument sounds more like a production solution, not a Beta > solution. We should be pushing it a little bit in Beta. Shipping with max_parallel_workers=2 *is* pushing it a little bit. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 23/04/16 00:56, Robert Haas wrote: > On Thu, Apr 21, 2016 at 7:20 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> Robert Haas <robertmhaas@gmail.com> writes: >>> On Thu, Apr 21, 2016 at 4:01 PM, Gavin Flower >>> <GavinFlower@archidevsys.co.nz> wrote: >>>> Why not 4? As most processors now have at least 4 physical cores, & surely >>>> it be more likely to flush out race conditions. >>> Because if we did that, then it's extremely likely that people would >>> end up writing queries that are faster only if workers are present, >>> and then not get any workers. >> Is that because max_worker_processes is only 8 by default? Maybe we >> need to raise that, at least for beta purposes? > I'm not really in favor of that. I mean, almost all of our default > settings are optimized for running PostgreSQL on, for example, a > Raspberry Pi 2, so it would seem odd to suddenly swing the other > direction and assume that there are more than 8 unused CPU cores. It > doesn't make sense to me to roll out settings in beta that we wouldn't > be willing to release with if they work out. That's why, honestly, I > would prefer max_parallel_degree=1, which I think would be practical > for many real-world deployments. max_parallel_degree=2 is OK. Beyond > that, we're just setting people up to fail, I think. Higher settings > should probably only be used on substantial hardware, and not > everybody has that. > If Java can find out how many processors there are available to it, since JDK1.4, then surely PostgreSQL can do the same? So how about the default being half the available processors rounded up to the nearest integer? Perhaps the GUC for workers should be a percentage of the available processors, with the minimum & maximum workers optionally specified - or something of that nature? Cheers, Gavin
On 22/04/16 17:36, Amit Kapila wrote: > On Fri, Apr 22, 2016 at 1:31 AM, Gavin Flower > <GavinFlower@archidevsys.co.nz <mailto:GavinFlower@archidevsys.co.nz>> > wrote: > > On 22/04/16 06:07, Robert Haas wrote: > > On Thu, Apr 21, 2016 at 1:48 PM, Tom Lane <tgl@sss.pgh.pa.us > <mailto:tgl@sss.pgh.pa.us>> wrote: > > Robert Haas <robertmhaas@gmail.com > <mailto:robertmhaas@gmail.com>> writes: > > On Wed, Apr 20, 2016 at 2:28 PM, Tom Lane > <tgl@sss.pgh.pa.us <mailto:tgl@sss.pgh.pa.us>> wrote: > > Andres Freund <andres@anarazel.de > <mailto:andres@anarazel.de>> writes: > > max_parallel_degree currently defaults to 0. > I think we should enable > it by default for at least the beta period. > Otherwise we're primarily > going to get reports back after the release. > > So, I suggest that the only sensible non-zero values > here are probably > "1" or "2", given a default pool of 8 worker processes > system-wide. > Andres told me yesterday he'd vote for "2". Any other > opinions? > > It has to be at least 2 for beta purposes, else you are > not testing > situations with more than one worker process at all, which > would be > rather a large omission no? > > That's what Andres, thought, too. From my point of view, the big > thing is to be using workers at all. It is of course possible > that > there could be some bugs where a single worker is not enough, but > there's a lot of types of bug where even one worker would probably > find the problem. But I'm OK with changing the default to 2. > > I'm curious. > > Why not 4? > > > IIUC, the idea to change max_parallel_degree for beta is to catch any > bugs in parallelism code, not to do any performance testing of same. > So, I think either 1 or 2 should be sufficient to hit the bugs if > there are any. Do you have any reason to think that we might miss > some category of bugs if we don't use higher max_parallel_degree? > > > With Regards, > Amit Kapila. > EnterpriseDB: http://www.enterprisedb.com <http://www.enterprisedb.com/> No. Just felt that 4 would not be too great for the type of processor chips used on servers to handle. For complications, such as race conditions and implied logical assumptions - I tend to think of 0, 1, 2, 3, many. Essentially just a gut feeling that 4 might reveal more corner cases. Cheers, Gavin
On Wed, Apr 20, 2016 at 02:28:15PM -0400, Tom Lane wrote: > Andres Freund <andres@anarazel.de> writes: > > max_parallel_degree currently defaults to 0. I think we should enable > > it by default for at least the beta period. Otherwise we're primarily > > going to get reports back after the release. > > > Then, at the end of beta, we can decide what the default should be. > > +1, but let's put an entry on the 9.6 open-items page to remind us to > make that decision at the right time. It's that time. Do we restore the max_parallel_workers_per_gather=0 default, or is enabling this by default the right thing after all?
Noah Misch <noah@leadboat.com> writes: > On Wed, Apr 20, 2016 at 02:28:15PM -0400, Tom Lane wrote: >> +1, but let's put an entry on the 9.6 open-items page to remind us to >> make that decision at the right time. > It's that time. Do we restore the max_parallel_workers_per_gather=0 default, > or is enabling this by default the right thing after all? At this point I'd have to vote against enabling by default in 9.6. The fact that in the past week we've found bugs as bad as e1a93dd6a does not give me a warm fuzzy feeling about the parallel-query code being ready for prime time. Of course the question is how do we ever get to that point if we chicken out with enabling it by default now. Maybe we could keep it turned on in HEAD. regards, tom lane
On Thu, Aug 4, 2016 at 12:56 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Noah Misch <noah@leadboat.com> writes: >> On Wed, Apr 20, 2016 at 02:28:15PM -0400, Tom Lane wrote: >>> +1, but let's put an entry on the 9.6 open-items page to remind us to >>> make that decision at the right time. > >> It's that time. Do we restore the max_parallel_workers_per_gather=0 default, >> or is enabling this by default the right thing after all? > > At this point I'd have to vote against enabling by default in 9.6. The > fact that in the past week we've found bugs as bad as e1a93dd6a does not > give me a warm fuzzy feeling about the parallel-query code being ready > for prime time. > > Of course the question is how do we ever get to that point if we chicken > out with enabling it by default now. Maybe we could keep it turned on > in HEAD. What do other people think about this topic? Personally, I've found the process of fixing the parallel query bugs rather exhausting, and if we leave it turned on by default in 9.6, we'll probably get a lot more of those bug reports a lot sooner than we will otherwise. But I'd kind of rather get it out of the way than put it off: those bugs need to be fixed at some point, and we won't find them if nobody runs the code. However, we have a long track record of being cautious about things like this, so I would be OK with the idea of disabling this in the 9.6 branch and leaving it turned on in master, with the hope of shipping it enabled-by-default in v10. I think it would not be good to leave disabled it by default in master -- the code won't get much testing then, we won't find the bugs, and we'll build more stuff on top of what we've already got without finding the cracks in the foundation. I think users will be more inclined to forgive us for parallel query bugs in 2016 than in 2026; better to debug it before the honeymoon wears off. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Thu, Aug 4, 2016 at 12:56 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Noah Misch <noah@leadboat.com> writes:
>> On Wed, Apr 20, 2016 at 02:28:15PM -0400, Tom Lane wrote:
>>> +1, but let's put an entry on the 9.6 open-items page to remind us to
>>> make that decision at the right time.
>
>> It's that time. Do we restore the max_parallel_workers_per_gather=0 default,
>> or is enabling this by default the right thing after all?
>
> At this point I'd have to vote against enabling by default in 9.6. The
> fact that in the past week we've found bugs as bad as e1a93dd6a does not
> give me a warm fuzzy feeling about the parallel-query code being ready
> for prime time.
>
> Of course the question is how do we ever get to that point if we chicken
> out with enabling it by default now. Maybe we could keep it turned on
> in HEAD.
However, we have a long track
record of being cautious about things like this, so I would be OK with
the idea of disabling this in the 9.6 branch and leaving it turned on
in master, with the hope of shipping it enabled-by-default in v10.
My initial reaction was +1 but now I'm leaning toward enabled by default.
Those who would upgrade to 9.6 within a year of its release are most likely, process and personality wise, to be those for whom the risks and rewards of new features is part of their everyday routine.
If indeed we release 10.0 with it enabled by default we should be confident in its stability at that point in the 9.6.x branch.
David J.
On Thu, Aug 4, 2016 at 10:52 PM, David G. Johnston <david.g.johnston@gmail.com> wrote: > My initial reaction was +1 but now I'm leaning toward enabled by default. > > Those who would upgrade to 9.6 within a year of its release are most likely, > process and personality wise, to be those for whom the risks and rewards of > new features is part of their everyday routine. > > If indeed we release 10.0 with it enabled by default we should be confident > in its stability at that point in the 9.6.x branch. Being cautious pays more in the long term, so seeing the number of bugs that showed up I'd rather vote for having it disabled by default in 9.6 stable, and enabled on master to aim at enabling it in 10.0. -- Michael
Michael Paquier wrote: > Being cautious pays more in the long term, so seeing the number of > bugs that showed up I'd rather vote for having it disabled by default > in 9.6 stable, and enabled on master to aim at enabling it in 10.0. I too prefer to keep it turned off in 9.6 and consider enabling it by default on a future release (10 is probably good). Interested users can carefully test the feature without endangering other unsuspecting users. I agree with the idea of keeping it enabled in master, so that it'll get a modicum of testing there by hackers, too. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Sat, Aug 13, 2016 at 11:56 AM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote: > Michael Paquier wrote: > >> Being cautious pays more in the long term, so seeing the number of >> bugs that showed up I'd rather vote for having it disabled by default >> in 9.6 stable, and enabled on master to aim at enabling it in 10.0. > > I too prefer to keep it turned off in 9.6 and consider enabling it by > default on a future release (10 is probably good). Interested users can > carefully test the feature without endangering other unsuspecting users. > > I agree with the idea of keeping it enabled in master, so that it'll get > a modicum of testing there by hackers, too. Sounds like that is the consensus. Who's going to implement it? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Robert Haas <robertmhaas@gmail.com> writes: > On Sat, Aug 13, 2016 at 11:56 AM, Alvaro Herrera > <alvherre@2ndquadrant.com> wrote: >> I too prefer to keep it turned off in 9.6 and consider enabling it by >> default on a future release (10 is probably good). Interested users can >> carefully test the feature without endangering other unsuspecting users. >> I agree with the idea of keeping it enabled in master, so that it'll get >> a modicum of testing there by hackers, too. > Sounds like that is the consensus. Who's going to implement it? I believe we're talking about reverting 77cd477c4 (in 9.6 only not master), correct? It's a little harder than just "git revert" because of the subsequent max_parallel_degree -> max_parallel_workers_per_gather name change, but still not exactly rocket science. Since 77cd477c4 was your commit, I'd sort of expect you to do the honors, but if you don't want to I can. regards, tom lane
On Mon, Aug 15, 2016 at 9:10 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Robert Haas <robertmhaas@gmail.com> writes: >> On Sat, Aug 13, 2016 at 11:56 AM, Alvaro Herrera >> <alvherre@2ndquadrant.com> wrote: >>> I too prefer to keep it turned off in 9.6 and consider enabling it by >>> default on a future release (10 is probably good). Interested users can >>> carefully test the feature without endangering other unsuspecting users. >>> I agree with the idea of keeping it enabled in master, so that it'll get >>> a modicum of testing there by hackers, too. > >> Sounds like that is the consensus. Who's going to implement it? > > I believe we're talking about reverting 77cd477c4 (in 9.6 only not > master), correct? It's a little harder than just "git revert" because > of the subsequent max_parallel_degree -> max_parallel_workers_per_gather > name change, but still not exactly rocket science. > > Since 77cd477c4 was your commit, I'd sort of expect you to do the > honors, but if you don't want to I can. OK, I'll do it. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company