Thread: Fw: server hardware tuning.

Hi Suganthi,

I can give you a start, some pro users can suggest you better.

 

1. Don't use this much of connections on a single postgres server. Use a connection pooler in front of it.

2. RAM: Depends upon how much data you want to be cached.

3. Use PCIe SATA SSD with RAID10, Postgres uses a lot of IO for its operations. 

4. For config tuning: https://pgtune.leopard.in.ua/#/ Though please go through all params for more understanding

 

Happy to help :)

Prince Pathria Systems Architect Intern Evive +91 9478670472 goevive.com

On Thu, Feb 14, 2019 at 09:38:52AM +0000, suganthi Sekar wrote:
>  i am using Postgresql 11, i have 2 partition table , when i joined both table in query
> a table  its goes exact partition table  , but other table scan all partition
> 
> please clarify on this .
> 
> Example :
> 
> explain analyze
> select * from call_report1 as a  inner join  call_report2 as b on a.call_id=b.call_id
>  where a.call_created_date ='2017-11-01' and '2017-11-30'

Looks like this query waas manally editted and should say:
>  where a.call_created_date >='2017-11-01' AND a.call_created_date<'2017-11-30'
Right?

The issue is described well here:
https://www.postgresql.org/message-id/flat/7DF51702-0F6A-4571-80BB-188AAEF260DA%40gmail.com
https://www.postgresql.org/message-id/499.1496696552%40sss.pgh.pa.us

You can work around it by specifying the same condition on b.call_created_date:
>  AND b.call_created_date >='2017-11-01' AND b.call_created_date<'2017-11-30'

Justin

On Thu, Feb 14, 2019 at 10:38:36AM +0000, suganthi Sekar wrote:
> u mean the below parameter need to set on . its already on only.
>   alter system set  constraint_exclusion  to 'on';

No, I said:
> You can work around it by specifying the same condition on b.call_created_date:
> >  AND b.call_created_date >='2017-11-01' AND b.call_created_date<'2017-11-30'

suganthi Sekar wrote:
>  i am using Postgresql 11, i have 2 partition table , when i joined both table in query 
> a table  its goes exact partition table  , but other table scan all partition
> 
> please clarify on this .
> 
> i have enabled below parameter on in configuration file
> Note  :  alter system set enable_partitionwise_join  to 'on';
> 
> 
> Example : 
> 
> explain analyze
> select * from call_report1 as a  inner join  call_report2 as b on a.call_id=b.call_id
>  where a.call_created_date ='2017-11-01' and '2017-11-30'
> 
> 
> 
>  "Hash Right Join  (cost=8.19..50.47 rows=2 width=3635) (actual time=0.426..0.447 rows=7 loops=1)"
> "  Hash Cond: (b.call_id = a.call_id)"
> "  ->  Append  (cost=0.00..41.81 rows=121 width=2319) (actual time=0.040..0.170 rows=104 loops=1)"
> "        ->  Seq Scan on call_report2 b  (cost=0.00..0.00 rows=1 width=528) (actual time=0.010..0.010 rows=0
loops=1)"
> "        ->  Seq Scan on call_report2_201803 b_1  (cost=0.00..10.30 rows=30 width=2334) (actual time=0.029..0.031
rows=14loops=1)"
 
> "        ->  Seq Scan on call_report2_201711 b_2  (cost=0.00..10.30 rows=30 width=2334) (actual time=0.014..0.015
rows=7loops=1)"
 
> "        ->  Seq Scan on call_report2_201712 b_3  (cost=0.00..10.30 rows=30 width=2334) (actual time=0.017..0.047
rows=34loops=1)"
 
> "        ->  Seq Scan on call_report2_201801 b_4  (cost=0.00..10.30 rows=30 width=2334) (actual time=0.017..0.058
rows=49loops=1)"
 
> "  ->  Hash  (cost=8.17..8.17 rows=2 width=1314) (actual time=0.104..0.104 rows=7 loops=1)"
> "        Buckets: 1024  Batches: 1  Memory Usage: 12kB"
> "        ->  Append  (cost=0.00..8.17 rows=2 width=1314) (actual time=0.053..0.060 rows=7 loops=1)"
> "              ->  Seq Scan on call_report1 a  (cost=0.00..0.00 rows=1 width=437) (actual time=0.022..0.022 rows=0
loops=1)"
> "                    Filter: ((call_created_date >= '2017-11-01'::date) AND (call_created_date <=
'2017-11-30'::date))"
> "              ->  Index Scan using idx_call_report1_201711_ccd on call_report1_201711 a_1  (cost=0.14..8.16 rows=1
width=2190)(actual time=0.029..0.034 rows=7 loops=1)"
 
> "                    Index Cond: ((call_created_date >= '2017-11-01'::date) AND (call_created_date <=
'2017-11-30'::date))"
> "Planning Time: 20.866 ms"
> "Execution Time: 1.205 ms"

There is no condition on the table "call_report2" in your query,
so it is not surprising that all partitions are scanned, right?

You have to add a WHERE condition that filters on the partitioning
column(s) of "call_report2".

Yours,
Laurenz Albe
-- 
Cybertec | https://www.cybertec-postgresql.com

Both table Portion by  same column call_created_date
________________________________
From: Michael Lewis <mlewis@entrata.com>
Sent: 14 February 2019 19:35:48
To: suganthi Sekar
Cc: Justin Pryzby; pgsql-performance@lists.postgresql.org
Subject: Re: constraint exclusion with ineq condition (Re: server hardware tuning.)

What are these two tables partitioned by?

On Thu, Feb 14, 2019, 5:03 AM suganthi Sekar <suganthi@uniphore.com<mailto:suganthi@uniphore.com> wrote:
Hi,

Thanks, i know if explicitly we give in where condition it is working.

i thought with below parameter in Postgresq11 this issue is fixed ?

 enable_partitionwise_join  to 'on';

 what is the use of enable_partitionwise_join  to 'on';

Thanks for your response.

Regards
Suganthi Sekar
________________________________
From: Justin Pryzby <pryzby@telsasoft.com<mailto:pryzby@telsasoft.com>>
Sent: 14 February 2019 16:10:01
To: suganthi Sekar
Cc: pgsql-performance@lists.postgresql.org<mailto:pgsql-performance@lists.postgresql.org>
Subject: Re: constraint exclusion with ineq condition (Re: server hardware tuning.)

On Thu, Feb 14, 2019 at 10:38:36AM +0000, suganthi Sekar wrote:
> u mean the below parameter need to set on . its already on only.
>   alter system set  constraint_exclusion  to 'on';

No, I said:
> You can work around it by specifying the same condition on b.call_created_date:
> >  AND b.call_created_date >='2017-11-01' AND b.call_created_date<'2017-11-30'

On Thu, Feb 14, 2019 at 01:37:49PM +0100, Laurenz Albe wrote:
> There is no condition on the table "call_report2" in your query,
> so it is not surprising that all partitions are scanned, right?

Some people find it surprising, since: a.call_id=b.call_id

suganthi Sekar wrote:
> > explain analyze
> > select * from call_report1 as a  inner join  call_report2 as b on a.call_id=b.call_id
> >  where a.call_created_date ='2017-11-01' and '2017-11-30'

Justin

On Fri, Feb 15, 2019 at 12:24:18PM -0500, Benedict Holland wrote:
> That sounds like a perfect task for a view if the joins are all the same.

But note that either the view itself needs to have both where clauses (with
hardcoded dates?), or otherwise the view needs to be on only one table, and the
toplevel query needs to have where clause on each view, or else one of the
tables won't get constraint exclusion.

On Fri, Feb 15, 2019 at 9:36 AM suganthi Sekar <suganthi@uniphore.com> wrote:
> > yes i accept , but when i will do for existing tables, i am facing issue.
> >
> > I have created 100 Function , all the function having  five table join(*now
> > all partition by date*) , now its not possible to change where condition
> > in all 100 Function.
> >
> > so that i am trying any other possibilities are there.

> > From: Justin Pryzby <pryzby@telsasoft.com<mailto:pryzby@telsasoft.com>>
> > Sent: 14 February 2019 16:10:01
> > To: suganthi Sekar
> > Cc: pgsql-performance@lists.postgresql.org<mailto:
> > pgsql-performance@lists.postgresql.org>
> > Subject: Re: constraint exclusion with ineq condition (Re: server hardware
> > tuning.)
> >
> > On Thu, Feb 14, 2019 at 10:38:36AM +0000, suganthi Sekar wrote:
> > > u mean the below parameter need to set on . its already on only.
> > >   alter system set  constraint_exclusion  to 'on';
> >
> > No, I said:
> > > You can work around it by specifying the same condition on
> > b.call_created_date:
> > > >  AND b.call_created_date >='2017-11-01' AND
> > b.call_created_date<'2017-11-30'

-- 
Justin Pryzby
System Administrator
Telsasoft
+1-952-707-8581