Re: need suggestion on querying big tables - Mailing list pgsql-admin

From Ron
Subject Re: need suggestion on querying big tables
Date
Msg-id d926084d-63a7-854b-2d8d-459d33ec7f70@gmail.com
Whole thread Raw
In response to need suggestion on querying big tables  (Ebin Jozer <ebinjozer@gmail.com>)
Responses Re: need suggestion on querying big tables  (Thomas Kellerer <shammat@gmx.net>)
Re: need suggestion on querying big tables  (Tom Lane <tgl@sss.pgh.pa.us>)
Re: need suggestion on querying big tables  (Scott Ribe <scott_ribe@elevated-dev.com>)
List pgsql-admin
On 12/5/22 00:29, Ebin Jozer wrote:
Hi Team,
in postgresql 11 version we have two tables of size 435 GB and 347 GB. 
if we query on single table or if we are doing inner join on both the big tables, it is not displacing any output, it keeps running 

We can see the wait event is IO and directDatafile .

Server Spec : 8 cores and  64GB RAM
PG config : 53 GB(effective_cache), 12 GB(shared buffer) 

can you please suggest some ideas , how we can query on big tables and fasten them to get the output??

https://www.postgresql.org/docs/11/runtime-config-query.html

effective_cache_size is how much RAM a single query gets.  "When setting this parameter you should consider both PostgreSQL's shared buffers".

https://www.postgresql.org/docs/11/runtime-config-resource.html

"a reasonable starting value for shared_buffers is 25% of the memory in your system. There are some workloads where even larger settings for shared_buffers are effective, but because PostgreSQL also relies on the operating system cache, it is unlikely that an allocation of more than 40% of RAM to shared_buffers will work better than a smaller amount."

You've set effective_cache_size to 83%, when it should be at most 20%.

--
Angular momentum makes the world go 'round.

pgsql-admin by date:

Previous
From: Ebin Jozer
Date:
Subject: Re: need suggestion on querying big tables
Next
From: Thomas Kellerer
Date:
Subject: Re: need suggestion on querying big tables