Thread: Slow Index Creation, why is it not consuming more memory.

Slow Index Creation, why is it not consuming more memory.

From
Tory M Blue
Date:

Just trying to figure something out.

9.3.4, CentOS6.5
256GB Ram

Maintenance_work_mem = 125GB
Effective_Cache = 65GB

I have 2 indexes running, started at the same time, they are not small and one will take 7 hours to complete.

I see almost zero disk access, very minor, not what I want to see when I have an index that will be as large as it is! , but whatever.

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND   

18059 postgres  20   0 5119m 2.4g  20m R 100.0  0.9  12:31.05 postmaster                                               

18091 postgres  20   0 9794m 6.9g  20m R 100.0  2.7  11:41.61 postmaster 

Why are 2 index using different amounts of resident mem, when they have the keys to the castle, and why are they not taking more?

I've tried this with as low as 4GB of maintenance_work_mem and the numbers in TOP stay the same.

Work_mem is 7GB.

What am I not understanding missing?

Thanks
Tory

Re: Slow Index Creation, why is it not consuming more memory.

From
Peter Geoghegan
Date:
On Mon, Dec 7, 2015 at 10:36 PM, Tory M Blue <tmblue@gmail.com> wrote:
> What am I not understanding missing?

Yes. There is a hard limit on the number of tuples than can be sorted
in memory prior to PostgreSQL 9.4. It's also the case that very large
work_mem or maintenance_work_mem settings are unlikely to help unless
they result in a fully internal sort.

There is evidence that the heap that tuple sorting uses benefits from
*lower* settings. Sometimes as low as 64MB.

We're working to make this better in 9.6.

--
Regards,
Peter Geoghegan