VACUUM memory management - Mailing list pgsql-hackers

From Ibrar Ahmed
Subject VACUUM memory management
Date
Msg-id CALtqXTdbBKRLdUuekg-2iYN06EX7BrodU1PheNgB6Bx6SuvCvg@mail.gmail.com
Whole thread Raw
Responses Re: VACUUM memory management  (Alvaro Herrera <alvherre@2ndquadrant.com>)
List pgsql-hackers
Hi,

The memory consumption of VACUUM has some issues and could be improved. Some of its limitations are recorded in the comments of the “vacuumlazy.c” file. The current design of VACUUM memory usage is that it stores the TID in a fixed-size array which is allocated at the start, based upon maintenance_work_mem. There are three problems with that design

 - If the value of maintenance_work_mem is too large then it is a waste of memory for small tables.
 - If the value of maintenance_work_mem is too small or “TIDs” do not fit in the array then multiple scans happen.
 - In cases where maintainess_work_mem is set too large, and we have a bigger value of vacuume_count, then the system can be out-of-memory.

There are two solutions for these problems. The first is to use a list instead of a fixed size array.  The second solution is to allocate the memory in chunks.
The attached WIP patch creates an array of ItemPointers and allocates memory in chunks by dividing the maintenance_work_mem into multiple chunks.  

Comments?
--
Ibrar Ahmed
Attachment

pgsql-hackers by date:

Previous
From: "Karl O. Pinc"
Date:
Subject: Re: proposal: minscale, rtrim, btrim functions for numeric
Next
From: Tom Lane
Date:
Subject: Re: Windows buildfarm members vs. new async-notify isolation test