Memory leak in CachememoryContext - Mailing list pgsql-hackers
| From | Ajit Awekar |
|---|---|
| Subject | Memory leak in CachememoryContext |
| Date | |
| Msg-id | CAHv6PyrNaqdvyWUspzd3txYQguFTBSnhx+m6tS06TnM+KWc_LQ@mail.gmail.com Whole thread Raw |
| Responses |
Re: Memory leak in CachememoryContext
|
| List | pgsql-hackers |
Please find below simple repro for CacheMemoryContext memory leak
create type two_int4s as (f1 int4, f2 int4);
create type two_int8s as (q1 int8, q2 int8);
PLpgSQL example:
do $$ declare c4 two_int4s; c8 two_int8s;
begin
c8 := row(1,2);
c4 := c8;
end$$;
Executing above plpgsql in same memory session we observe cachememorycontext goes on increasing as below which is captured using MemoryContextStats
1590:2023-04-19 13:31:54.336 IST [31615] LOG: Grand total: 1213440 bytes in 153 blocks; 496000 free (53 chunks); 717440 used
1687:2023-04-19 13:31:54.348 IST [31615] LOG: Grand total: 1220608 bytes in 160 blocks; 497160 free (53 chunks); 723448 used
1781:2023-04-19 13:31:59.919 IST [31615] LOG: Grand total: 1213440 bytes in 154 blocks; 494168 free (45 chunks); 719272 used
1880:2023-04-19 13:31:59.924 IST [31615] LOG: Grand total: 1220608 bytes in 161 blocks; 496128 free (45 chunks); 724480 used
1976:2023-04-19 13:32:29.977 IST [31615] LOG: Grand total: 1215488 bytes in 156 blocks; 495144 free (45 chunks); 720344 used
2077:2023-04-19 13:32:29.978 IST [31615] LOG: Grand total: 1222656 bytes in 163 blocks; 497104 free (45 chunks); 725552 used
Root cause:
Memory leak is in function "GetCachedExpression" which creates context under CacheMemoryContext. During each execution in the same session memory gets allocated and it is never freed resulting in memory leak.
During anonymous block execution in the function "plpgsql_estate_setup", a local casting hash table gets created in SPI memory context. When hash table look up is performed in "get_cast_hashenty" function if entry is no present , memory is allocated in CacheMemoryContext in function "GetCachedExpression".At the end of proc execution SPI memory context is deleted and hence local hash table gets deleted, but still entries remain in Cachemeorycontext.
create type two_int4s as (f1 int4, f2 int4);
create type two_int8s as (q1 int8, q2 int8);
PLpgSQL example:
do $$ declare c4 two_int4s; c8 two_int8s;
begin
c8 := row(1,2);
c4 := c8;
end$$;
Executing above plpgsql in same memory session we observe cachememorycontext goes on increasing as below which is captured using MemoryContextStats
1590:2023-04-19 13:31:54.336 IST [31615] LOG: Grand total: 1213440 bytes in 153 blocks; 496000 free (53 chunks); 717440 used
1687:2023-04-19 13:31:54.348 IST [31615] LOG: Grand total: 1220608 bytes in 160 blocks; 497160 free (53 chunks); 723448 used
1781:2023-04-19 13:31:59.919 IST [31615] LOG: Grand total: 1213440 bytes in 154 blocks; 494168 free (45 chunks); 719272 used
1880:2023-04-19 13:31:59.924 IST [31615] LOG: Grand total: 1220608 bytes in 161 blocks; 496128 free (45 chunks); 724480 used
1976:2023-04-19 13:32:29.977 IST [31615] LOG: Grand total: 1215488 bytes in 156 blocks; 495144 free (45 chunks); 720344 used
2077:2023-04-19 13:32:29.978 IST [31615] LOG: Grand total: 1222656 bytes in 163 blocks; 497104 free (45 chunks); 725552 used
Root cause:
Memory leak is in function "GetCachedExpression" which creates context under CacheMemoryContext. During each execution in the same session memory gets allocated and it is never freed resulting in memory leak.
During anonymous block execution in the function "plpgsql_estate_setup", a local casting hash table gets created in SPI memory context. When hash table look up is performed in "get_cast_hashenty" function if entry is no present , memory is allocated in CacheMemoryContext in function "GetCachedExpression".At the end of proc execution SPI memory context is deleted and hence local hash table gets deleted, but still entries remain in Cachemeorycontext.
During the next execution in the same session, a brand new hash table is created and if entry is not present memory will be repeatedly assigned in CacheMemoryContext.
Solution:
Please find attached(memoryleakfix.patch) to this email. We need to keep track of the local casting hash table or session wide cast hash table which gets created in the function "plpgsql_estate_setup". We need to allocate CacheMemorycontext only for session wide cast hash table and for local cast hash table memory will be allocated from SPI context.
Please find below CacheMemory Context stats with fix as below
3316:2023-04-19 14:07:23.391 IST [38021] LOG: Grand total: 1210368 bytes in 151 blocks; 492704 free (45 chunks); 717664 used
3411:2023-04-19 14:07:23.391 IST [38021] LOG: Grand total: 1216512 bytes in 157 blocks; 494176 free (45 chunks); 722336 used
3502:2023-04-19 14:07:23.932 IST [38021] LOG: Grand total: 1210368 bytes in 151 blocks; 492704 free (45 chunks); 717664 used
3597:2023-04-19 14:07:23.932 IST [38021] LOG: Grand total: 1216512 bytes in 157 blocks; 494176 free (45 chunks); 722336 used
3688:2023-04-19 14:07:24.464 IST [38021] LOG: Grand total: 1210368 bytes in 151 blocks; 492704 free (45 chunks); 717664 used
3783:2023-04-19 14:07:24.464 IST [38021] LOG: Grand total: 1216512 bytes in 157 blocks; 494176 free (45 chunks); 722336 used
3874:2023-04-19 14:07:25.012 IST [38021] LOG: Grand total: 1210368 bytes in 151 blocks; 492704 free (45 chunks); 717664 used
3969:2023-04-19 14:07:25.012 IST [38021] LOG: Grand total: 1216512 bytes in 157 blocks; 494176 free (45 chunks); 722336 used
4060:2023-04-19 14:07:25.552 IST [38021] LOG: Grand total: 1210368 bytes in 151 blocks; 492704 free (45 chunks); 717664 used
4155:2023-04-19 14:07:25.552 IST [38021] LOG: Grand total: 1216512 bytes in 157 blocks; 494176 free (45 chunks); 722336 used
Thanks & Best Regards,
Solution:
Please find attached(memoryleakfix.patch) to this email. We need to keep track of the local casting hash table or session wide cast hash table which gets created in the function "plpgsql_estate_setup". We need to allocate CacheMemorycontext only for session wide cast hash table and for local cast hash table memory will be allocated from SPI context.
Please find below CacheMemory Context stats with fix as below
3316:2023-04-19 14:07:23.391 IST [38021] LOG: Grand total: 1210368 bytes in 151 blocks; 492704 free (45 chunks); 717664 used
3411:2023-04-19 14:07:23.391 IST [38021] LOG: Grand total: 1216512 bytes in 157 blocks; 494176 free (45 chunks); 722336 used
3502:2023-04-19 14:07:23.932 IST [38021] LOG: Grand total: 1210368 bytes in 151 blocks; 492704 free (45 chunks); 717664 used
3597:2023-04-19 14:07:23.932 IST [38021] LOG: Grand total: 1216512 bytes in 157 blocks; 494176 free (45 chunks); 722336 used
3688:2023-04-19 14:07:24.464 IST [38021] LOG: Grand total: 1210368 bytes in 151 blocks; 492704 free (45 chunks); 717664 used
3783:2023-04-19 14:07:24.464 IST [38021] LOG: Grand total: 1216512 bytes in 157 blocks; 494176 free (45 chunks); 722336 used
3874:2023-04-19 14:07:25.012 IST [38021] LOG: Grand total: 1210368 bytes in 151 blocks; 492704 free (45 chunks); 717664 used
3969:2023-04-19 14:07:25.012 IST [38021] LOG: Grand total: 1216512 bytes in 157 blocks; 494176 free (45 chunks); 722336 used
4060:2023-04-19 14:07:25.552 IST [38021] LOG: Grand total: 1210368 bytes in 151 blocks; 492704 free (45 chunks); 717664 used
4155:2023-04-19 14:07:25.552 IST [38021] LOG: Grand total: 1216512 bytes in 157 blocks; 494176 free (45 chunks); 722336 used
Thanks & Best Regards,
Ajit
Attachment
pgsql-hackers by date: