Re: BUG #16663: DROP INDEX did not free up disk space: idle connection hold file marked as deleted - Mailing list pgsql-hackers

From David Zhang
Subject Re: BUG #16663: DROP INDEX did not free up disk space: idle connection hold file marked as deleted
Date
Msg-id 160624299337.7563.13562988821961887884.pgcf@coridan.postgresql.org
Whole thread Raw
Responses Re: BUG #16663: DROP INDEX did not free up disk space: idle connection hold file marked as deleted  (Pavel Borisov <pashkin.elfe@gmail.com>)
List pgsql-hackers
I verified the patch "v2-0001-Free-disk-space-for-dropped-relations-on-commit.patch" on master branch
"0cc99327888840f2bf572303b68438e4caf62de9".It works for me. Below is my test procedure and results.
 

=== Before the patch ===
#1 from psql console 1, create table and index then insert enough data
postgres=# CREATE TABLE test_tbl ( a int, b text);
postgres=# CREATE INDEX idx_test_tbl on test_tbl (a);
postgres=# INSERT INTO test_tbl SELECT generate_series(1,80000000),'Hello world!';
postgres=# INSERT INTO test_tbl SELECT generate_series(1,80000000),'Hello world!';

#2 check files size 
david:12867$ du -h
12G    .

#3 from psql console 2, drop the index
postgres=# drop index idx_test_tbl;

#4 check files size in different ways,
david:12867$ du -h
7.8G    .
david:12867$ ls -l
...
-rw------- 1 david david          0 Nov 23 20:07 16402
...

$ lsof -nP | grep '(deleted)' |grep pgdata
...
postgres  25736                  david   45u      REG              259,2          0   12592758
/home/david/sandbox/postgres/pgdata/base/12867/16402(deleted)
 
postgres  25736                  david   49u      REG              259,2 1073741824   12592798
/home/david/sandbox/postgres/pgdata/base/12867/16402.1(deleted)
 
postgres  25736                  david   53u      REG              259,2 1073741824   12592739
/home/david/sandbox/postgres/pgdata/base/12867/16402.2(deleted)
 
postgres  25736                  david   59u      REG              259,2  372604928   12592800
/home/david/sandbox/postgres/pgdata/base/12867/16402.3(deleted)
 
...

The index relnode id "16402" displays size "0" from postgres database folder, but when using lsof to check, all 16402.x
arestill in used by a psql connection except 16402 is set to 0. Check it again after an hour, lsof shows the same
results.

=== After the patch ===
Repeat step 1 ~ 4, lsof shows all the index relnode files (in this case, the index relnode id 16389) are removed within
about1 minute.
 
$ lsof -nP | grep '(deleted)' |grep pgdata
...
postgres  32707                  david   66u      REG              259,2          0   12592763
/home/david/sandbox/postgres/pgdata/base/12867/16389.1(deleted)
 
postgres  32707                  david   70u      REG              259,2          0   12592823
/home/david/sandbox/postgres/pgdata/base/12867/16389.2(deleted)
 
postgres  32707                  david   74u      REG              259,2          0   12592805
/home/david/sandbox/postgres/pgdata/base/12867/16389.3(deleted)
 
...

One thing interesting for me is that, if the index is created after data records has been inserted, then lsof doesn't
showthis issue. 

pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: libpq compression
Next
From: Robert Haas
Date:
Subject: Re: [HACKERS] Custom compression methods