Failed to delete old ReorderBuffer spilled files - Mailing list pgsql-hackers

From atorikoshi
Subject Failed to delete old ReorderBuffer spilled files
Date
Msg-id 54e4e488-186b-a056-6628-50628e4e4ebc@lab.ntt.co.jp
Whole thread Raw
Responses Re: Failed to delete old ReorderBuffer spilled files  (Masahiko Sawada <sawada.mshk@gmail.com>)
Re: Failed to delete old ReorderBuffer spilled files  (Craig Ringer <craig@2ndquadrant.com>)
List pgsql-hackers
Hi,

I put many queries into one transaction and made ReorderBuffer spill
data to disk, and sent SIGKILL to postgres before the end of the
transaction.

After starting up postgres again, I observed the files spilled to
data wasn't deleted.

I think these files should be deleted because its transaction was no
more valid, so no one can use these files.


Below is a reproduction instructions.

------------------------------------------------
1. Create table and publication at publiser.

   @pub =# CREATE TABLE t1(
       id INT PRIMARY KEY,
       name TEXT);

   @pub =# CREATE PUBLICATION pub FOR TABLE t1;

2. Create table and subscription at subscriber.

   @sub =# CREATE TABLE t1(
       id INT PRIMARY KEY,
       name TEXT
       );

   @sub =# CREATE SUBSCRIPTION sub
       CONNECTION 'host=[hostname] port=[port] dbname=[dbname]'
       PUBLICATION pub;

3. Put many queries into one transaction.

   @pub =# BEGIN;
         INSERT INTO t1
         SELECT
           i,
           'aaaaaaaaaa'
         FROM
         generate_series(1, 1000000) as i;

4. Then we can see spilled files.

   @pub $ ls -1 ${PGDATA}/pg_replslot/sub/
     state
     xid-561-lsn-0-1000000.snap
     xid-561-lsn-0-2000000.snap
     xid-561-lsn-0-3000000.snap
     xid-561-lsn-0-4000000.snap
     xid-561-lsn-0-5000000.snap
     xid-561-lsn-0-6000000.snap
     xid-561-lsn-0-7000000.snap
     xid-561-lsn-0-8000000.snap
     xid-561-lsn-0-9000000.snap

5. Kill publisher's postgres process before COMMIT.

   @pub $ kill -s SIGKILL [pid of postgres]

6. Start publisher's postgres process.

   @pub $ pg_ctl start -D ${PGDATA}

7. After a while, we can see the files remaining.
   (Immediately after starting publiser, we can not see these files.)

   @pub $ pg_ctl start -D ${PGDATA}

   When I configured with '--enable-cassert', below assertion error
   was appeared.

     TRAP: FailedAssertion("!(txn->final_lsn != 0)", File:  
"reorderbuffer.c", Line: 2576)
------------------------------------------------

Attached patch sets final_lsn to the last ReorderBufferChange if
final_lsn == 0.

--
Atsushi Torikoshi
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


Attachment

pgsql-hackers by date:

Previous
From: Ildus Kurbangaliev
Date:
Subject: Re: [HACKERS] Custom compression methods
Next
From: Dmitry Dolgov
Date:
Subject: Re: [HACKERS] [PATCH] Generic type subscripting