Re: Checkpoint cost, looks like it is WAL/CRC - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Checkpoint cost, looks like it is WAL/CRC
Date
Msg-id 7278.1120743488@sss.pgh.pa.us
Whole thread Raw
In response to Re: Checkpoint cost, looks like it is WAL/CRC  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Checkpoint cost, looks like it is WAL/CRC  (Josh Berkus <josh@agliodbs.com>)
Re: Checkpoint cost, looks like it is WAL/CRC  (Josh Berkus <josh@agliodbs.com>)
List pgsql-hackers
I wrote:
> We still don't know enough about the situation to know what a solution
> might look like.  Is the slowdown Josh is seeing due to the extra CPU
> cost of the CRCs, or the extra I/O cost, or excessive locking of the
> WAL-related data structures while we do this stuff, or ???.  Need more
> data.

Josh, is OSDL up enough that you can try another comparison run?
If so, please undo the previous patch (which disabled page dumping
entirely) and instead try removing this block of code, starting
at about xlog.c line 620 in CVS tip:
   /*    * Now add the backup block headers and data into the CRC    */   for (i = 0; i < XLR_MAX_BKP_BLOCKS; i++)   {
    if (dtbuf_bkp[i])       {           BkpBlock   *bkpb = &(dtbuf_xlg[i]);           char       *page;
 
           COMP_CRC32(rdata_crc,                      (char *) bkpb,                      sizeof(BkpBlock));
page= (char *) BufferGetBlock(dtbuf[i]);           if (bkpb->hole_length == 0)           {
COMP_CRC32(rdata_crc,                         page,                          BLCKSZ);           }           else
  {               /* must skip the hole */               COMP_CRC32(rdata_crc,                          page,
              bkpb->hole_offset);               COMP_CRC32(rdata_crc,                          page +
(bkpb->hole_offset+ bkpb->hole_length),                          BLCKSZ - (bkpb->hole_offset + bkpb->hole_length));
     }       }   }
 

This will remove just the CRC calculation work associated with backed-up
pages.  Note that any attempt to recover from the WAL will fail, but I
assume you don't need that for the purposes of the test run.
        regards, tom lane


pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Checkpoint cost, looks like it is WAL/CRC
Next
From: "Zeugswetter Andreas DAZ SD"
Date:
Subject: Re: Checkpoint cost, looks like it is WAL/CRC