Re: Freeze avoidance of very large table. - Mailing list pgsql-hackers

From Jeff Janes
Subject Re: Freeze avoidance of very large table.
Date
Msg-id CAMkU=1xKDq0oUmvC1O-J6nguNcjUWitMbvxrxZ5yPvPe0Fv18w@mail.gmail.com
Whole thread Raw
In response to Re: Freeze avoidance of very large table.  (Masahiko Sawada <sawada.mshk@gmail.com>)
Responses Re: Freeze avoidance of very large table.  (Masahiko Sawada <sawada.mshk@gmail.com>)
List pgsql-hackers
On Mon, Nov 30, 2015 at 9:18 AM, Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> On Sun, Nov 29, 2015 at 2:21 AM, Jeff Janes <jeff.janes@gmail.com> wrote:
>> On Tue, Nov 24, 2015 at 3:13 PM, Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>>>
>>> Yeah, we need to consider to compute checksum if enabled.
>>> I've changed the patch, and attached.
>>> Please review it.
>>
>> Thanks for the update.  This now conflicts with the updates doesn to
>> fix pg_upgrade out-of-space issue on Windows. I've fixed (I think) the
>> conflict in order to do some testing, but I'd like to get an updated
>> patch from the author in case I did it wrong.  I don't want to find
>> bugs that I just introduced myself.
>>
>
> Thank you for having a look.
>
> Attached updated v28 patch.
> Please review it.
>
> Regards,

After running pg_upgrade, if I manually vacuum a table a start getting warnings:

WARNING:  page is not marked all-visible (and all-frozen) but
visibility map bit(s) is set in relation "foo" page 32756
WARNING:  page is not marked all-visible (and all-frozen) but
visibility map bit(s) is set in relation "foo" page 32756
WARNING:  page is not marked all-visible (and all-frozen) but
visibility map bit(s) is set in relation "foo" page 32757
WARNING:  page is not marked all-visible (and all-frozen) but
visibility map bit(s) is set in relation "foo" page 32757

The warnings are right where the blocks would start using the 2nd page
of the _vm, so I think the problem is there.  And looking at the code,
I think that "cur += SizeOfPageHeaderData;" in the inner loop cannot
be correct.  We can't skip a header in the current (old) block each
time we reach the end of the new block.  The thing we are skipping in
the current block is half the time not a header, but the data at the
halfway point through the block.

Cheers,

Jeff



pgsql-hackers by date:

Previous
From: Julien Rouhaud
Date:
Subject: Re: On-demand running query plans using auto_explain and signals
Next
From: Julien Rouhaud
Date:
Subject: gincostestimate and hypothetical indexes