Thread: Heap page diagnostic/test functions (v2)
New functions to examine the contents of heap pages, as discussed recently on -hackers. These are fully integrated into backend. Designed to be extended for other page layouts/contents. (Heikki has some similar functions for index pages). Docs included, applies cleanly, tests good. I'll maintain this with immediate fixes/additions as we go up to 8.3 and beyond, to assist review process of various patches that alter page contents. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com
Attachment
On Thu, 2007-03-08 at 15:44 +0000, Simon Riggs wrote: > Docs included Just noticed a typo. File mentioned in func.sgml, line 11049 should be <filename>src/include/access/htup.h</> and not <filename>src/include/storage/bufpage.h</> -- Simon Riggs EnterpriseDB http://www.enterprisedb.com
Your patch has been added to the PostgreSQL unapplied patches list at:
http://momjian.postgresql.org/cgi-bin/pgpatches
It will be applied as soon as one of the PostgreSQL committers reviews
and approves it.
---------------------------------------------------------------------------
Simon Riggs wrote:
> New functions to examine the contents of heap pages, as discussed
> recently on -hackers. These are fully integrated into backend.
>
> Designed to be extended for other page layouts/contents. (Heikki has
> some similar functions for index pages).
>
> Docs included, applies cleanly, tests good.
>
> I'll maintain this with immediate fixes/additions as we go up to 8.3 and
> beyond, to assist review process of various patches that alter page
> contents.
>
> --
> Simon Riggs
> EnterpriseDB http://www.enterprisedb.com
>
[ Attachment, skipping... ]
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Your patch has been added to the PostgreSQL unapplied patches list at:
http://momjian.postgresql.org/cgi-bin/pgpatches
It will be applied as soon as one of the PostgreSQL committers reviews
and approves it.
---------------------------------------------------------------------------
Simon Riggs wrote:
> On Thu, 2007-03-08 at 15:44 +0000, Simon Riggs wrote:
>
> > Docs included
>
> Just noticed a typo. File mentioned in func.sgml, line 11049 should be
> <filename>src/include/access/htup.h</> and not
> <filename>src/include/storage/bufpage.h</>
>
> --
> Simon Riggs
> EnterpriseDB http://www.enterprisedb.com
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faq
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
This looks useful, but shouldn't it be part of /contrib/pgstattuple rather than in the backend? --------------------------------------------------------------------------- Simon Riggs wrote: > New functions to examine the contents of heap pages, as discussed > recently on -hackers. These are fully integrated into backend. > > Designed to be extended for other page layouts/contents. (Heikki has > some similar functions for index pages). > > Docs included, applies cleanly, tests good. > > I'll maintain this with immediate fixes/additions as we go up to 8.3 and > beyond, to assist review process of various patches that alter page > contents. > > -- > Simon Riggs > EnterpriseDB http://www.enterprisedb.com > [ Attachment, skipping... ] > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Have you searched our list archives? > > http://archives.postgresql.org -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
On Sat, 2007-04-07 at 11:53 -0400, Bruce Momjian wrote: > This looks useful, but shouldn't it be part of /contrib/pgstattuple > rather than in the backend? Well, this was written with a view to it being usable for writing test cases that checked the various tuple states as we went. It was originally proposed when Tom asked "How will we test HOT?" (concurrent psql is the other half of the required base functionality to write sensible test cases). If we see it as a manual test tool only, contrib is OK. But it has to be in the backend to be usable in the regression test suite, so thats where it was designed to go. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com
Simon Riggs wrote: > On Sat, 2007-04-07 at 11:53 -0400, Bruce Momjian wrote: > > > This looks useful, but shouldn't it be part of /contrib/pgstattuple > > rather than in the backend? > > Well, this was written with a view to it being usable for writing test > cases that checked the various tuple states as we went. It was > originally proposed when Tom asked "How will we test HOT?" (concurrent > psql is the other half of the required base functionality to write > sensible test cases). > > If we see it as a manual test tool only, contrib is OK. But it has to be > in the backend to be usable in the regression test suite, so thats where > it was designed to go. Well, contrib can have its own regression tests. We can put the HOT tests in there too. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
"Simon Riggs" <simon@2ndquadrant.com> writes:
> If we see it as a manual test tool only, contrib is OK. But it has to be
> in the backend to be usable in the regression test suite, so thats where
> it was designed to go.
The core regression tests have depended on some contrib stuff forever,
so the above argument holds no water.
I'm uncomfortable with putting these things in core because I suspect
it's not hard to crash the backend (or worse) by feeding one of them
artfully corrupted data.
regards, tom lane
On Sat, 2007-04-07 at 18:19 -0400, Bruce Momjian wrote: > Simon Riggs wrote: > > On Sat, 2007-04-07 at 11:53 -0400, Bruce Momjian wrote: > > > > > This looks useful, but shouldn't it be part of /contrib/pgstattuple > > > rather than in the backend? > > > > Well, this was written with a view to it being usable for writing test > > cases that checked the various tuple states as we went. It was > > originally proposed when Tom asked "How will we test HOT?" (concurrent > > psql is the other half of the required base functionality to write > > sensible test cases). > > > > If we see it as a manual test tool only, contrib is OK. But it has to be > > in the backend to be usable in the regression test suite, so thats where > > it was designed to go. > > Well, contrib can have its own regression tests. We can put the HOT > tests in there too. OK. I'll rework it once I've completed the other items on my list. Feel free to have a hack at it if I'm a little slow. I only have today left before I'm away for two weeks - travelling tomorrow. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com
Simon Riggs wrote: > New functions to examine the contents of heap pages, as discussed > recently on -hackers. These are fully integrated into backend. > ... > I'll maintain this with immediate fixes/additions as we go up to 8.3 and > beyond, to assist review process of various patches that alter page > contents. Excellent. I agree with the other comments that these should all be in contrib. The functions need to be very careful to handle invalid data gracefully. For example, this will return bogus data or crash if the offset in the line pointer is corrupt, past the end of page for example: + tuphdr = (HeapTupleHeader) PageGetItem((Page) inter_call_data->page, id); It's particularly important if the page-examining functions are not superuser-only, to avoid vulnerabilities. Even if they are, the functions are for diagnostic purposes and diagnosis is often done on something that's corrupt. Dare I propose deprecating the all the system columns except for ctid in favor of these functions? I guess there's no immediate reason to remove them, but these functions feel like a much better design than tightly integrated system columns. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com