Thread: SearchSysCache() tutorial?

SearchSysCache() tutorial?

From
Paul Ramsey
Date:
Is there anywhere any documentation on SearchSysCache? I find I end up
on these long spelunking expeditions through the code base for a
particular problem, find the answer after many hours, then forget
everything I learned because I don't exercise my knowledge frequently
enough. A decent reference guide would help a lot. What do the various
SysCacheIdentifier numbers mean/search, some examples, of usage, etc,
etc.

I can accept if there are not, but it would be a shame to keep on
hunting like this if there were a good reference lying around.

Thanks!
P


Re: SearchSysCache() tutorial?

From
Paul Ramsey
Date:
On Mon, Dec 11, 2017 at 11:25 AM, Paul Ramsey <pramsey@cleverelephant.ca> wrote:
> Is there anywhere any documentation on SearchSysCache? I find I end up
> on these long spelunking expeditions through the code base for a
> particular problem, find the answer after many hours, then forget
> everything I learned because I don't exercise my knowledge frequently
> enough. A decent reference guide would help a lot. What do the various
> SysCacheIdentifier numbers mean/search, some examples, of usage, etc,
> etc.
>
> I can accept if there are not, but it would be a shame to keep on
> hunting like this if there were a good reference lying around.

My particular hunt today is "for a given table relation, find any
indexes that use the gist access method and are on a single attribute
of type geometry".


Re: SearchSysCache() tutorial?

From
Melvin Davidson
Date:


On Mon, Dec 11, 2017 at 2:26 PM, Paul Ramsey <pramsey@cleverelephant.ca> wrote:
On Mon, Dec 11, 2017 at 11:25 AM, Paul Ramsey <pramsey@cleverelephant.ca> wrote:
> Is there anywhere any documentation on SearchSysCache? I find I end up
> on these long spelunking expeditions through the code base for a
> particular problem, find the answer after many hours, then forget
> everything I learned because I don't exercise my knowledge frequently
> enough. A decent reference guide would help a lot. What do the various
> SysCacheIdentifier numbers mean/search, some examples, of usage, etc,
> etc.
>
> I can accept if there are not, but it would be a shame to keep on
> hunting like this if there were a good reference lying around.

My particular hunt today is "for a given table relation, find any
indexes that use the gist access method and are on a single attribute
of type geometry".


>My particular hunt today is "for a given table relation, find any
>indexes that use the gist access method and are on a single attribute
>of type geometry".


For that information, you are better off querying the system catalogs!


adjust the WHERE clause below to include the attribute you are looking for.

SELECT n.nspname as schema,
       i.relname as table,
       i.indexrelname as index,
       i.idx_scan,
       i.idx_tup_read,
       i.idx_tup_fetch,
       CASE WHEN idx.indisprimary
            THEN 'pkey'
            WHEN idx.indisunique
            THEN 'uidx'
            ELSE 'idx'
            END AS type,
       idx.indisexclusion,
       pg_get_indexdef(idx.indexrelid),
       CASE WHEN idx.indisvalid
            THEN 'valid'
            ELSE 'INVALID'
            END as statusi,
       pg_relation_size(quote_ident(n.nspname)|| '.' || quote_ident(i.relname)) as size_in_bytes,
       pg_size_pretty(pg_relation_size(quote_ident(n.nspname)|| '.' || quote_ident(i.relname))) as size
  FROM pg_stat_all_indexes i
  JOIN pg_class c ON (c.oid = i.relid)
  JOIN pg_namespace n ON (n.oid = c.relnamespace)
  JOIN pg_index idx ON (idx.indexrelid =  i.indexrelid )
 WHERE i.relname = 'your_table'
   AND n.nspname NOT LIKE 'pg_%'
   AND  pg_get_indexdef(idx.indexrelid) LIKE '%gist%'
ORDER BY 1, 2, 3;

--
Melvin Davidson
I reserve the right to fantasize.  Whether or not you
wish to share my fantasy is entirely up to you.

Re: SearchSysCache() tutorial?

From
Paul Ramsey
Date:

On Dec 11, 2017, at 11:37 AM, Melvin Davidson <melvin6925@gmail.com> wrote:



On Mon, Dec 11, 2017 at 2:26 PM, Paul Ramsey <pramsey@cleverelephant.ca> wrote:
On Mon, Dec 11, 2017 at 11:25 AM, Paul Ramsey <pramsey@cleverelephant.ca> wrote:
> Is there anywhere any documentation on SearchSysCache? I find I end up
> on these long spelunking expeditions through the code base for a
> particular problem, find the answer after many hours, then forget
> everything I learned because I don't exercise my knowledge frequently
> enough. A decent reference guide would help a lot. What do the various
> SysCacheIdentifier numbers mean/search, some examples, of usage, etc,
> etc.
>
> I can accept if there are not, but it would be a shame to keep on
> hunting like this if there were a good reference lying around.

My particular hunt today is "for a given table relation, find any
indexes that use the gist access method and are on a single attribute
of type geometry".


>My particular hunt today is "for a given table relation, find any
>indexes that use the gist access method and are on a single attribute
>of type geometry".


For that information, you are better off querying the system catalogs!


adjust the WHERE clause below to include the attribute you are looking for.

SELECT n.nspname as schema,
       i.relname as table,
       i.indexrelname as index,
       i.idx_scan,
       i.idx_tup_read,
       i.idx_tup_fetch,
       CASE WHEN idx.indisprimary
            THEN 'pkey'
            WHEN idx.indisunique
            THEN 'uidx'
            ELSE 'idx'
            END AS type,
       idx.indisexclusion,
       pg_get_indexdef(idx.indexrelid),
       CASE WHEN idx.indisvalid
            THEN 'valid'
            ELSE 'INVALID'
            END as statusi,
       pg_relation_size(quote_ident(n.nspname)|| '.' || quote_ident(i.relname)) as size_in_bytes,
       pg_size_pretty(pg_relation_size(quote_ident(n.nspname)|| '.' || quote_ident(i.relname))) as size
  FROM pg_stat_all_indexes i
  JOIN pg_class c ON (c.oid = i.relid)
  JOIN pg_namespace n ON (n.oid = c.relnamespace)
  JOIN pg_index idx ON (idx.indexrelid =  i.indexrelid )
 WHERE i.relname = 'your_table'
   AND n.nspname NOT LIKE 'pg_%'
   AND  pg_get_indexdef(idx.indexrelid) LIKE '%gist%'
ORDER BY 1, 2, 3;

Thanks. I’m working on doing this at the C level however, so using syscache seems like the right way to go about it. I’d like to avoid doing an SPI thing, if I can, tho I suppose I could always suck it up and just do that.

P

Re: SearchSysCache() tutorial?

From
Melvin Davidson
Date:
There is no guarantee that information will be in syscache at any point in time. It will, however, always be in the postgreSQL catalogs. That is the whole point
of having them, and the SQL language.

On Mon, Dec 11, 2017 at 2:39 PM, Paul Ramsey <pramsey@cleverelephant.ca> wrote:

On Dec 11, 2017, at 11:37 AM, Melvin Davidson <melvin6925@gmail.com> wrote:



On Mon, Dec 11, 2017 at 2:26 PM, Paul Ramsey <pramsey@cleverelephant.ca> wrote:
On Mon, Dec 11, 2017 at 11:25 AM, Paul Ramsey <pramsey@cleverelephant.ca> wrote:
> Is there anywhere any documentation on SearchSysCache? I find I end up
> on these long spelunking expeditions through the code base for a
> particular problem, find the answer after many hours, then forget
> everything I learned because I don't exercise my knowledge frequently
> enough. A decent reference guide would help a lot. What do the various
> SysCacheIdentifier numbers mean/search, some examples, of usage, etc,
> etc.
>
> I can accept if there are not, but it would be a shame to keep on
> hunting like this if there were a good reference lying around.

My particular hunt today is "for a given table relation, find any
indexes that use the gist access method and are on a single attribute
of type geometry".


>My particular hunt today is "for a given table relation, find any
>indexes that use the gist access method and are on a single attribute
>of type geometry".


For that information, you are better off querying the system catalogs!


adjust the WHERE clause below to include the attribute you are looking for.

SELECT n.nspname as schema,
       i.relname as table,
       i.indexrelname as index,
       i.idx_scan,
       i.idx_tup_read,
       i.idx_tup_fetch,
       CASE WHEN idx.indisprimary
            THEN 'pkey'
            WHEN idx.indisunique
            THEN 'uidx'
            ELSE 'idx'
            END AS type,
       idx.indisexclusion,
       pg_get_indexdef(idx.indexrelid),
       CASE WHEN idx.indisvalid
            THEN 'valid'
            ELSE 'INVALID'
            END as statusi,
       pg_relation_size(quote_ident(n.nspname)|| '.' || quote_ident(i.relname)) as size_in_bytes,
       pg_size_pretty(pg_relation_size(quote_ident(n.nspname)|| '.' || quote_ident(i.relname))) as size
  FROM pg_stat_all_indexes i
  JOIN pg_class c ON (c.oid = i.relid)
  JOIN pg_namespace n ON (n.oid = c.relnamespace)
  JOIN pg_index idx ON (idx.indexrelid =  i.indexrelid )
 WHERE i.relname = 'your_table'
   AND n.nspname NOT LIKE 'pg_%'
   AND  pg_get_indexdef(idx.indexrelid) LIKE '%gist%'
ORDER BY 1, 2, 3;

Thanks. I’m working on doing this at the C level however, so using syscache seems like the right way to go about it. I’d like to avoid doing an SPI thing, if I can, tho I suppose I could always suck it up and just do that.

P



--
Melvin Davidson
I reserve the right to fantasize.  Whether or not you
wish to share my fantasy is entirely up to you.

Re: SearchSysCache() tutorial?

From
Andres Freund
Date:
On 2017-12-11 14:42:33 -0500, Melvin Davidson wrote:
> There is no guarantee that information will be in syscache at any point in
> time. It will, however, always be in the postgreSQL catalogs. That is the
> whole point
> of having them, and the SQL language.

This doesn't make much sense. Paul's working on a C extension that's
then usable via SQL. Using the syscaches is the perfectly right thing to
do.

- Andres


Re: SearchSysCache() tutorial?

From
Andres Freund
Date:
On 2017-12-11 11:26:51 -0800, Paul Ramsey wrote:
> On Mon, Dec 11, 2017 at 11:25 AM, Paul Ramsey <pramsey@cleverelephant.ca> wrote:
> > Is there anywhere any documentation on SearchSysCache? I find I end up
> > on these long spelunking expeditions through the code base for a
> > particular problem, find the answer after many hours, then forget
> > everything I learned because I don't exercise my knowledge frequently
> > enough. A decent reference guide would help a lot. What do the various
> > SysCacheIdentifier numbers mean/search, some examples, of usage, etc,
> > etc.
> >
> > I can accept if there are not, but it would be a shame to keep on
> > hunting like this if there were a good reference lying around.
> 
> My particular hunt today is "for a given table relation, find any
> indexes that use the gist access method and are on a single attribute
> of type geometry".

I don't think there's a way to do this with a single syscache, there
won't be an index than can cover all of these. I'd suggest using
RelationGetIndexList(), and then filtering for gist and attribute type
on the returned value.

Greetings,

Andres Freund


Re: SearchSysCache() tutorial?

From
Paul Ramsey
Date:
> On Dec 11, 2017, at 12:40 PM, Andres Freund <andres@anarazel.de> wrote:
>
> On 2017-12-11 11:26:51 -0800, Paul Ramsey wrote:
>> On Mon, Dec 11, 2017 at 11:25 AM, Paul Ramsey <pramsey@cleverelephant.ca> wrote:
>>> Is there anywhere any documentation on SearchSysCache? I find I end up
>>> on these long spelunking expeditions through the code base for a
>>> particular problem, find the answer after many hours, then forget
>>> everything I learned because I don't exercise my knowledge frequently
>>> enough. A decent reference guide would help a lot. What do the various
>>> SysCacheIdentifier numbers mean/search, some examples, of usage, etc,
>>> etc.
>>>
>>> I can accept if there are not, but it would be a shame to keep on
>>> hunting like this if there were a good reference lying around.
>>
>> My particular hunt today is "for a given table relation, find any
>> indexes that use the gist access method and are on a single attribute
>> of type geometry".
>
> I don't think there's a way to do this with a single syscache, there
> won't be an index than can cover all of these. I'd suggest using
> RelationGetIndexList(), and then filtering for gist and attribute type
> on the returned value.

Thanks, A, I seem to be on the right path then. Loop through all indexes in RelationGetIndexList(), for those that have
arelam == GIST_AM_OID, loop through the associated columns (looks like I actually have to use a SysScanDesc on
AttributeRelationId)and look for atts that have the type I’m interested in, and if I find one, yay, we have a winner. 
Thanks,
P



Re: SearchSysCache() tutorial?

From
Andres Freund
Date:
On 2017-12-11 12:48:27 -0800, Paul Ramsey wrote:
> 
> > On Dec 11, 2017, at 12:40 PM, Andres Freund <andres@anarazel.de> wrote:
> > 
> > On 2017-12-11 11:26:51 -0800, Paul Ramsey wrote:
> >> On Mon, Dec 11, 2017 at 11:25 AM, Paul Ramsey <pramsey@cleverelephant.ca> wrote:
> >>> Is there anywhere any documentation on SearchSysCache? I find I end up
> >>> on these long spelunking expeditions through the code base for a
> >>> particular problem, find the answer after many hours, then forget
> >>> everything I learned because I don't exercise my knowledge frequently
> >>> enough. A decent reference guide would help a lot. What do the various
> >>> SysCacheIdentifier numbers mean/search, some examples, of usage, etc,
> >>> etc.
> >>> 
> >>> I can accept if there are not, but it would be a shame to keep on
> >>> hunting like this if there were a good reference lying around.
> >> 
> >> My particular hunt today is "for a given table relation, find any
> >> indexes that use the gist access method and are on a single attribute
> >> of type geometry".
> > 
> > I don't think there's a way to do this with a single syscache, there
> > won't be an index than can cover all of these. I'd suggest using
> > RelationGetIndexList(), and then filtering for gist and attribute type
> > on the returned value.
> 
> Thanks, A, I seem to be on the right path then. Loop through all
> indexes in RelationGetIndexList(), for those that have a relam ==
> GIST_AM_OID, loop through the associated columns (looks like I
> actually have to use a SysScanDesc on AttributeRelationId) and look
> for atts that have the type I’m interested in, and if I find one, yay,
> we have a winner.

Depending on what you do, it might not be a bad idea to do an
index_open() on the oid. The returned Relation entry will already have
looked up the relevant types, and you possibly want to lock anyway...

Greetings,

Andres Freund