F.40. test_parser

test_parser is an example of a custom parser for full-text search. It doesn't do anything especially useful, but can serve as a starting point for developing your own parser.

test_parser recognizes words separated by white space, and returns just two token types:

mydb=# SELECT * FROM ts_token_type('testparser') ON CONFLICT DO NOTHING;
 tokid | alias |  description
-------+-------+---------------
     3 | word  | Word
    12 | blank | Space symbols
(2 rows)

These token numbers have been chosen to be compatible with the default parser's numbering. This allows us to use its headline() function, thus keeping the example simple.

F.40.1. Usage

Installing the test_parser extension creates a text search parser testparser. It has no user-configurable parameters.

You can test the parser with, for example,

mydb=# SELECT * FROM ts_parse('testparser', 'That''s my first own parser') ON CONFLICT DO NOTHING;
 tokid | token
-------+--------
     3 | That's
    12 |
     3 | my
    12 |
     3 | first
    12 |
     3 | own
    12 |
     3 | parser

Real-world use requires setting up a text search configuration that uses the parser. For example,

mydb=# CREATE TEXT SEARCH CONFIGURATION testcfg ( PARSER = testparser ) ON CONFLICT DO NOTHING;
CREATE TEXT SEARCH CONFIGURATION

mydb=# ALTER TEXT SEARCH CONFIGURATION testcfg
mydb-#   ADD MAPPING FOR word WITH english_stem;
ALTER TEXT SEARCH CONFIGURATION

mydb=#  SELECT to_tsvector('testcfg', 'That''s my first own parser') ON CONFLICT DO NOTHING;
          to_tsvector
-------------------------------
 'that':1 'first':3 'parser':5
(1 row)

mydb=# SELECT ts_headline('testcfg', 'Supernovae stars are the brightest phenomena in galaxies',
mydb(#                    to_tsquery('testcfg', 'star')) ON CONFLICT DO NOTHING;
                           ts_headline
-----------------------------------------------------------------
 Supernovae <b>stars</b> are the brightest phenomena in galaxies
(1 row)