Thread: BUG #16647: ts_lexize with dict_voikko no longer works as of PostgreSQL 13.0

The following bug has been logged on the website:

Bug reference:      16647
Logged by:          Matti Linnanvuori
Email address:      matti.linnanvuori@portalify.com
PostgreSQL version: 13.0
Operating system:   Red Hat Enterprise Linux 8.2
Description:

ts_lexize with dict_voikko no longer works as of PostgreSQL 13.0 but it
worked in PostgreSQL 12.4.
dnf install postgresql13
dnf install postgresql13-server
dnf install postgresql13-devel
dnf install libvoikko
dnf install malaga-suomi-voikko
wget
https://www.puimula.org/htp/testing/voikko-snapshot-v5/dict-morpho.zip
unzip dict-morpho.zip
mkdir /etc/voikko
mv 5 /etc/voikko
git clone https://github.com/MattiL/dict_voikko.git
cp -r dict_voikko /usr/pgsql-13/share/contrib
cd /usr/pgsql-13/share/contrib/dict_voikko
PATH="/usr/pgsql-13/bin:$PATH" make clean install USE_PGXS=1
PGSETUP_INITDB_OPTIONS=--locale=fi_FI.utf8
/usr/pgsql-13/bin/postgresql-13-setup initdb
createdb -E UTF8 test
CREATE EXTENSION dict_voikko;
CREATE TEXT SEARCH DICTIONARY voikko_stopwords (TEMPLATE = voikko_template,
StopWords = finnish);
CREATE TEXT SEARCH CONFIGURATION voikko (COPY = pg_catalog.finnish);
ALTER TEXT SEARCH CONFIGURATION voikko ALTER MAPPING FOR asciiword,
asciihword, hword_asciipart, word, hword, hword_part WITH voikko_stopwords,
finnish_stem;' tasks";
test=# select ts_lexize('voikko', 'kerrostalollekohan');
 ts_lexize 
-----------
 
(1 row)

PostgreSQL 12.4:
test=# select ts_lexize('voikko', 'kerrostalollekohan');
   ts_lexize   
---------------
 {kerros,talo}
(1 row)

On 30/09/2020 13:19, PG Bug reporting form wrote:
> The following bug has been logged on the website:
> 
> Bug reference:      16647
> Logged by:          Matti Linnanvuori
> Email address:      matti.linnanvuori@portalify.com
> PostgreSQL version: 13.0
> Operating system:   Red Hat Enterprise Linux 8.2
> Description:
> 
> ts_lexize with dict_voikko no longer works as of PostgreSQL 13.0 but it
> worked in PostgreSQL 12.4.
> dnf install postgresql13
> dnf install postgresql13-server
> dnf install postgresql13-devel
> dnf install libvoikko
> dnf install malaga-suomi-voikko
> wget
> https://www.puimula.org/htp/testing/voikko-snapshot-v5/dict-morpho.zip
> unzip dict-morpho.zip
> mkdir /etc/voikko
> mv 5 /etc/voikko
> git clone https://github.com/MattiL/dict_voikko.git
> cp -r dict_voikko /usr/pgsql-13/share/contrib
> cd /usr/pgsql-13/share/contrib/dict_voikko
> PATH="/usr/pgsql-13/bin:$PATH" make clean install USE_PGXS=1
> PGSETUP_INITDB_OPTIONS=--locale=fi_FI.utf8
> /usr/pgsql-13/bin/postgresql-13-setup initdb
> createdb -E UTF8 test
> CREATE EXTENSION dict_voikko;
> CREATE TEXT SEARCH DICTIONARY voikko_stopwords (TEMPLATE = voikko_template,
> StopWords = finnish);
> CREATE TEXT SEARCH CONFIGURATION voikko (COPY = pg_catalog.finnish);
> ALTER TEXT SEARCH CONFIGURATION voikko ALTER MAPPING FOR asciiword,
> asciihword, hword_asciipart, word, hword, hword_part WITH voikko_stopwords,
> finnish_stem;' tasks";
> test=# select ts_lexize('voikko', 'kerrostalollekohan');
>   ts_lexize
> -----------
>   
> (1 row)

Is that a NULL? Are you getting a NULL result for all inputs?

Did you install voikko and dict-morpho the same way on both PostgreSQL 
versions?

It works for me. I installed voikko from the Debian packages with:

sudo apt install libvoikko-dev voikko-fi

And then:

git clone git@github.com:MattiL/dict_voikko.git
cd dict_voikko
PATH=~/pgsql.13stable/bin:$PATH make USE_PGXS=1 install

And in psql:

postgres=# select version();
                                         version 

----------------------------------------------------------------------------------------
  PostgreSQL 13.0 on x86_64-pc-linux-gnu, compiled by gcc (Debian 
8.3.0-6) 8.3.0, 64-bit
(1 row)

postgres=# CREATE EXTENSION dict_voikko;
CREATE EXTENSION
postgres=# CREATE TEXT SEARCH DICTIONARY voikko_stopwords (TEMPLATE = 
voikko_template,
StopWords = finnish);
CREATE TEXT SEARCH DICTIONARY
postgres=# CREATE TEXT SEARCH CONFIGURATION voikko (COPY = finnish);
CREATE TEXT SEARCH CONFIGURATION
postgres=# ALTER TEXT SEARCH CONFIGURATION voikko ALTER MAPPING FOR 
asciiword,
asciihword, hword_asciipart, word, hword, hword_part WITH voikko_stopwords,
finnish_stem;
ALTER TEXT SEARCH CONFIGURATION
postgres=# select ts_lexize('voikko', 'kerrostalollekohan');
    ts_lexize
---------------
  {kerros,talo}
(1 row)

- Heikki

I forgot to install libvoikko.so dynamic library in PostgreSQL 13 lib directory. After I installed it, ts_lexize started working.