Thread: tsearch2 problems / limitations

Hi,

I've successfully added tsearch2 to an existing database and the speedup
of searches is brilliant. I'm now trying to extend this to other parts
of our system.

One of the tables holds reasonable amounts of text, some fields hold up
to 2Mb. When I try and run

UPDATE table SET idxfti=to_tsvector('default', field);

it runs for a while then aborts with the following message

ERROR:  value is too big

Does anyone know what the problem might be here? Does tsearch2 have
configurable (or compilable) settings that could get around this?

Many Thanks,
James

On Tue, 22 Feb 2005, James Croft wrote:

> Hi,
>
> I've successfully added tsearch2 to an existing database and the speedup of
> searches is brilliant. I'm now trying to extend this to other parts of our
> system.
>
> One of the tables holds reasonable amounts of text, some fields hold up to
> 2Mb. When I try and run
>
> UPDATE table SET idxfti=to_tsvector('default', field);
>
> it runs for a while then aborts with the following message
>
> ERROR:  value is too big
>
> Does anyone know what the problem might be here? Does tsearch2 have
> configurable (or compilable) settings that could get around this?

this is tsearch2 limitation on length of ts_vector about 1mb.
you may stop using postional information for this vector.

>
> Many Thanks,
> James
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
>

     Regards,
         Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83

On Tue, Feb 22, 2005 at 01:29:45PM +0000, James Croft wrote:

> One of the tables holds reasonable amounts of text, some fields hold up
> to 2Mb. When I try and run
>
> UPDATE table SET idxfti=to_tsvector('default', field);
>
> it runs for a while then aborts with the following message
>
> ERROR:  value is too big

README.tsearch2 says that the maximum size of a tsvector is 2^20
bytes (1M).  You can see that limit enforced in the makevalue()
function in tsvector.c (similarly in pushquery() in query.c):

    if (cur - str > MAXSTRPOS)
        ereport(ERROR,
                (errcode(ERRCODE_SYNTAX_ERROR),
                 errmsg("value is too big")));

MAXSTRPOS is defined in tsvector.h:

  #define MAXSTRPOS ( 1<<20 )

Maybe Oleg will reply and say whether it's safe to change that
or not.

--
Michael Fuhr
http://www.fuhr.org/~mfuhr/