tsvector field length limitation - Mailing list pgsql-general

From Jonathan Marks
Subject tsvector field length limitation
Date
Msg-id 5A4EDC33-60F6-4998-AA8D-9B2ED615D87C@gmail.com
Whole thread Raw
Responses Re: tsvector field length limitation  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-general
Hi folks —

We utilize Postgres’ full text search system pretty heavily in our team’s operations and often index tens of millions
ofrecords with varying lengths of text. In most cases, the text we need to index is pretty short (no more than.
hundredsof words) but in rare cases a single record is very very long (high hundreds of thousands of words or longer).
Withthose records, we run into the max tsvector length requirement "The length of a tsvector (lexemes + positions) must
beless than 1 megabyte” 

I understand the performance implications of having very long tsvectors (our gin index updates are pretty terrible in
somecases) but would really appreciate it if the max tsvector length were larger (even 5MB would make a huge
difference)or if that error were a stern warning rather than a hard error. 

Is there any way to disable or increase that limit in Postgres 10.3? Perhaps in a future version?

Thank you!
Jonathan

pgsql-general by date:

Previous
From: Łukasz Jarych
Date:
Subject: Re: Using DSN Connection and knowing windows username
Next
From: Thomas Kellerer
Date:
Subject: Re: Plan output: actual execution time not considering loops?