Re: string_to_array eats too much memory? - Mailing list pgsql-hackers

From Teodor Sigaev
Subject Re: string_to_array eats too much memory?
Date
Msg-id 45519AEE.8040704@sigaev.ru
Whole thread Raw
In response to string_to_array eats too much memory?  (Tatsuo Ishii <ishii@sraoss.co.jp>)
Responses Re: string_to_array eats too much memory?
List pgsql-hackers
> I'm playing with GIN to make a full text search system. GIN comes with
> built-in TEXT[] support and I use string_to_array() to make a
> TEXT[]. Problem is, if there's large number of array elemets,
> string_to_array() consumes too much memory. For example, to make ~70k
> array elements, string_to_array seems to eat several Gig bytes of
> memory. ~70k array elements means there are same number of words in a
> document which is not too big in a large text IMO.

Do you mean 70k unique lexemes? Ugh.
Why do not you use tsearch framework?

-- 
Teodor Sigaev                                   E-mail: teodor@sigaev.ru
  WWW: http://www.sigaev.ru/
 


pgsql-hackers by date:

Previous
From: Teodor Sigaev
Date:
Subject: Re: Index ignored with "is not distinct from", 8.2 beta2
Next
From: Teodor Sigaev
Date:
Subject: Re: Tsearch Index Size and GiST vs. GIN