Re: [GENERAL] FTS query, statistics and planner estimations… - Mailing list pgsql-general

From Francisco Olarte
Subject Re: [GENERAL] FTS query, statistics and planner estimations…
Date
Msg-id CA+bJJbxA2wRBNnm5ZhMNtKgf8VWhwY0-xE8FVvZ5AXpHuYbaRw@mail.gmail.com
Whole thread Raw
In response to FTS query, statistics and planner estimations…  (Pierre Ducroquet <pierre.ducroquet@people-doc.com>)
Responses Re: [GENERAL] Re: [GENERAL] FTS query, statistics and planner estimations…
Re: FTS query, statistics and planner estimations…
List pgsql-general
Pierre:

On Wed, Nov 9, 2016 at 10:22 AM, Pierre Ducroquet
<pierre.ducroquet@people-doc.com> wrote:
> The query does a few joins «after» running a FTS query on a main table.
> The FTS query returns a few thousand rows, but the estimations are wrong,
> leading the optimizer to terrible plans compared to what should happen, and
> thus creates a far higher execution time.
....
> but the issue remain the same. The table contains about 295,000 documents, and
....
>  Request                          | Estimated rows | Real rows
> ----------------------------------+----------------+-----------
> 'word1'                           | 38050          | 37500
> 'word1 word2'                     | 4680           | 32000
> 'word1 word2 word3'               | 270            | 12300
> 'word1 word2 word3 word4'         | 10             | 9930
> 'word1 word2 word3 word4 word5'   | 1              | 9930
>
> You can see that with more words in query, the estimation falls far behind
> reality.

I'm not really familiar with FTS but, doing a few division of
estimations and rows it seems it estimates as uncorrelated words, and
you real rows clearly indicate some of them are clearly correlated (
like w1/w2 and w4/s5, and partially w3/w45 ) and very common.

> Is that a known limitation of the FTS indexing ? Am I missing something
> obvious, or a poor configuration ?

Someone more familiar with it needed for that, but what I've found
several times is FTS does not mix too well with relational queries at
the optimizer level ( as FTS terms can have very diverse degrees of
correlation, which is very difficult to store in the statistics a
relational optimizer normally uses ).

Francisco Olarte.


pgsql-general by date:

Previous
From: Pierre Ducroquet
Date:
Subject: FTS query, statistics and planner estimations…
Next
From: Pavel Stehule
Date:
Subject: Re: [GENERAL] Re: [GENERAL] FTS query, statistics and planner estimations…