Re: MCV lists for highly skewed distributions - Mailing list pgsql-hackers

From Dean Rasheed
Subject Re: MCV lists for highly skewed distributions
Date
Msg-id CAEZATCXUE1zB1tHFx9YdztgDvovbANjK_JinM9X-XJ62AkYFqw@mail.gmail.com
Whole thread Raw
In response to Re: MCV lists for highly skewed distributions  (Tomas Vondra <tomas.vondra@2ndquadrant.com>)
Responses Re: MCV lists for highly skewed distributions  (Tomas Vondra <tomas.vondra@2ndquadrant.com>)
List pgsql-hackers
On 16 March 2018 at 15:26, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
> Actually, one question - when deciding whether to keep the item in the
> MCV list, analyze_mcv_list only compares it's frequency with an average
> of the rest. But as we're removing items from the MCV list, the average
> frequency of the non-MCV items is growing (we're removing items with
> higher and higher frequencies). That means the estimates for the least
> common items will get higher and higher - essentially overestimates. So,
> couldn't/shouldn't analyze_mcv_list consider this too?
>

Yes, that's the intention. At the start, sumcount is set to the count
of all but the last (least common) MCV item, so it can estimate the
frequency of the non-MCV items if the last MCV item were to be
removed. Then each time through the loop, sumcount is decreased by the
removed item's count, increasing the estimated frequency of the
non-MCV items accordingly.

Regards,
Dean


pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: strange failure in plpgsql_control tests (on fulmar, ICC 14.0.3)
Next
From: Tom Lane
Date:
Subject: Re: strange failure in plpgsql_control tests (on fulmar, ICC 14.0.3)