Re: Optimize external TOAST storage - Mailing list pgsql-hackers

From Dilip Kumar
Subject Re: Optimize external TOAST storage
Date
Msg-id CAFiTN-vOGZCsSHKQMh4OB+F--Tdb78ibqDgK6eY7WaxbFie0FA@mail.gmail.com
Whole thread Raw
In response to Re: Optimize external TOAST storage  (Michael Paquier <michael@paquier.xyz>)
List pgsql-hackers
On Wed, Mar 23, 2022 at 9:58 AM Michael Paquier <michael@paquier.xyz> wrote:
>
> On Tue, Mar 22, 2022 at 02:42:53PM -0700, Nathan Bossart wrote:
> > On Tue, Mar 22, 2022 at 04:34:05PM -0400, Robert Haas wrote:
> >> Then, too, I'm not very confident about the usefulness of EXTENDED,
> >> EXTERNAL, and MAIN. I think it's useful to be able to categorically
> >> disable compression (as EXTERNAL does), but you can't categorically
> >> disable out-of-line storage because the value can be bigger than the
> >> page, so MAIN vs. EXTENDED is just changing the threshold for the use
> >> of out-of-line storage. However, it does so in a way that IMHO is not
> >> particularly intuitive, which goes back to my earlier point about the
> >> algorithm seeming wacky, and it's in any case unclear why we should
> >> offer exactly two possible thresholds and not any of the intermediate
> >> values.
> >
> > I agree with all of this.  Adding configurability for each constant might
> > help folks in the short term, but using these knobs probably requires quite
> > a bit of expertise in Postgres internals as well as a good understanding of
> > your data.  I think we ought to revist TOAST configurability from a user
> > perspective.  IOW what can be chosen automatically, and how do we enable
> > users to effectively configure the settings that cannot be chosen
> > automatically?  IMO this is a worthwhile conversation to have as long as it
> > doesn't stray too far into the "let's rewrite TOAST" realm.  I think there
> > is always going to be some level of complexity with stuff like TOAST, but
> > there are probably all sorts of ways to simplify/improve it also.
>
> I agree with this feeling.  TOAST has already too many configuration
> parameters that have their own way of behaving slightling differently.
> If we could reduce this number rather than increasing it, the better.
> I would be also curious to see how much those parameters become
> relevant with more compression options possible with toast values.

Agree with you, it doesn't make much sense here to add an extra user
visible configuration parameter, where it is not really clear for
users how to really configure it to get the best out of it.

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com



pgsql-hackers by date:

Previous
From: Maxim Orlov
Date:
Subject: Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)
Next
From: Tatsuo Ishii
Date:
Subject: Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors