Re: Early WIP/PoC for inlining CTEs - Mailing list pgsql-hackers

From Mike Rylander
Subject Re: Early WIP/PoC for inlining CTEs
Date
Msg-id CAO8ar==+dFPzT36WztJbBDQ-gEfZSHKZLrGP8t96aD=idyDOkw@mail.gmail.com
Whole thread Raw
In response to Re: Early WIP/PoC for inlining CTEs  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
On Fri, Jan 11, 2019 at 2:10 PM Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Fri, Jan 11, 2019 at 2:04 PM Andres Freund <andres@anarazel.de> wrote:
> > > Maybe we could consider a more extensible syntax that is attached to
> > > the contained SELECT rather than the containing WITH.  Then CTEs would
> > > be less special; there'd be a place to put hints controlling top-level
> > > queries, subselects, views etc too (perhaps eventually join hints,
> > > parallelism hints etc, but "materialize this" would be just another
> > > one of those things).  That'd be all-in.
> >
> > I think you have some purity arguments here, but the likelihood of us
> > developing a full-blown solution is not that high, and the lack of
> > inlinable CTEs is *really* hurting us. As long as the design doesn't
> > block a full solution, if we go there, I think it's a very acceptable
> > blemish in comparison to the benefits we'd get.
>
> Also, it seems to me that this is properly a property of the
> individual WITH clause, not the query as a whole.
>
> I mean I suppose we could do
>
> WITH or_with_out_you OPTIONS (materialized false) AS (SELECT 'mariah
> carey') SELECT ...
>
> That'd allow for extensibility, have the write scope, and look like
> what we do elsewhere.  It looks a little less elegant than
>
> WITH cte_name [[NOT] MATERIALIZED] AS (query) main_query...
>
> ...but maybe elegance for extensibility is a good trade.
>

Here, have $0.02 from the peanut gallery...

I mildly prefer the latter, elegant spelling, but if CTE inlining does
become a thing then I would /really/ want some way, any way, of
telling Postgres that I want it to materialize a particular CTE.

I use that currently-documented property of CTEs to structure large,
complicated OLAP queries on a regular basis, for performance.
Sometimes, such as when you have dozens of tables in a complex join
tree, breaking the query into logically related chunks (which I know
about, but the planner does not) via CTE is the only way to give the
planner a fighting chance of finding a good plan.  Otherwise you get
stuck in the GEQO ghetto, or planning time is some non-trivial
multiple of execution time.

Thanks,

--
Mike Rylander
 | Executive Director
 | Equinox Open Library Initiative
 | phone:  1-877-OPEN-ILS (673-6457)
 | email:  miker@equinoxinitiative.org
 | web:  http://equinoxinitiative.org


pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: Prevent extension creation in temporary schemas
Next
From: Mitar
Date:
Subject: Re: Feature: temporary materialized views