Thread: Top features in 9.6?
So, let's make a list. Mine: * parallel query (seqscan & aggregation) * bloom filter supporting indexes The generic progress reporting framework is tempting, but seems hard to explain as a marquee feature. What else have we got? -- -- Josh Berkus Red Hat OSAS (any opinions are my own)
On 11 April 2016 at 23:02, Josh berkus <josh@agliodbs.com> wrote:
--
So, let's make a list.
Mine:
* parallel query (seqscan & aggregation)
* bloom filter supporting indexes
The generic progress reporting framework is tempting, but seems hard to
explain as a marquee feature. What else have we got?
Could I suggest we do the DRAFT release notes now, then discuss what gets highlighted?
If the RMT's schedule is good, we have about 1 month to prepare it.
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Tue, Apr 12, 2016 at 1:02 AM, Josh berkus <josh@agliodbs.com> wrote:
Create AM+Generic WAL - these are C-API, but we can work on various AM as extensions, expect some to 9.6.
So, let's make a list.
Mine:
* parallel query (seqscan & aggregation)
* bloom filter supporting indexes
Create AM+Generic WAL - these are C-API, but we can work on various AM as extensions, expect some to 9.6.
The generic progress reporting framework is tempting, but seems hard to
explain as a marquee feature. What else have we got?
--
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)
--
Sent via pgsql-advocacy mailing list (pgsql-advocacy@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-advocacy
On 04/11/2016 03:09 PM, Simon Riggs wrote: > Could I suggest we do the DRAFT release notes now, then discuss what > gets highlighted? > > If the RMT's schedule is good, we have about 1 month to prepare it. Hey, I'm trying to get some parallel execution going here. -- -- Josh Berkus Red Hat OSAS (any opinions are my own)
* Josh berkus (josh@agliodbs.com) wrote: > So, let's make a list. > > Mine: > > * parallel query (seqscan & aggregation) > * bloom filter supporting indexes > > The generic progress reporting framework is tempting, but seems hard to > explain as a marquee feature. What else have we got? I like the progress reporting, we need to do that with a lot more stuff. postgres_fdw join pushdown is pretty huge. Relation extension speed improvements are pretty huge too. snapshot_too_old is quite important in some environments. Degree-based geometry funcs Phrase full text search jsonb_insert Index-only scans with partial indexes was improved synch_commit = 'remote_apply' ... Thanks! Stephen
Attachment
On 2016-04-11 3:09 PM, Simon Riggs wrote: > On 11 April 2016 at 23:02, Josh berkus <josh@agliodbs.com> wrote: > So, let's make a list. > > Mine: > > * parallel query (seqscan & aggregation) > * bloom filter supporting indexes > > The generic progress reporting framework is tempting, but seems hard to > explain as a marquee feature. What else have we got? > > Could I suggest we do the DRAFT release notes now, then discuss what gets > highlighted? > > If the RMT's schedule is good, we have about 1 month to prepare it. If we're doing this now, along with seemingly the 9.6 feature freeze having occurred, does that mean there is an effort to move the Postgres major release dates back to September or so? September-ish was held consistently for 9.0-9.3, with 9.4 only going to December due to having to re-work jsonb. Or is there a stronger precedent for time between releases, eg aiming for a 9.6 next December or January since that's after a year passed? I think aiming for September again would be preferable, especially with a feature cutoff around now. -- Darren Duncan
* Darren Duncan (darren@darrenduncan.net) wrote: > On 2016-04-11 3:09 PM, Simon Riggs wrote: > >On 11 April 2016 at 23:02, Josh berkus <josh@agliodbs.com> wrote: > > So, let's make a list. > > > > Mine: > > > > * parallel query (seqscan & aggregation) > > * bloom filter supporting indexes > > > > The generic progress reporting framework is tempting, but seems hard to > > explain as a marquee feature. What else have we got? > > > >Could I suggest we do the DRAFT release notes now, then discuss what gets > >highlighted? > > > >If the RMT's schedule is good, we have about 1 month to prepare it. > > If we're doing this now, along with seemingly the 9.6 feature freeze > having occurred, does that mean there is an effort to move the > Postgres major release dates back to September or so? September-ish > was held consistently for 9.0-9.3, with 9.4 only going to December > due to having to re-work jsonb. Yes, we're trying to get back to the September/October release timing for major versions. Of course, the RMT seems more on schedule to release in July, but, hey, it's far easier to hold off releasing something that's ready than to try and push out something that isn't, so more power to them. Thanks! Stephen
Attachment
On 2016-04-11 7:49 PM, Stephen Frost wrote: > * Darren Duncan (darren@darrenduncan.net) wrote: >> On 2016-04-11 3:09 PM, Simon Riggs wrote: >>> On 11 April 2016 at 23:02, Josh berkus <josh@agliodbs.com> wrote: >>> So, let's make a list. >>> >>> Mine: >>> >>> * parallel query (seqscan & aggregation) >>> * bloom filter supporting indexes >>> >>> The generic progress reporting framework is tempting, but seems hard to >>> explain as a marquee feature. What else have we got? >>> >>> Could I suggest we do the DRAFT release notes now, then discuss what gets >>> highlighted? >>> >>> If the RMT's schedule is good, we have about 1 month to prepare it. >> >> If we're doing this now, along with seemingly the 9.6 feature freeze >> having occurred, does that mean there is an effort to move the >> Postgres major release dates back to September or so? September-ish >> was held consistently for 9.0-9.3, with 9.4 only going to December >> due to having to re-work jsonb. > > Yes, we're trying to get back to the September/October release timing > for major versions. > > Of course, the RMT seems more on schedule to release in July, but, hey, > it's far easier to hold off releasing something that's ready than to try > and push out something that isn't, so more power to them. Well hey, if 9.6 can possibly be released in July, or any necessary delays being from that point, that would be great news! The relatively short time frame since 9.5 could even be considered a feature itself. -- Darren Duncan
On 2016/04/12 11:46, Stephen Frost wrote: > * Josh berkus (josh@agliodbs.com) wrote: >> So, let's make a list. >> >> Mine: >> >> * parallel query (seqscan & aggregation) >> * bloom filter supporting indexes >> >> The generic progress reporting framework is tempting, but seems hard to >> explain as a marquee feature. What else have we got? > > I like the progress reporting, we need to do that with a lot more stuff. > > postgres_fdw join pushdown is pretty huge. > > Relation extension speed improvements are pretty huge too. > > snapshot_too_old is quite important in some environments. > > Degree-based geometry funcs > Phrase full text search > jsonb_insert > Index-only scans with partial indexes was improved > synch_commit = 'remote_apply' I think "No more full-table vacuums [1]" sounds like it could be mentioned more prominently. People are quite excited about it [2]. Thanks, Amit [1] http://rhaas.blogspot.jp/2016/03/no-more-full-table-vacuums.html [2] https://news.ycombinator.com/item?id=11311208
On Tue, Apr 12, 2016 at 8:06 AM, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
On 2016/04/12 11:46, Stephen Frost wrote:
> * Josh berkus (josh@agliodbs.com) wrote:
>> So, let's make a list.
>>
>> Mine:
>>
>> * parallel query (seqscan & aggregation)
>> * bloom filter supporting indexes
>>
>> The generic progress reporting framework is tempting, but seems hard to
>> explain as a marquee feature. What else have we got?
>
> I like the progress reporting, we need to do that with a lot more stuff.
>
> postgres_fdw join pushdown is pretty huge.
>
> Relation extension speed improvements are pretty huge too.
>
> snapshot_too_old is quite important in some environments.
>
> Degree-based geometry funcs
> Phrase full text search
> jsonb_insert
> Index-only scans with partial indexes was improved
> synch_commit = 'remote_apply'
I think "No more full-table vacuums [1]" sounds like it could be mentioned
more prominently. People are quite excited about it [2].
Yes!
I think that may turn out to be one of those "hidden gems" of this release. As in being the one that nobody talks about now, but then a few years down the road it's the one that everybody talks about. But it's somewhat hard to explain to people who (1) don't know how the system really works (though that would count for things like snapshot too old as well) or (2) actually have run into the current problem (why hey, that's also the same with snapshot too old)
On Tue, Apr 12, 2016 at 3:21 AM, Magnus Hagander <magnus@hagander.net> wrote: > I think that may turn out to be one of those "hidden gems" of this release. > As in being the one that nobody talks about now, but then a few years down > the road it's the one that everybody talks about. But it's somewhat hard to > explain to people who (1) don't know how the system really works (though > that would count for things like snapshot too old as well) or (2) actually > have run into the current problem (why hey, that's also the same with > snapshot too old) Agreed. Unfortunately, for many people, the first time they really become aware of autovacuum is when all of their tables hit the freeze threshold for the first time. And this doesn't help with that. You still have to scan everything after 200 million transactions; it's just that you no longer have to do it again every 200 million transactions after that. I still think it's a great feature, though. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 12 Apr 2016, at 13:37, Robert Haas <robertmhaas@gmail.com> wrote: > On Tue, Apr 12, 2016 at 3:21 AM, Magnus Hagander <magnus@hagander.net> wrote: >> I think that may turn out to be one of those "hidden gems" of this release. >> As in being the one that nobody talks about now, but then a few years down >> the road it's the one that everybody talks about. But it's somewhat hard to >> explain to people who (1) don't know how the system really works (though >> that would count for things like snapshot too old as well) or (2) actually >> have run into the current problem (why hey, that's also the same with >> snapshot too old) > > Agreed. Unfortunately, for many people, the first time they really > become aware of autovacuum is when all of their tables hit the freeze > threshold for the first time. And this doesn't help with that. You > still have to scan everything after 200 million transactions; it's > just that you no longer have to do it again every 200 million > transactions after that. I still think it's a great feature, though. Er... we don't provide a warning ahead of time in the logs or something? + Justin -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi
On Tue, Apr 12, 2016 at 8:59 AM, Justin Clift <justin@postgresql.org> wrote: > On 12 Apr 2016, at 13:37, Robert Haas <robertmhaas@gmail.com> wrote: >> On Tue, Apr 12, 2016 at 3:21 AM, Magnus Hagander <magnus@hagander.net> wrote: >>> I think that may turn out to be one of those "hidden gems" of this release. >>> As in being the one that nobody talks about now, but then a few years down >>> the road it's the one that everybody talks about. But it's somewhat hard to >>> explain to people who (1) don't know how the system really works (though >>> that would count for things like snapshot too old as well) or (2) actually >>> have run into the current problem (why hey, that's also the same with >>> snapshot too old) >> >> Agreed. Unfortunately, for many people, the first time they really >> become aware of autovacuum is when all of their tables hit the freeze >> threshold for the first time. And this doesn't help with that. You >> still have to scan everything after 200 million transactions; it's >> just that you no longer have to do it again every 200 million >> transactions after that. I still think it's a great feature, though. > > Er... we don't provide a warning ahead of time in the logs or something? No. That would be a little strange, honestly. I have to assume that many wraparound vacuums go totally unnoticed; how would you distinguish the ones that are likely to annoy somebody from the other ones? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 2016-04-12 10:54 AM, Robert Haas wrote: > On Tue, Apr 12, 2016 at 8:59 AM, Justin Clift <justin@postgresql.org> wrote: >> On 12 Apr 2016, at 13:37, Robert Haas <robertmhaas@gmail.com> wrote: >>> On Tue, Apr 12, 2016 at 3:21 AM, Magnus Hagander <magnus@hagander.net> wrote: >>>> I think that may turn out to be one of those "hidden gems" of this release. >>>> As in being the one that nobody talks about now, but then a few years down >>>> the road it's the one that everybody talks about. But it's somewhat hard to >>>> explain to people who (1) don't know how the system really works (though >>>> that would count for things like snapshot too old as well) or (2) actually >>>> have run into the current problem (why hey, that's also the same with >>>> snapshot too old) >>> >>> Agreed. Unfortunately, for many people, the first time they really >>> become aware of autovacuum is when all of their tables hit the freeze >>> threshold for the first time. And this doesn't help with that. You >>> still have to scan everything after 200 million transactions; it's >>> just that you no longer have to do it again every 200 million >>> transactions after that. I still think it's a great feature, though. >> >> Er... we don't provide a warning ahead of time in the logs or something? > > No. That would be a little strange, honestly. I have to assume that > many wraparound vacuums go totally unnoticed; how would you > distinguish the ones that are likely to annoy somebody from the other > ones? I suggest providing a config option for those warnings, if there isn't one already. Global plus override per table etc. The option says we only log the warnings if the wraparound vacuum is likely to take more than a certain amount of time, and pick something reasonable for that default time. Or to generalize, have a config saying what to do if a wraparound vacuum is coming up soon, including what amount of estimated time may be considered inconvenient. -- Darren Duncan
On Wed, Apr 13, 2016 at 7:13 AM, Darren Duncan <darren@darrenduncan.net> wrote: > On 2016-04-12 10:54 AM, Robert Haas wrote: >> >> On Tue, Apr 12, 2016 at 8:59 AM, Justin Clift <justin@postgresql.org> >> wrote: >>> >>> On 12 Apr 2016, at 13:37, Robert Haas <robertmhaas@gmail.com> wrote: >>>> >>>> On Tue, Apr 12, 2016 at 3:21 AM, Magnus Hagander <magnus@hagander.net> >>>> wrote: >>>>> >>>>> I think that may turn out to be one of those "hidden gems" of this >>>>> release. >>>>> As in being the one that nobody talks about now, but then a few years >>>>> down >>>>> the road it's the one that everybody talks about. But it's somewhat >>>>> hard to >>>>> explain to people who (1) don't know how the system really works >>>>> (though >>>>> that would count for things like snapshot too old as well) or (2) >>>>> actually >>>>> have run into the current problem (why hey, that's also the same with >>>>> snapshot too old) >>>> >>>> >>>> Agreed. Unfortunately, for many people, the first time they really >>>> become aware of autovacuum is when all of their tables hit the freeze >>>> threshold for the first time. And this doesn't help with that. You >>>> still have to scan everything after 200 million transactions; it's >>>> just that you no longer have to do it again every 200 million >>>> transactions after that. I still think it's a great feature, though. >>> >>> >>> Er... we don't provide a warning ahead of time in the logs or something? >> >> >> No. That would be a little strange, honestly. I have to assume that >> many wraparound vacuums go totally unnoticed; how would you >> distinguish the ones that are likely to annoy somebody from the other >> ones? > > > I suggest providing a config option for those warnings, if there isn't one > already. Global plus override per table etc. The option says we only log > the warnings if the wraparound vacuum is likely to take more than a certain > amount of time, and pick something reasonable for that default time. Or to > generalize, have a config saying what to do if a wraparound vacuum is coming > up soon, including what amount of estimated time may be considered > inconvenient. -- Darren Duncan postgres users no longer care about anti-wraparound vacuum at 9.6 and postgres already emits WARNING log at 10 million transaction remaining. Regards, -- Masahiko Sawada
On 4/12/16 5:13 PM, Darren Duncan wrote: >>> Er... we don't provide a warning ahead of time in the logs or something? >> >> No. That would be a little strange, honestly. I have to assume that >> many wraparound vacuums go totally unnoticed; how would you >> distinguish the ones that are likely to annoy somebody from the other >> ones? > > I suggest providing a config option for those warnings Warnings just plaster around the real problem, which is we need more intelligent Autovac launcher and worker scheduling. -- Jim C. Nasby, Data Architect jim@nasby.net 512.569.9461 (cell) http://jim.nasby.net
On 12/04/16 10:25, Josh berkus wrote: > On 04/11/2016 03:09 PM, Simon Riggs wrote: > >> Could I suggest we do the DRAFT release notes now, then discuss what >> gets highlighted? >> >> If the RMT's schedule is good, we have about 1 month to prepare it. > Hey, I'm trying to get some parallel execution going here. > > Who's being executed? I have a little list, of those who won't be missed... More seriously, 9.6 looks exciting! Cheers, Gavin P.S. I blame everything on being in Australia for a few days (back in NZ now)!