Thread: Time to finalize patches for 8.4 beta
Now that we are two months into the final commit fest, it is time to finalize all the open patches so we can target a February beta. The two major outstanding patches are: o SE-PostgreSQL: The author has done an outstanding job of reworking the patch so the burden is now on the community. We have to decide if we want to add this amount of new code to have both SQL row permissions and SE-Linux support. o Recovery, Replication, Hot Standby: We need a _final_ version of any patches that are targeted for 8.4. There is so much activity in this area I am unclear what is ready for 8.4. I think the remaining patches can be addressed pretty easily but we need final versions from any authors who are still adjusting them. Let's see what we can get done in the next two weeks and reevaluate. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
On Sat, 2009-01-03 at 22:34 -0500, Bruce Momjian wrote: > o Recovery, Replication, Hot Standby: We need a _final_ version > of any patches that are targeted for 8.4. There is so much activity in > this area I am unclear what is ready for 8.4. What do we mean by _final_? This a serious, polite question. Hot Standby hasn't had any serious code review yet by anybody, AFAIK. Hot Standby is not a "WIP" patch. >96-8% of the patch is identical to the way it was on Nov 1: ready for some review comments and testing/usability feedback. The later a review happens the less time we have to change anything within the patch in response to comments. If anybody is deliberately holding back on review, they need to talk to me, directly, soon. The main problem seems to be that all qualified developers are allocated elsewhere, or working on their own patches. If we want this in 8.4, then other developers or companies need to allocate some time to assist with its inclusion. All help greatly appreciated. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
Simon Riggs wrote: > > On Sat, 2009-01-03 at 22:34 -0500, Bruce Momjian wrote: > > > o Recovery, Replication, Hot Standby: We need a _final_ version > > of any patches that are targeted for 8.4. There is so much activity in > > this area I am unclear what is ready for 8.4. > > What do we mean by _final_? This a serious, polite question. > > Hot Standby hasn't had any serious code review yet by anybody, AFAIK. > > Hot Standby is not a "WIP" patch. >96-8% of the patch is identical to > the way it was on Nov 1: ready for some review comments and > testing/usability feedback. The later a review happens the less time we > have to change anything within the patch in response to comments. If > anybody is deliberately holding back on review, they need to talk to me, > directly, soon. > > The main problem seems to be that all qualified developers are allocated > elsewhere, or working on their own patches. > > If we want this in 8.4, then other developers or companies need to > allocate some time to assist with its inclusion. All help greatly > appreciated. I assume hot standby is ready for review/commit. The commit fest wiki has: o Infrastructure changes for recoveryo Hot Standby - queries during archive recoveryo rmgr hooks and contrib/rmgr_hooko Synchronous log-shipping replication Are these all ready for review/commit? I assume all are except I don't know the answer for the last one. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
On Mon, 2009-01-05 at 15:05 -0500, Bruce Momjian wrote: > I assume hot standby is ready for review/commit. It's in good shape, yes. With such a large patch I don't expect review->commit to be a single iteration. Though that isn't because I think there is flaky stuff in there, just that reviewers may prefer coding/modularisation to be different than it is. > The commit fest wiki > has: > > o Infrastructure changes for recovery > o Hot Standby - queries during archive recovery This is just one patch now, though Heikki has broken the first piece off again for review/commit. > o rmgr hooks and contrib/rmgr_hook Yes > o Synchronous log-shipping replication It is in rework. As *was*, IMHO, no. As agreed, yes, possibly. > Are these all ready for review/commit? I assume all are except I don't > know the answer for the last one. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
Bruce, Aside from the big patches you mentioned. The following patches do not appear to be ready to commit based on most recent feedback, and should probably be deferred for 8.5: -- Column-Level permissions -- Automatic View Update Rules -- GIN Fast Insert -- On-Disk Bitmap Index (Gianni says new patch in a week.) -- Hash-Join Filter Using Bloom Filters -- Reducing some DDL Locks to ShareLock (remaining unapplied portions) The following patches are just waiting for one more fix, and have recently work by authors. We should set a deadline for them: -- B-Tree GIN (this was reviewed late, so Oleg may fix it) -- Multi-Batch Hash Join -- PITR Performance Improvement These patches might be ready to commit, but need further review. I've attempted to get reviewers for them, but none of the RRR have the right skills/experience to do the remaining review needed (i.e. we need some reviewers to step up from -hackers!): -- posix_fadvise -- parallel restore -- Solve a problem of LC_TIME of windows Finally, these patches need performance / field testing by a wide variety of users: -- posix_fadvise -- Simple postgresql.conf wizard (or is this pushed into pgFoundry?) -- pg_upgrade script for 8.3->8.4
On Mon, 2009-01-05 at 17:12 -0800, Josh Berkus wrote: > -- B-Tree GIN (this was reviewed late, so Oleg may fix it) This one is really pretty much ready. The only reason I didn't pass it along for committer review are a couple very minor things that I thought the authors might want to comment on. If I don't see any comments later this week, I'll post a new patch. Regards,Jeff Davis
Bruce Momjian wrote: > Now that we are two months into the final commit fest, it is time to > finalize all the open patches so we can target a February beta. > > The two major outstanding patches are: > > o SE-PostgreSQL: The author has done an outstanding job of > reworking the patch so the burden is now on the community. We have to > decide if we want to add this amount of new code to have both SQL row > permissions and SE-Linux support. At least, this patch set has been ready for reviewing for several months, but it also hasn't get any serious code reviews since CommitFest:May. (No need to say, I feel grateful for some of proposals, like PGACE framework, Row-level ACLS, simultaneous DAC&MAC, ...) I've already prepared to rework the patch set on reviewer's comments, and our employer allows me to pay my efforts to v8.4 development cycle with the "highest" priority. Again, I strongly want it to be progressed. If necessary, I'll update my patches as soon as possible. Thanks, > o Recovery, Replication, Hot Standby: We need a _final_ version > of any patches that are targeted for 8.4. There is so much activity in > this area I am unclear what is ready for 8.4. > > I think the remaining patches can be addressed pretty easily but we need > final versions from any authors who are still adjusting them. > > Let's see what we can get done in the next two weeks and reevaluate. -- OSS Platform Development Division, NEC KaiGai Kohei <kaigai@ak.jp.nec.com>
On Mon, 2009-01-05 at 17:12 -0800, Josh Berkus wrote: > -- Reducing some DDL Locks to ShareLock (remaining unapplied portions) This just needs some testing by me, so don't kick this one out please. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
Simon Riggs <simon@2ndQuadrant.com> writes:
> On Mon, 2009-01-05 at 17:12 -0800, Josh Berkus wrote:
>> -- Reducing some DDL Locks to ShareLock (remaining unapplied portions)
> This just needs some testing by me, so don't kick this one out please.
The point of the discussion is that we are not going to wait too darn
much longer for patches that are "waiting on author". If you want this
in 8.4, get that testing finished. Soon.
regards, tom lane
--On Montag, Januar 05, 2009 17:12:27 -0800 Josh Berkus <josh@agliodbs.com>
wrote:
> -- Automatic View Update Rules
I'm currently adressing the issues Jaime mentioned and will provide an
updated patch very soon.
-- Thanks
Bernd
Dear Josh, On Mon, Jan 05, 2009 at 05:12:27PM -0800, Josh Berkus wrote: > -- On-Disk Bitmap Index (Gianni says new patch in a week.) while we are still working on the issues raised on pgsql-hackers, at this point it looks like that we will not be able to resolve all of them in time for the 8.4 release; thus we prefer to withdraw the patch from the current commitfest. We will update the corresponding Wiki page as soon as possible, although not immediately (I am experiencing some connectivity problems). Best regards, Dr. Gianni Ciolli - 2ndQuadrant Italia PostgreSQL Training, Services and Support gianni.ciolli@2ndquadrant.it | www.2ndquadrant.it
On Sat, 2009-01-03 at 22:34 -0500, Bruce Momjian wrote: > Now that we are two months into the final commit fest, it is time to > finalize all the open patches so we can target a February beta. > > The two major outstanding patches are: ... > o Recovery, Replication, Hot Standby: We need a _final_ version > of any patches that are targeted for 8.4. There is so much activity in > this area I am unclear what is ready for 8.4. > > I think the remaining patches can be addressed pretty easily but we need > final versions from any authors who are still adjusting them. > > Let's see what we can get done in the next two weeks and reevaluate. I'm continuing to work on Hot Standby as we speak, implementing requested changes and fixing bugs found by myself and others. Detailed project status is updated as it happens here http://wiki.postgresql.org/wiki/Hot_Standby#Project_Status_Tracking Some major refactoring was completed yesterday (after being agreed earlier that day) and there were still other things being cleaned up. There have been calls to complete this immediately from a number of hackers and although I understand the urgency, blurting out a patch that has not been through a full test cycle will hinder not help this patch. We expect a full test cycle to be completed on Friday. I will publish a new version each day on hackers until then, but please don't start shouting "you haven't tested it" cos the simple truth is we won't have *re*-tested it to the standard we all expect from PostgreSQL before then given the nature of testing required. I also want to reserve the right to discuss things on-list. I don't want to be forced into discussing things secretly simply to give a good impression. And I may even ask for more help. If you have a bug or a question or a refactoring change, ask now - on list. So let's follow what Bruce suggested above and wait the rest of the two weeks and then reevaluate. Thanks. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
Bruce Momjian wrote: > Now that we are two months into the final commit fest, it is time to > finalize all the open patches so we can target a February beta. > > The two major outstanding patches are: > > o SE-PostgreSQL: The author has done an outstanding job of > reworking the patch so the burden is now on the community. We have to > decide if we want to add this amount of new code to have both SQL row > permissions and SE-Linux support. > > o Recovery, Replication, Hot Standby: We need a _final_ version > of any patches that are targeted for 8.4. There is so much activity in > this area I am unclear what is ready for 8.4. > > I think the remaining patches can be addressed pretty easily but we need > final versions from any authors who are still adjusting them. > > Let's see what we can get done in the next two weeks and reevaluate. From the recent pgsql-hackers, it seems me people implicitly considers the last commit-fest is forcibly called off at end of the January. I believe we can have an opportunity to review our proposed patches at least, because Tom commented as follows two months ago. http://archives.postgresql.org/message-id/25086.1227240251@sss.pgh.pa.us Yes, I can understand the planned schedule to release v8.4 beta is now coming and we cannot wait for them forever. However, IIRC, SE-PostgreSQL patches have been neglected for a month (except for comments from Alvaro Herrera) since I released a revised version which supports simultaneous DAC and MAC at Dec.17. I'm afraid of SE-PostgreSQL rejected for v8.4 without serious reviews due to lack of time, if nothing is done. So, I had to review my patches by myseld for the recent weeks and rework some items which will be commented later. As I said a few times, I can work to upstream it with my highest priority (my employer also allows it for v8.4), but it is impossible by myself only. We need all your help! I'm sorry, if you felt above my concern uncomfortable. Thanks, -- KaiGai Kohei <kaigai@kaigai.gr.jp>
KaiGai Kohei wrote: > Bruce Momjian wrote: > > Now that we are two months into the final commit fest, it is time to > > finalize all the open patches so we can target a February beta. > > > > The two major outstanding patches are: > > > > o SE-PostgreSQL: The author has done an outstanding job of > > reworking the patch so the burden is now on the community. We have to > > decide if we want to add this amount of new code to have both SQL row > > permissions and SE-Linux support. > > > > o Recovery, Replication, Hot Standby: We need a _final_ version > > of any patches that are targeted for 8.4. There is so much activity in > > this area I am unclear what is ready for 8.4. > > > > I think the remaining patches can be addressed pretty easily but we need > > final versions from any authors who are still adjusting them. > > > > Let's see what we can get done in the next two weeks and reevaluate. > > From the recent pgsql-hackers, it seems me people implicitly considers > the last commit-fest is forcibly called off at end of the January. No, we still have many people working on patches and the commit-fest is going to extend into February. I am about to post to the 'announce' list saying discussion on this patch will begin on Wednesday, at 12:00 GMT. The text is below: --------------------------------------------------------------------------- The PostgreSQL community is considering including security enhancements in Postgres 8.4, e.g. row-level permissions and SE-Linux security. However, to evaluate the patch and its usefulness, we need security experts who want to use this capability or have used it in other databases. The most recent version of the patch is mentioned here: http://archives.postgresql.org/pgsql-hackers/2009-01/msg01680.php Particularly interesting is the documentation patch: http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1460.patch If you know someone who is interested in these features or can help in discussing them, please have them subscribe to pgsql-hackers here: http://www.postgresql.org/community/lists/subscribe Email discussion about this topic will start on Wednesday, 12:00 GMT, and will include the subject text "SE-PostgreSQL". -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
On 2009-01-24, at 15:33, Bruce Momjian wrote: > > The PostgreSQL community is considering including security > enhancements > in Postgres 8.4, e.g. row-level permissions and SE-Linux security. > However, to evaluate the patch and its usefulness, we need security > experts who want to use this capability or have used it in other > databases. With that sort of features, usually what happens in other projects, is that despite their evaluation during beta period - they are explicitly marked and considered as 'beta', even when product reaches release status. Maybe PostgresSql should follow that tactic too. After all, security stuff needs to be tested and exposed for sometime before considered rock solid, and trustworthy. Which of course doesn't mean, that it shouldn't start in beta's.
Bruce Momjian wrote: > KaiGai Kohei wrote: >> Bruce Momjian wrote: >>> Now that we are two months into the final commit fest, it is time to >>> finalize all the open patches so we can target a February beta. >>> >>> The two major outstanding patches are: >>> >>> o SE-PostgreSQL: The author has done an outstanding job of >>> reworking the patch so the burden is now on the community. We have to >>> decide if we want to add this amount of new code to have both SQL row >>> permissions and SE-Linux support. >>> >>> o Recovery, Replication, Hot Standby: We need a _final_ version >>> of any patches that are targeted for 8.4. There is so much activity in >>> this area I am unclear what is ready for 8.4. >>> >>> I think the remaining patches can be addressed pretty easily but we need >>> final versions from any authors who are still adjusting them. >>> >>> Let's see what we can get done in the next two weeks and reevaluate. >> From the recent pgsql-hackers, it seems me people implicitly considers >> the last commit-fest is forcibly called off at end of the January. > > No, we still have many people working on patches and the commit-fest is > going to extend into February. All right, I'll prepare the discussion on the list. I know a few SELinux developers good for database also, so I would forward the message to them. They are surely experts in security field. Thanks, > I am about to post to the 'announce' list saying discussion on this > patch will begin on Wednesday, at 12:00 GMT. The text is below: > > --------------------------------------------------------------------------- > > The PostgreSQL community is considering including security enhancements > in Postgres 8.4, e.g. row-level permissions and SE-Linux security. > However, to evaluate the patch and its usefulness, we need security > experts who want to use this capability or have used it in other > databases. > > The most recent version of the patch is mentioned here: > > http://archives.postgresql.org/pgsql-hackers/2009-01/msg01680.php > > Particularly interesting is the documentation patch: > > http://sepgsql.googlecode.com/files/sepostgresql-docs-8.4devel-3-r1460.patch Also, http://wiki.postgresql.org/wiki/SEPostgreSQL > > If you know someone who is interested in these features or can help in > discussing them, please have them subscribe to pgsql-hackers here: > > http://www.postgresql.org/community/lists/subscribe > > Email discussion about this topic will start on Wednesday, 12:00 GMT, > and will include the subject text "SE-PostgreSQL". > > -- KaiGai Kohei <kaigai@kaigai.gr.jp>
Grzegorz Jaskiewicz wrote: > > On 2009-01-24, at 15:33, Bruce Momjian wrote: > > > > The PostgreSQL community is considering including security > > enhancements > > in Postgres 8.4, e.g. row-level permissions and SE-Linux security. > > However, to evaluate the patch and its usefulness, we need security > > experts who want to use this capability or have used it in other > > databases. > > With that sort of features, usually what happens in other projects, is > that despite their evaluation during beta period - they are explicitly > marked and considered as 'beta', even when product reaches release > status. > Maybe PostgresSql should follow that tactic too. After all, security > stuff needs to be tested and exposed for sometime before considered > rock solid, and trustworthy. > Which of course doesn't mean, that it shouldn't start in beta's. Once we include the code in an official release, it is much harder to make changes so I would like to get to 8.4 final at least _thinking_ it is ready, rather than trying to adjust it after the final release. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +