Thread: dynamic shared memory: wherein I am punished for good intentions
Since, as has been previously discussed in this forum on multiple occasions [citation needed], the default System V shared memory limits are absurdly low on many systems, the dynamic shared memory patch defaults to POSIX shared memory, which has often been touted as a superior alternative [citation needed]. Unfortunately, the buildfarm isn't entirely happy with this decision. On buildfarm member anole (HP-UX B.11.31), allocation of dynamic shared memory fails with a "Permission denied" error, and on smew (Debian GNU/Linux 6.0), it fails with "Function not implemented", which according to a forum post[1] I found probably indicates that /dev/shm doesn't mount a tmpfs on that box. What shall we do about this? I see a few options. (1) Define the issue as "not our problem". IOW, as of now, if you want to use PostgreSQL, you've got to either make POSIX shared memory work on your machine, or change the GUC that selects the type of dynamic shared memory used. (2) Default to using System V shared memory. If people want POSIX shared memory, let them change the default. (3) Add a new setting that auto-probes for a type of shared memory that works. Try POSIX first, then System V. Maybe even fall back to mmap'd files if neither of those works. (4) Remove the option to use POSIX shared memory. System V FTW! After some consideration, I think my vote is for option #2. Option #1 seems too user-hostile, especially for a facility that has no in-core users yet, though I can imagine we might want to go that way eventually, especially if we at some point try to dump System V shared memory altogether, as has been proposed. Option #4 seems sad; we know that System V shared memory limits are a problem for plenty of people, so it'd be a shame not to have a way to use the POSIX facilities if they're available. Option #3 is fine as far as it goes, but it adds a significant amount of complexity I'd rather not deal with. Other votes? Other ideas? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company [1] http://forums.justlinux.com/showthread.php?142429-shm_open-problem
Robert Haas wrote > Unfortunately, the buildfarm > isn't entirely happy with this decision. On buildfarm member anole > (HP-UX B.11.31), allocation of dynamic shared memory fails with a > "Permission denied" error, and on smew (Debian GNU/Linux 6.0), it > fails with "Function not implemented", which according to a forum > post[1] I found probably indicates that /dev/shm doesn't mount a tmpfs > on that box. > > What shall we do about this? I see a few options. Is this something that rightly falls into being a distro/package specific setting? If so then the first goal should be to ensure the maximum number of successful basic installation scenarios - namely someone installing PostgreSQL and connect to the running postgres database without encountering an error. As a default I would presume the current System V behavior is sufficient to accomplish this goal. If package maintainers can then guarantee that changing the default will improve the user experience they should be supported and encouraged to do so but if they are at all unsure they should leave the default in place. As long as a new user is able to get a running database on their machine if/when they run up against the low defaults of System V memory they will at least be able to focus on that single problem as opposed to having a failed initial install and being unsure exactly what they may have done wrong. Thus option # 2 seems sufficient. I do think that having some kind of shared-memory-manager utility could have value but I'd rather see that be a standalone utility as opposed to something magical done inside the bowels of the database. While probably harder to code and learn such a utility would provide for a much greater UX if implemented well. David J. -- View this message in context: http://postgresql.1045698.n5.nabble.com/dynamic-shared-memory-wherein-I-am-punished-for-good-intentions-tp5774055p5774080.html Sent from the PostgreSQL - hackers mailing list archive at Nabble.com.
On Thu, Oct 10, 2013 at 1:13 PM, Robert Haas <robertmhaas@gmail.com> wrote: > (1) Define the issue as "not our problem". IOW, as of now, if you > want to use PostgreSQL, you've got to either make POSIX shared memory > work on your machine, or change the GUC that selects the type of > dynamic shared memory used. > > (2) Default to using System V shared memory. If people want POSIX > shared memory, let them change the default. > > (3) Add a new setting that auto-probes for a type of shared memory > that works. Try POSIX first, then System V. Maybe even fall back to > mmap'd files if neither of those works. > > (4) Remove the option to use POSIX shared memory. System V FTW! > > After some consideration, I think my vote is for option #2. Option #1 > seems too user-hostile, especially for a facility that has no in-core > users yet, though I can imagine we might want to go that way > eventually, especially if we at some point try to dump System V shared > memory altogether, as has been proposed. Option #4 seems sad; we know > that System V shared memory limits are a problem for plenty of people, > so it'd be a shame not to have a way to use the POSIX facilities if > they're available. Option #3 is fine as far as it goes, but it adds a > significant amount of complexity I'd rather not deal with. > > Other votes? Other ideas? I believe option 2 is not only good for now, but also a necessary previous step in the way to option 3, which I believe should be the goal. So, ship a version with option 2, then implement 3, and make it the default when enough people (using option 2) have successfully tested pg's implementation of POSIX shared memory.
On 10/10/2013 12:13 PM, Robert Haas wrote: > Since, as has been previously discussed in this forum on multiple > occasions [citation needed], the default System V shared memory limits > are absurdly low on many systems, the dynamic shared memory patch > defaults to POSIX shared memory, which has often been touted as a > superior alternative [citation needed]. Unfortunately, the buildfarm > isn't entirely happy with this decision. On buildfarm member anole > (HP-UX B.11.31), allocation of dynamic shared memory fails with a > "Permission denied" error, and on smew (Debian GNU/Linux 6.0), it > fails with "Function not implemented", which according to a forum > post[1] I found probably indicates that /dev/shm doesn't mount a tmpfs > on that box. > > What shall we do about this? I see a few options. > > (1) Define the issue as "not our problem". IOW, as of now, if you > want to use PostgreSQL, you've got to either make POSIX shared memory > work on your machine, or change the GUC that selects the type of > dynamic shared memory used. > > (2) Default to using System V shared memory. If people want POSIX > shared memory, let them change the default. > > (3) Add a new setting that auto-probes for a type of shared memory > that works. Try POSIX first, then System V. Maybe even fall back to > mmap'd files if neither of those works. > > (4) Remove the option to use POSIX shared memory. System V FTW! > > After some consideration, I think my vote is for option #2. Option #1 > seems too user-hostile, especially for a facility that has no in-core > users yet, though I can imagine we might want to go that way > eventually, especially if we at some point try to dump System V shared > memory altogether, as has been proposed. Option #4 seems sad; we know > that System V shared memory limits are a problem for plenty of people, > so it'd be a shame not to have a way to use the POSIX facilities if > they're available. Option #3 is fine as far as it goes, but it adds a > significant amount of complexity I'd rather not deal with. > > Other votes? Other ideas? > 5) test and set it in initdb. cheers andrew
On Thu, Oct 10, 2013 at 2:21 PM, Andrew Dunstan <andrew@dunslane.net> wrote: >> Other votes? Other ideas? > > 5) test and set it in initdb. Are you advocating for that option, or just calling out that it's possible? I'd say that's closely related to option #3, except at initdb time rather than run-time - and it might be preferable to #3 for some of the same reasons discussed on the thread about tuning work_mem, namely, that having it change from one postmaster lifetime to the next might lead to user astonishment. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Thu, Oct 10, 2013 at 9:13 AM, Robert Haas <robertmhaas@gmail.com> wrote: > (2) Default to using System V shared memory. If people want POSIX > shared memory, let them change the default. > After some consideration, I think my vote is for option #2. Wouldn't that become the call of packagers? Wasn't there already some reason why it was advantageous for FreeBSD to continue to use System V shared memory? -- Peter Geoghegan
On Thu, Oct 10, 2013 at 2:36 PM, Peter Geoghegan <pg@heroku.com> wrote: > On Thu, Oct 10, 2013 at 9:13 AM, Robert Haas <robertmhaas@gmail.com> wrote: >> (2) Default to using System V shared memory. If people want POSIX >> shared memory, let them change the default. > >> After some consideration, I think my vote is for option #2. > > Wouldn't that become the call of packagers? Packagers can certainly override whatever we do, but we still need to make the buildfarm green again. > Wasn't there already some > reason why it was advantageous for FreeBSD to continue to use System V > shared memory? Yes, but this code doesn't affect the main shared memory segment, so I think that's sort of a separate point. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 10/10/2013 02:35 PM, Robert Haas wrote: > On Thu, Oct 10, 2013 at 2:21 PM, Andrew Dunstan <andrew@dunslane.net> wrote: >>> Other votes? Other ideas? >> 5) test and set it in initdb. > Are you advocating for that option, or just calling out that it's > possible? I'd say that's closely related to option #3, except at > initdb time rather than run-time - and it might be preferable to #3 > for some of the same reasons discussed on the thread about tuning > work_mem, namely, that having it change from one postmaster lifetime > to the next might lead to user astonishment. Mainly just to throw it into the mix, But like you I think it's probably a better option than #3 for the reason you give. It also has the advantage of keeping any probing code out of the backend. cheers andrew
* Robert Haas (robertmhaas@gmail.com) wrote:
> On Thu, Oct 10, 2013 at 2:36 PM, Peter Geoghegan <pg@heroku.com> wrote:
> > On Thu, Oct 10, 2013 at 9:13 AM, Robert Haas <robertmhaas@gmail.com> wrote:
> >> (2) Default to using System V shared memory. If people want POSIX
> >> shared memory, let them change the default.
> >
> >> After some consideration, I think my vote is for option #2.
> >
> > Wouldn't that become the call of packagers?
>
> Packagers can certainly override whatever we do, but we still need to
> make the buildfarm green again.
While I agree that making the buildfarm green is valuable, I really
wonder about a system where /dev/shm is busted.
Personally, I like Andrew's suggestion to test and set accordingly, with
the default being POSIX if it's available and a fall-back to SysV (maybe
with a warning..). Going back to the situation where our default set-up
limits us to the ridiculously small SysV value would *really* suck; even
if we don't have any users today, we're certainly going to have some
soon and I don't think they'll be happy with a 24MB (or whatever) limit.
Thanks,
Stephen
On Thu, Oct 10, 2013 at 11:13 AM, Robert Haas <robertmhaas@gmail.com> wrote: > Since, as has been previously discussed in this forum on multiple > occasions [citation needed], the default System V shared memory limits > are absurdly low on many systems, the dynamic shared memory patch > defaults to POSIX shared memory, which has often been touted as a > superior alternative [citation needed]. Unfortunately, the buildfarm > isn't entirely happy with this decision. On buildfarm member anole > (HP-UX B.11.31), allocation of dynamic shared memory fails with a > "Permission denied" error, and on smew (Debian GNU/Linux 6.0), it > fails with "Function not implemented", which according to a forum > post[1] I found probably indicates that /dev/shm doesn't mount a tmpfs > on that box. > > What shall we do about this? I see a few options. > > (1) Define the issue as "not our problem". IOW, as of now, if you > want to use PostgreSQL, you've got to either make POSIX shared memory > work on your machine, or change the GUC that selects the type of > dynamic shared memory used. > > (2) Default to using System V shared memory. If people want POSIX > shared memory, let them change the default. Doesn't #2 negate all advantages of this effort? Bringing sysv management back on the table seems like a giant step backwards -- or am I missing something? http://www.postgresql.org/docs/9.3/interactive/kernel-resources.html#SYSVIPC merlin
On 10/10/2013 02:45 PM, Robert Haas wrote: > On Thu, Oct 10, 2013 at 2:36 PM, Peter Geoghegan <pg@heroku.com> wrote: >> On Thu, Oct 10, 2013 at 9:13 AM, Robert Haas <robertmhaas@gmail.com> wrote: >>> (2) Default to using System V shared memory. If people want POSIX >>> shared memory, let them change the default. >>> After some consideration, I think my vote is for option #2. >> Wouldn't that become the call of packagers? > Packagers can certainly override whatever we do, but we still need to > make the buildfarm green again. > > I really dislike throwing things over the wall to packagers like this, anyway. Quite apart from anything else, not everyone uses pre-built packages, and we should make things as as easy as possible for those who don't, too. cheers andrew
On Thu, Oct 10, 2013 at 12:13:20PM -0400, Robert Haas wrote: > Since, as has been previously discussed in this forum on multiple > occasions [citation needed], the default System V shared memory limits > are absurdly low on many systems, the dynamic shared memory patch > defaults to POSIX shared memory, which has often been touted as a > superior alternative [citation needed]. Unfortunately, the buildfarm > isn't entirely happy with this decision. On buildfarm member anole > (HP-UX B.11.31), allocation of dynamic shared memory fails with a > "Permission denied" error, and on smew (Debian GNU/Linux 6.0), it > fails with "Function not implemented", which according to a forum > post[1] I found probably indicates that /dev/shm doesn't mount a tmpfs > on that box. > > What shall we do about this? I see a few options. > > (1) Define the issue as "not our problem". IOW, as of now, if you > want to use PostgreSQL, you've got to either make POSIX shared memory > work on your machine, or change the GUC that selects the type of > dynamic shared memory used. +1 for this. Cheers, David. -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
On Thu, Oct 10, 2013 at 4:00 PM, Merlin Moncure <mmoncure@gmail.com> wrote: >> (2) Default to using System V shared memory. If people want POSIX >> shared memory, let them change the default. > > Doesn't #2 negate all advantages of this effort? Bringing sysv > management back on the table seems like a giant step backwards -- or > am I missing something? Not unless there's no difference between "the default" and "the only option". -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Robert, >> Doesn't #2 negate all advantages of this effort? Bringing sysv >> management back on the table seems like a giant step backwards -- or >> am I missing something? > > Not unless there's no difference between "the default" and "the only option". Well, per our earlier discussion about "the first 15 minutes", there actually isn't a difference. We got rid of SHMMAX for the majority of our users for 9.3. We should NOT revert that just so we can support older platforms -- especially since, if anything, Kernel.org is going to cripple SysV support even further in the future. The platforms where it does work represent the vast majority of our users, and are only on the increase. I can only see two reasonable alternatives: This one: > (3) Add a new setting that auto-probes for a type of shared memory > that works. Try POSIX first, then System V. Maybe even fall back to > mmap'd files if neither of those works. Or: (5) Default to POSIX, and allow for SysV as a compile-time option for platforms with poor POSIX memory support. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
On 2013-10-10 12:13:20 -0400, Robert Haas wrote: > and on smew (Debian GNU/Linux 6.0), it > fails with "Function not implemented", which according to a forum > post[1] I found probably indicates that /dev/shm doesn't mount a tmpfs > on that box. It would be nice to get confirmed what the reason for that is - afaik debian has mounted /dev/shm for ages. The most likely issue I can see is an incorrectly setup chroot. If it's just that I wouldn't be too concerned about it. That's a scenario that won't happen to many users and relatively easily fixed Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
On Thu, Oct 10, 2013 at 5:14 PM, Josh Berkus <josh@agliodbs.com> wrote: >>> Doesn't #2 negate all advantages of this effort? Bringing sysv >>> management back on the table seems like a giant step backwards -- or >>> am I missing something? >> >> Not unless there's no difference between "the default" and "the only option". > > Well, per our earlier discussion about "the first 15 minutes", there > actually isn't a difference. Harsh. :-) > I can only see two reasonable alternatives: > > This one: >> (3) Add a new setting that auto-probes for a type of shared memory >> that works. Try POSIX first, then System V. Maybe even fall back to >> mmap'd files if neither of those works. > > Or: > > (5) Default to POSIX, and allow for SysV as a compile-time option for > platforms with poor POSIX memory support. OK, I did #5. Let's see how that works. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
>> (5) Default to POSIX, and allow for SysV as a compile-time option for >> platforms with poor POSIX memory support. > > OK, I did #5. Let's see how that works. Andrew pointed out upthread that, since platforms are unlikely to change what they support dynamically, we could do this at initdb time instead. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
On Thu, Oct 10, 2013 at 7:59 PM, Josh Berkus <josh@agliodbs.com> wrote: >>> (5) Default to POSIX, and allow for SysV as a compile-time option for >>> platforms with poor POSIX memory support. >> >> OK, I did #5. Let's see how that works. > > Andrew pointed out upthread that, since platforms are unlikely to change > what they support dynamically, we could do this at initdb time instead. Oh, sorry, that's what I did. I missed the fact that your #5 and his #5 were different. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company