Thread: Re: Re[2]: Re: [PATCHES] A patch for xlog.c

Re: Re[2]: Re: [PATCHES] A patch for xlog.c

From
The Hermit Hacker
Date:
On Mon, 26 Feb 2001, Bruce Momjian wrote:

> > > the only problem is because if we need to tune Postermaster to use
> > > large buffer while system havn't so many SYSV shared memory, in many
> > > systemes, we need to recompile OS kernel, this is a small problem to install
> > > PGSQL to product environment.
> >
> > What?  You don't automatically recompile your OS kernel when you build a
> > system in the first place??  First step on any OS install of FreeBSD is to
> > rid myself of the 'extras' that are in the generic kernel, and enable
> > SharedMemory (even if I'm not using PgSQL on that machine) ...
>
> He is saying the machine is already in production.  Suppose he has run
> PostgreSQL for a few months, then needs to increase number of buffers.
> He can't exceed the kernel limit unless he recompiles.

Okay ... same applies to MMAP() though, I had to disappoint ... there are
kernel limits that, at least under FreeBSD, do require a kernel
recompile in order to exceed ... alot of them have been moved (maybe all
now) to sysctl settable values ... but, again, under some of the
commercial OSs, I don't think that anything but (as in Solaris) modifying
something like /etc/system and rebooting will fix ...




Re: Re[2]: Re: [PATCHES] A patch for xlog.c

From
Bruce Momjian
Date:
> Okay ... same applies to MMAP() though, I had to disappoint ... there are
> kernel limits that, at least under FreeBSD, do require a kernel
> recompile in order to exceed ... alot of them have been moved (maybe all
> now) to sysctl settable values ... but, again, under some of the
> commercial OSs, I don't think that anything but (as in Solaris) modifying
> something like /etc/system and rebooting will fix ...

But the mmap() limits are much larger than the SysV limits, aren't they,
to the point where you would never have to fiddle with the mmap() limits
to get 100mb of buffers, right?

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: Re[2]: Re: [PATCHES] A patch for xlog.c

From
The Hermit Hacker
Date:
On Mon, 26 Feb 2001, Bruce Momjian wrote:

> > Okay ... same applies to MMAP() though, I had to disappoint ... there are
> > kernel limits that, at least under FreeBSD, do require a kernel
> > recompile in order to exceed ... alot of them have been moved (maybe all
> > now) to sysctl settable values ... but, again, under some of the
> > commercial OSs, I don't think that anything but (as in Solaris) modifying
> > something like /etc/system and rebooting will fix ...
>
> But the mmap() limits are much larger than the SysV limits, aren't they,
> to the point where you would never have to fiddle with the mmap() limits
> to get 100mb of buffers, right?

Not necessarily ... it depends on the admin of the server ... then again,
I don't consider it a hassle to add a couple of lines to my kernel config
(or /etc/system) and reboot *shrug*  to me, its just part of the admin
process ...




Re: Re[2]: Re: [PATCHES] A patch for xlog.c

From
Bruce Momjian
Date:
> > But the mmap() limits are much larger than the SysV limits, aren't they,
> > to the point where you would never have to fiddle with the mmap() limits
> > to get 100mb of buffers, right?
> 
> Not necessarily ... it depends on the admin of the server ... then again,
> I don't consider it a hassle to add a couple of lines to my kernel config
> (or /etc/system) and reboot *shrug*  to me, its just part of the admin
> process ...

Are the kernel SysV defaults smaller than the mmap() kernel defaults?

I know it is easy for you, but the number of reports and problems we
hear about shows it is an issue for some.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: Re[2]: Re: [PATCHES] A patch for xlog.c

From
Tom Lane
Date:
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> I know it is easy for you, but the number of reports and problems we
> hear about shows it is an issue for some.

We hear some reports, but not a lot.  We have no idea whatever what
problems might ensue if we used mmap instead.  I'm dubious that SysV
shmem creates enough problems to justify replacing it with a solution
of essentially unknown portability characteristics...
        regards, tom lane


Re[4]: Re: [PATCHES] A patch for xlog.c

From
Xu Yifeng
Date:
Hello Tom,

Tuesday, February 27, 2001, 12:45:18 PM, you wrote:

TL> Bruce Momjian <pgman@candle.pha.pa.us> writes:
>> I know it is easy for you, but the number of reports and problems we
>> hear about shows it is an issue for some.

TL> We hear some reports, but not a lot.  We have no idea whatever what
TL> problems might ensue if we used mmap instead.  I'm dubious that SysV
TL> shmem creates enough problems to justify replacing it with a solution
TL> of essentially unknown portability characteristics...

TL>                         regards, tom lane

could anyone investigate mmap() in many modern UNIX systems to prove that
mmap() is so un-portable?

it seems mmap() is a portable problem like you said, but I think SYSV
shmem for PGSQL is a installation problem. you push some difficults to
end user, and take easy taskes for yourself.

Xu Yifeng




Re[4]: Re: [PATCHES] A patch for xlog.c

From
The Hermit Hacker
Date:
On Tue, 27 Feb 2001, Xu Yifeng wrote:

> Hello Tom,
>
> Tuesday, February 27, 2001, 12:45:18 PM, you wrote:
>
> TL> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> >> I know it is easy for you, but the number of reports and problems we
> >> hear about shows it is an issue for some.
>
> TL> We hear some reports, but not a lot.  We have no idea whatever what
> TL> problems might ensue if we used mmap instead.  I'm dubious that SysV
> TL> shmem creates enough problems to justify replacing it with a solution
> TL> of essentially unknown portability characteristics...
>
> TL>                         regards, tom lane
>
> could anyone investigate mmap() in many modern UNIX systems to prove that
> mmap() is so un-portable?
>
> it seems mmap() is a portable problem like you said, but I think SYSV
> shmem for PGSQL is a installation problem. you push some difficults to
> end user, and take easy taskes for yourself.

Considering that, so far as I can tell, both you and Bruce are the only
ones that are really heavy on moving away from SysV ... how many ppl are
actually finding it to be that much more difficult? :)




Re: Re[4]: Re: [PATCHES] A patch for xlog.c

From
Bruce Momjian
Date:
> > could anyone investigate mmap() in many modern UNIX systems to prove that
> > mmap() is so un-portable?
> >
> > it seems mmap() is a portable problem like you said, but I think SYSV
> > shmem for PGSQL is a installation problem. you push some difficults to
> > end user, and take easy taskes for yourself.
> 
> Considering that, so far as I can tell, both you and Bruce are the only
> ones that are really heavy on moving away from SysV ... how many ppl are
> actually finding it to be that much more difficult? :)

I am not sure I would call myself _heavy_ on it.  I suggest researching
it on platforms that support anon. mmap() to reduce administration load
when increasing the number of buffers.  If it is not a big win, there is
no reason to add support for it.  We clearly will be keeping SysV for a
long time, so adding another shared memory system, mmap(), should only
be done for a very good reason.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026