Re: Memory Usage and OpenBSD - Mailing list pgsql-general

From Anton Maksimenkov
Subject Re: Memory Usage and OpenBSD
Date
Msg-id 8cac8dd1002090719i7ac66896pf0541ee9bcae37c8@mail.gmail.com
Whole thread Raw
In response to Re: Memory Usage and OpenBSD  (Scott Marlowe <scott.marlowe@gmail.com>)
Responses Re: Memory Usage and OpenBSD  (Martijn van Oosterhout <kleptog@svana.org>)
List pgsql-general
2010/2/9 Scott Marlowe <scott.marlowe@gmail.com>:
> On Tue, Feb 9, 2010 at 3:18 AM, Anton Maksimenkov <anton200@gmail.com> wrote:
>>> Isn't the usual advice here is to log the ulimit setting from the pg
>>> startup script so you can what it really is for the user at the moment
>> I think that "su" is enough:
> In previous discussions it was mentioned that startup scripts often
> inherit different settings from the default shell of a user, hence the

Hm. My postgres starts with following /etc/rc.local
...
if [ -x /usr/local/bin/pg_ctl ]; then
        echo -n ' postgresql'
        su -l _postgresql -c "nohup /usr/local/bin/pg_ctl start \
            -D /var/postgresql/data -l /var/postgresql/logfile \
            -o '-i -D /var/postgresql/data' >/dev/null"
fi
...
I start it manually by copy/paste of text above.

But I want explain something, just to be clear.

In openbsd on i386 we have 4G virtual space.
768M of it used for kernel virtual space. So 4G - 768 = about 3,2G -
that is userspace.
userspace itself is "divided" to 2 parts. First part of that virtual
space is about 1G and reserved for old *brk() interface. The rest 3,2G
- 1G = about 2,2 G is the virtual space which used by mmap, shm*,
malloc (it use mmap).

It means that on openbsd i386 we have about 2,2G of virtual space for
malloc, shm*. So, postgres will use that space.

But mmap() use random addresses. So when you get big chunk of memory
for shared buffers (say, 2G) then you may get it somewhere in middle
of virtual space (2,2G). In worst case it might be:
100Mb freespace from (start)...to (start+100Mb)
2G shared memory chunk from (start+100Mb) to (start + 100 Mb + 2G)
100Mb freespace from (start + 100 Mb + 2G) to (end)

After that, if you want to allocate more than 100Mb by one big chunk -
you can't. This is why postgres may fail at "vacuum" or "dump" or
somewhere else - it may happen when postgres try to allocate one big
chunk.

Can anybody briefly explain me how one postgres process allocate
memory for it needs?
I mean, what is the biggest size of malloc() it may want? How many
such chunks? What is the average size of allocations?
--
antonvm

pgsql-general by date:

Previous
From: Korry Douglas
Date:
Subject: Re: ERROR: could not load library "...": Exec format error
Next
From: David Kerr
Date:
Subject: Re: viewing large queries in pg_stat_activity