Thread: initdb fails on Centos 5.4 x64
Hi,
i've tried to install 8.4.3 for about two days, but i don't get it.
8.1.x has been installed before and was runnig but never used. I've removed it with yum and wiped /var/lib/pgsl.
-=>> service postgresql initdb
Initializing database: [FAILED]
-=>> cat /var/lib/pgsql/pgstartup.log
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale en_US.UTF-8.
The default database encoding has accordingly been set to UTF8.
The default text search configuration will be set to "english".
fixing permissions on existing directory /var/lib/pgsql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 24MB
creating configuration files ... ok
creating template1 database in /var/lib/pgsql/data/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating conversions ... FATAL: could not load library "/usr/lib64/pgsql/utf8_and_shift_jis_2004.so":
/usr/lib64/pgsql/utf8_and_shift_jis_2004.so:failed to map segment from shared object: Cannot allocate memory
STATEMENT: CREATE OR REPLACE FUNCTION shift_jis_2004_to_utf8 (INTEGER, INTEGER, CSTRING, INTERNAL, INTEGER) RETURNS
VOIDAS '$libdir/utf8_and_shift_jis_2004', 'shift_jis_2004_to_utf8' LANGUAGE C STRICT;
child process exited with exit code 1
initdb: removing contents of data directory "/var/lib/pgsql/data"
-=>> rpm -qa | grep postgre
postgresql-libs-8.1.18-2.el5_4.1
postgresql-libs-8.4.3-2PGDG.el5
postgresql-devel-8.4.3-2PGDG.el5
compat-postgresql-libs-4-1PGDG.rhel5
postgresql-server-8.4.3-2PGDG.el5
postgresql-plperl-8.4.3-2PGDG.el5
postgresql-8.4.3-2PGDG.el5
postgresql-contrib-8.4.3-2PGDG.el5
-=>> ldd /usr/lib64/pgsql/utf8_and_shift_jis_2004.so
libc.so.6 => /lib64/libc.so.6 (0x00002aad6e656000)
/lib64/ld-linux-x86-64.so.2 (0x0000003777a00000)
The server is running with 8 gigs ram, so i think there should be enough mem?
Thanks,
Valentin
valentin.hocher@kabelbw.de (Valentin Hocher) writes:
> creating conversions ... FATAL: could not load library "/usr/lib64/pgsql/utf8_and_shift_jis_2004.so":
/usr/lib64/pgsql/utf8_and_shift_jis_2004.so:failed to map segment from shared object: Cannot allocate memory
Hmm, there was something similar reported last month, but that person
was completely unhelpful at tracking down the problem. Could we see
the output of "ldd /usr/lib64/pgsql/utf8_and_shift_jis_2004.so"
and "readelf -d /usr/lib64/pgsql/utf8_and_shift_jis_2004.so" ?
regards, tom lane
valentin.hocher@kabelbw.de (Valentin Hocher) writes: > creating conversions ... FATAL: could not load library "/usr/lib64/pgsql/utf8_and_shift_jis_2004.so": /usr/lib64/pgsql/utf8_and_shift_jis_2004.so:failed to map segment from shared object: Cannot allocate memory FWIW, I tried to duplicate this on a RHEL5-U4 installation, without success: initdb runs fine for me. I assume you're using the RPMs from http://yum.pgsqlrpms.org/8.4/redhat/rhel-5-x86_64/ ? If so, it seems like this must be some problem in CentOS' version of the distro. What I find installed on Red Hat's machine is kernel-2.6.18-164.el5 glibc-2.5-42 (and a boatload of other stuff of course, but those seem like the packages most likely to be involved). Maybe CentOS 5.4 is a bit different? regards, tom lane
> > The solution is very simple and can be done in the cPanel configuration, just disabled "Shell Fork Bomb Protection"
inthe security center. That's all. The ulimit restrictions are removed!
>
> Huh, that's interesting. With a name like that, I'd have thought it
> would set limits on number of processes, not virtual memory size.
> What ulimit settings did it change exactly?
"Shell Fork Bomb Protection"
Description:
Fork Bomb Protection will prevent users with terminal access (ssh/telnet) from using up all the resources on the
server.Unchecked resource allocation can potentially lead to a server crash.
It is recommended that this protection be enabled for servers providing terminal access.
The only thing I discoverd if enabled is the following entry in /etc/profile (if disabled, there is no ulimit set)
#cPanel Added Limit Protections -- BEGIN
#unlimit so we can run the whoami
ulimit -n 4096 -u 14335 -m unlimited -d unlimited -s 8192 -c 1000000 -v unlimited 2>/dev/null
LIMITUSER=$USER
if [ -e "/usr/bin/whoami" ]; then
LIMITUSER=`/usr/bin/whoami`
fi
if [ "$LIMITUSER" != "root" ]; then
ulimit -n 100 -u 20 -m 200000 -d 200000 -s 8192 -c 200000 -v 200000 2>/dev/null
else
ulimit -n 4096 -u 14335 -m unlimited -d unlimited -s 8192 -c 1000000 -v unlimited 2>/dev/null
fi
#cPanel Added Limit Protections -- END
valentin.hocher@kabelbw.de (Valentin Hocher) writes:
> [ cPanel's "Shell Fork Bomb Protection" actually does this: ]
> ulimit -n 100 -u 20 -m 200000 -d 200000 -s 8192 -c 200000 -v 200000 2>/dev/null
Just to annotate that: some experimentation I did confirms that on
RHEL5 x86_64, PG 8.4.3 falls over with the mentioned error when run
under ulimit -v in the vicinity of 200000 (ie 200MB). It's kind of
surprising that initdb eats that much virtual memory space, although
certainly loading all the encoding translation libraries simultaneously
is a bit of a stress test. But the actual memory footprint is surely a
lot less than that. Apparently there is a good deal of inefficiency in
address-space consumption when loading a bunch of .so's on this
platform. I'd be interested to know if people can reproduce similar
problems on other Linux variants.
regards, tom lane
On 11/05/10 08:04, Tom Lane wrote: > valentin.hocher@kabelbw.de (Valentin Hocher) writes: >> [ cPanel's "Shell Fork Bomb Protection" actually does this: ] >> ulimit -n 100 -u 20 -m 200000 -d 200000 -s 8192 -c 200000 -v 200000 2>/dev/null > > Just to annotate that: some experimentation I did confirms that on > RHEL5 x86_64, PG 8.4.3 falls over with the mentioned error when run > under ulimit -v in the vicinity of 200000 (ie 200MB). It's kind of > surprising that initdb eats that much virtual memory space, although > certainly loading all the encoding translation libraries simultaneously > is a bit of a stress test. But the actual memory footprint is surely a > lot less than that. Apparently there is a good deal of inefficiency in > address-space consumption when loading a bunch of .so's on this > platform. I'd be interested to know if people can reproduce similar > problems on other Linux variants. I wouldn't be surprised if prelinking turned out to be the culprit there. If it's loading shared libraries to pre-selected address ranges that're globally unique across the system, it might much some significant address space. ... though come to think of it, each should be an individual mapping (right?) so it shouldn't really matter where in virtual memory they're located, just their size. Scratch that idea? -- Craig Ringer Tech-related writing: http://soapyfrogs.blogspot.com/
On Mon, May 10, 2010 at 18:04, Tom Lane <tgl@sss.pgh.pa.us> wrote: > valentin.hocher@kabelbw.de (Valentin Hocher) writes: >> [ cPanel's "Shell Fork Bomb Protection" actually does this: ] >> ulimit -n 100 -u 20 -m 200000 -d 200000 -s 8192 -c 200000 -v 200000 2>/dev/null > > Just to annotate that: some experimentation I did confirms that on > RHEL5 x86_64, PG 8.4.3 falls over with the mentioned error when run > under ulimit -v in the vicinity of 200000 (ie 200MB). > ... > I'd be interested to know if people can reproduce similar > problems on other Linux variants. FWIW on arch linux x86_64 breaks ~130000 and 32bit ~55000.