Re: [SOLVED] Re: Hung postmaster (8.3.9) - Mailing list pgsql-general

On Monday 01 March 2010 @ 18:29, Tom Lane wrote:
> "Ed L." <pgsql@bluepolka.net> writes:
> > That did the trick.  Thank you very much, Sensei.
>
> I'd still like to know about platform etc.  I see that we
>  shouldn't be allowing a username to trigger @-file expansion,
>  but even with that it's not clear how it turned into an
>  infinite loop; and as I said, I can't reproduce that here.

The original problem occurred on that CentOS box, but I just
reproduced it on a new cluster on Ubuntu running LVM but no
RAID.  Here's the transcript:

ed@duke:~/test$ uname -a
Linux duke 2.6.31-15-generic #50-Ubuntu SMP Tue Nov 10 14:53:52 UTC 2009 x86_64 GNU/Linux

ed@duke:~/test$ export PGPORT=9000
ed@duke:~/test$ w^C
ed@duke:~/test$ svi
ed@duke:~/test$ export PGDATA=`pwd`/data
ed@duke:~/test$ initdb --locale=C -D $PGDATA
The files belonging to this database system will be owned by user "ed".
This user must also own the server process.

The database cluster will be initialized with locale C.
The default database encoding has accordingly been set to SQL_ASCII.
The default text search configuration will be set to "english".

creating directory /home/ed/test/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers/max_fsm_pages ... 32MB/204800
creating configuration files ... ok
creating template1 database in /home/ed/test/data/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the -A option the
next time you run initdb.

Success. You can now start the database server using:

    postgres -D /home/ed/test/data
or
    pg_ctl -D /home/ed/test/data -l logfile start

ed@duke:~/test$ pg_ctl -D /home/ed/test/data -l logfile start
server starting
ed@duke:~/test$ psql -c "select version()"
psql: FATAL:  database "ed" does not exist
ed@duke:~/test$ create^C
ed@duke:~/test$ createdb
ed@duke:~/test$ psql -c "select version()"
                                             version
-------------------------------------------------------------------------------------------------
 PostgreSQL 8.3.9 on x86_64-unknown-linux-gnu, compiled by GCC gcc (Ubuntu 4.4.1-4ubuntu8) 4.4.1
(1 row)

ed@duke:~/test$ createuser "@" with login
createuser: too many command-line arguments (first is "with")
Try "createuser --help" for more information.
ed@duke:~/test$ psql -c "create user \"@\" with login"
CREATE ROLE
ed@duke:~/test$ cat data/global/
1136         1214         1260         2396         2672         2694         2698         2844         2847
pg_database  
1137         1232         1261         2397         2676         2695         2842         2845         pg_auth
pgstat.stat  
1213         1233         1262         2671         2677         2697         2843         2846         pg_control
ed@duke:~/test$ cat data/global/pg_auth
"@" "" ""
"ed" "" ""
ed@duke:~/test$ pg_ctl -D $PGDATA stop
waiting for server to shut down...........................................^C
ed@duke:~/test$ ^C
ed@duke:~/test$ ^C
ed@duke:~/test$ ^C
ed@duke:~/test$ pg_ctl -D $PGDATA -m fast stop
waiting for server to shut down.............^C
ed@duke:~/test$ ^C
ed@duke:~/test$ ps augxwwwf | grep postma
ed        8419  0.0  0.0   7336   868 pts/4    S+   18:36   0:00      \_ grep postma
ohsdba    6115  0.0  0.3 503692 28028 pts/0    S    18:03   0:00 postmaster -D
/users/ohsdba/dbclusters/ohs/postgresql-8.3.x/data-i -p 9001 
ed@duke:~/test$ gdb `which postgre^C
ed@duke:~/test$ pg_ctl -D /home/ed/test/data -l logfile start
pg_ctl: another server might be running; trying to start server anyway
pg_ctl: could not start server
Examine the log output.
ed@duke:~/test$ cat logfile
LOG:  database system was shut down at 2010-03-01 18:34:25 MST
LOG:  autovacuum launcher started
LOG:  database system is ready to accept connections
FATAL:  database "ed" does not exist
WARNING:  worker took too long to start; cancelled
FATAL:  lock file "postmaster.pid" already exists
HINT:  Is another postmaster (PID 8203) running in data directory "/home/ed/test/data"?
ed@duke:~/test$ ps augxwwwf | grep 8203
ed        8465  0.0  0.0   7336   868 pts/4    S+   18:37   0:00      \_ grep 8203
ed        8203 73.7  0.0  63080  4248 pts/4    R    18:34   2:07 /opt/pgsql/installs/postgresql-8.3.9/bin/postgres -D
/home/ed/test/data
ed@duke:~/test$ gdb `which postgres`
GNU gdb (GDB) 7.0-ubuntu
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /opt/pgsql/installs/postgresql-8.3.9/bin/postgres...done.
(gdb) attach 8203
Attaching to program: /opt/pgsql/installs/postgresql-8.3.9/bin/postgres, process 8203
Reading symbols from /lib/libcrypt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypt.so.1
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libm.so.6
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib/libnss_compat.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_compat.so.2
Reading symbols from /lib/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /lib/libnss_nis.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_nis.so.2
Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_files.so.2
0x00007f3c58f9ea30 in read () from /lib/libc.so.6
(gdb) bt
#0  0x00007f3c58f9ea30 in read () from /lib/libc.so.6
#1  0x00007f3c58f3ead8 in _IO_file_underflow () from /lib/libc.so.6
#2  0x00007f3c58f4065e in _IO_default_uflow () from /lib/libc.so.6
#3  0x00007f3c58f3741b in getc () from /lib/libc.so.6
#4  0x00000000005460d8 in T.136 ()
#5  0x00000000005466e4 in tokenize_file ()
#6  0x00000000005467ed in tokenize_file ()
#7  0x0000000000546cef in load_role ()
#8  0x00000000005937ed in sigusr1_handler ()
#9  <signal handler called>
#10 0x00007f3c58fa5373 in select () from /lib/libc.so.6
#11 0x0000000000591cc2 in ServerLoop ()
#12 0x0000000000594467 in PostmasterMain ()
#13 0x000000000054a9b0 in main ()
(gdb) fin
Run till exit from #0  0x00007f3c58f9ea30 in read () from /lib/libc.so.6
0x00007f3c58f3ead8 in _IO_file_underflow () from /lib/libc.so.6
(gdb) fin
Run till exit from #0  0x00007f3c58f3ead8 in _IO_file_underflow () from /lib/libc.so.6
0x00007f3c58f4065e in _IO_default_uflow () from /lib/libc.so.6
(gdb) fin
Run till exit from #0  0x00007f3c58f4065e in _IO_default_uflow () from /lib/libc.so.6
0x00007f3c58f3741b in getc () from /lib/libc.so.6
(gdb) fin
Run till exit from #0  0x00007f3c58f3741b in getc () from /lib/libc.so.6
0x00000000005460d8 in T.136 ()
(gdb) fin
Run till exit from #0  0x00000000005460d8 in T.136 ()
0x00000000005466e4 in tokenize_file ()
(gdb) fin
Run till exit from #0  0x00000000005466e4 in tokenize_file ()

(Infinite loop here...)

Program terminated with signal SIGKILL, Killed.
The program no longer exists.
(gdb)
The program is not running.
(gdb)
The program is not running.
(gdb) quit
ed@duke:~/test$ uname -a
Linux duke 2.6.31-15-generic #50-Ubuntu SMP Tue Nov 10 14:53:52 UTC 2009 x86_64 GNU/Linux
ed@duke:~/test$ uname -a
Linux duke 2.6.31-15-generic #50-Ubuntu SMP Tue Nov 10 14:53:52 UTC 2009 x86_64 GNU/Linux
ed@duke:~/test$ df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/duke-root
                     468727296 229717260 215199988  52% /
udev                   4096736       320   4096416   1% /dev
none                   4096736      1280   4095456   1% /dev/shm
none                   4096736       156   4096580   1% /var/run
none                   4096736         0   4096736   0% /var/lock
none                   4096736         0   4096736   0% /lib/init/rw
/dev/sda5               233335     47309    173578  22% /boot
/dev/sr0                   388       388         0 100% /media/cdrw