AWS and postgres issues - Mailing list pgsql-general

Howdy,

I'm having a couple of problems that I believe are related to AWS and I'm wondering
if anyone's seen them / overcome them.

Brief background, I'm running PG 9.2.4 in a VPC on Amazon Linux.
I'm also (attempting) to use PgPool for load balancing/failover.

The overall problem is that it seems like some Postgres commands / operations get truncated
at a network/packet level.

For example when I try to run ( From PgPool Server => Postgres Server)
ssh -vvv -T postgres@10.0.1.30 "/usr/pgsql-9.2/bin/pg_ctl -D /db/pg -m fast restart"

The command completes successfully on the Postgres server, and the process goes away,
however on the PgPool server that process never dies, it just hangs.

PgPool box:
ps -ef|grep -i ssh|grep -v sshd|grep -v grep
pgpool   27196 26241  0 19:57 pts/0    00:00:00 ssh -vvv postgres@10.0.1.30 bash -c '/usr/pgsql-9.2/bin/pg_ctl -D
/db/pg-m fast restart' 

Postgres box:
ps -ef|grep -i pg_ctl
postgres  2376 26436  0 19:58 pts/1    00:00:00 grep -i pg_ctl

Other non-postgres commands run over ssh return as expected.

I don't know if this is helpful, but here's an strace of the process:
setsockopt(3, SOL_IP, IP_TOS, [8], 4)   = 0
time(NULL)                              = 1365450845
select(7, [3], [3], NULL, NULL)         = 1 (out [3])
time(NULL)                              = 1365450845
write(3, "2O\235qZ\333\2160\333\371\372\374\215\204\337X)\215\321J\5\343\240(\325\316\224W\370(7+"..., 176) = 176
time(NULL)                              = 1365450845
select(7, [3], [], NULL, NULL)          = 1 (in [3])
time(NULL)                              = 1365450845
read(3, "\303\223BDr5\376I\304Io\4\25\33\6\25>L\214\f_~J\342gc#w\365\5\320\242"..., 8192) = 80
time(NULL)                              = 1365450845
select(7, [3 4], [], NULL, NULL)        = 1 (in [3])
time(NULL)                              = 1365450845
read(3, "\352\366A\360c\315\t\310\361\24z\217H\t\314\342\361\322\335}l6\302)\223\343\361\27&{\234H"..., 8192) = 128
time(NULL)                              = 1365450845
select(7, [3 4], [5], NULL, NULL)       = 1 (out [5])
time(NULL)                              = 1365450845
write(5, "waiting for server to shut down."..., 35waiting for server to shut down....) = 35
time(NULL)                              = 1365450845
select(7, [3 4], [], NULL, NULL)        = 1 (in [3])
time(NULL)                              = 1365450846
read(3, "c\264\317\303Q\222\214b\323>\300\354\306j\36\31+\342\360\325Y8\345\322\211?<\0210n\253\211"..., 8192) = 64
time(NULL)                              = 1365450846
select(7, [3 4], [5], NULL, NULL)       = 1 (out [5])
time(NULL)                              = 1365450846
write(5, " done\nserver stopped\n", 21 done
server stopped
) = 21
time(NULL)                              = 1365450846
select(7, [3 4], [], NULL, NULL)        = 1 (in [3])
time(NULL)                              = 1365450846
read(3, "\253\210\306\251\343lF^6\32|v\374fe\23\32\3ylZ\325[\205\344,x@\4\201\213\351"..., 8192) = 64
time(NULL)                              = 1365450846
select(7, [3 4], [5], NULL, NULL)       = 1 (out [5])
time(NULL)                              = 1365450846
write(5, "server starting\n", 16server starting
)       = 16
time(NULL)                              = 1365450846
select(7, [3 4], [], NULL, NULL)        = 1 (in [3])
time(NULL)                              = 1365450846
read(3, "\373 \347w\354%\314<\6\215\314\207\7\202\274q\341:\270t\366\375\242{9\207:\222\374jy\373"..., 8192) = 128
close(4)                                = 0
time(NULL)                              = 1365450846
select(7, [3], [], NULL, NULL

and the same thing run with ssh -vvvv

debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug2: callback start
debug2: fd 3 setting TCP_NODELAY
debug3: packet_set_tos: set IP_TOS 0x08
debug2: client_session2_setup: id 0
debug1: Sending environment.
debug3: Ignored env HOSTNAME
debug3: Ignored env SHELL
debug3: Ignored env TERM
debug3: Ignored env HISTSIZE
debug3: Ignored env EC2_AMITOOL_HOME
debug3: Ignored env OLDPWD
debug3: Ignored env USER
debug3: Ignored env LS_COLORS
debug3: Ignored env EC2_HOME
debug3: Ignored env MAIL
debug3: Ignored env PATH
debug3: Ignored env PWD
debug3: Ignored env JAVA_HOME
debug1: Sending env LANG = en_US.UTF-8
debug2: channel 0: request env confirm 0
debug3: Ignored env AWS_CLOUDWATCH_HOME
debug3: Ignored env AWS_IAM_HOME
debug3: Ignored env HISTCONTROL
debug3: Ignored env SHLVL
debug3: Ignored env HOME
debug3: Ignored env AWS_PATH
debug3: Ignored env AWS_AUTO_SCALING_HOME
debug3: Ignored env LOGNAME
debug3: Ignored env AWS_ELB_HOME
debug3: Ignored env LESSOPEN
debug3: Ignored env AWS_RDS_HOME
debug3: Ignored env G_BROKEN_FILENAMES
debug3: Ignored env _
debug1: Sending command: bash -c '/usr/pgsql-9.2/bin/pg_ctl -D /db/pg -m fast restart'
debug2: channel 0: request exec confirm 1
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel 0: rcvd adjust 2097152
debug2: channel_input_status_confirm: type 99 id 0
debug2: exec request accepted on channel 0
waiting for server to shut down.... done
server stopped
server starting
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0
debug2: channel 0: rcvd eow
debug2: channel 0: close_read
debug2: channel 0: input open -> closed

Any help would be great. thanks

Dave