Re: 12.1 not useable: clientlib fails after a dozen queries (GSSAPI ?) - Mailing list pgsql-hackers

[ redirecting to -hackers ]

Peter <pmc@citylink.dinoex.sub.org> writes:
> But I just recognize something of interest (which I had taken for
> granted when importing the database): the flaw does NOT appear when
> accessing the database from the server's local system (with TCP and
> GSSAPI encryption active). Only from remote system.
> But then, if I go on the local system, and change the mtu:
> # ifconfig lo0 mtu 1500
> and restart the server, then I get the exact same errors locally.

Oh-ho, that is interesting.

Looking at our regression tests for gssenc, I observe that they
only try to transport a negligible amount of data (viz, single-row
boolean results).  So we'd not notice any problem that involved
multiple-packet messages.

I modified the kerberos test so that it tries a query with a less
negligible amount of data, and what I find is:

* passes on Fedora 30, with either default or 1500 mtu
* passes on FreeBSD 12.0 with default mtu
* FAILS on FreeBSD 12.0 with mtu = 1500

I haven't run it further to ground than that, but there's definitely
something fishy here.  Based on just these results one would be hard
pressed to say if it's our bug or FreeBSD's, but your report that you
don't see the failure with PG 11 makes it sound like our problem.

OTOH, I also find that there's some hysteresis in the behavior:
once it's failed, reverting the mtu to the default setting doesn't
necessarily make subsequent runs pass.  It's really hard to explain
that behavior if it's our bug.

I tested today's HEAD of our code with up-to-date FreeBSD 12.0-RELEASE-p12
running on amd64 bare metal, no jails or emulators or VIMAGE or anything.

Attached are proposed test patch, as well as client-side regression log
output from a failure.  (There's no evidence of distress in the
postmaster log, same as your report.)

            regards, tom lane

diff --git a/src/test/kerberos/t/001_auth.pl b/src/test/kerberos/t/001_auth.pl
index e3eb052..0fc18e3 100644
--- a/src/test/kerberos/t/001_auth.pl
+++ b/src/test/kerberos/t/001_auth.pl
@@ -19,7 +19,7 @@ use Test::More;
 
 if ($ENV{with_gssapi} eq 'yes')
 {
-    plan tests => 12;
+    plan tests => 15;
 }
 else
 {
@@ -195,6 +195,29 @@ sub test_access
     return;
 }
 
+# As above, but test for an arbitrary query result.
+sub test_query
+{
+    my ($node, $role, $query, $expected, $gssencmode, $test_name) = @_;
+
+    # need to connect over TCP/IP for Kerberos
+    my ($res, $stdoutres, $stderrres) = $node->psql(
+        'postgres',
+        "$query",
+        extra_params => [
+            '-XAtd',
+            $node->connstr('postgres')
+              . " host=$host hostaddr=$hostaddr $gssencmode",
+            '-U',
+            $role
+        ]);
+
+    is($res, 0, $test_name);
+    like($stdoutres, $expected, $test_name);
+    is($stderrres, "", $test_name);
+    return;
+}
+
 unlink($node->data_dir . '/pg_hba.conf');
 $node->append_conf('pg_hba.conf',
     qq{host all all $hostaddr/32 gss map=mymap});
@@ -231,6 +254,15 @@ test_access(
     "gssencmode=require",
     "succeeds with GSS-encrypted access required with host hba");
 
+# Test that we can transport a reasonable amount of data.
+test_query(
+    $node,
+    "test1",
+    'SELECT * FROM generate_series(1, 100000);',
+    qr/^1\n.*\n1024\n.*\n9999\n.*\n100000$/s,
+    "gssencmode=require",
+    "transporting 100K lines works");
+
 unlink($node->data_dir . '/pg_hba.conf');
 $node->append_conf('pg_hba.conf',
     qq{hostgssenc all all $hostaddr/32 gss map=mymap});
1..15
# Checking port 56615
# Found port 56615
# setting up Kerberos
# Running: /usr/local/bin/krb5-config --version
# Running: /usr/local/sbin/kdb5_util create -s -P secret0
Loading random data
Initializing database '/usr/home/tgl/pgsql/src/test/kerberos/tmp_check/krb5kdc/principal' for realm 'EXAMPLE.COM',
master key name 'K/M@EXAMPLE.COM'
# Running: /usr/local/sbin/kadmin.local -q addprinc -pw secret1 test1
WARNING: no policy specified for test1@EXAMPLE.COM; defaulting to no policy
Authenticating as principal test1/admin@EXAMPLE.COM with password.
Principal "test1@EXAMPLE.COM" created.
# Running: /usr/local/sbin/kadmin.local -q addprinc -randkey postgres/auth-test-localhost.postgresql.example.com
WARNING: no policy specified for postgres/auth-test-localhost.postgresql.example.com@EXAMPLE.COM; defaulting to no
policy
Authenticating as principal test1/admin@EXAMPLE.COM with password.
Principal "postgres/auth-test-localhost.postgresql.example.com@EXAMPLE.COM" created.
# Running: /usr/local/sbin/kadmin.local -q ktadd -k /usr/home/tgl/pgsql/src/test/kerberos/tmp_check/krb5.keytab
postgres/auth-test-localhost.postgresql.example.com
Authenticating as principal test1/admin@EXAMPLE.COM with password.
Entry for principal postgres/auth-test-localhost.postgresql.example.com with kvno 2, encryption type
aes256-cts-hmac-sha1-96added to keytab WRFILE:/usr/home/tgl/pgsql/src/test/kerberos/tmp_check/krb5.keytab. 
Entry for principal postgres/auth-test-localhost.postgresql.example.com with kvno 2, encryption type
aes128-cts-hmac-sha1-96added to keytab WRFILE:/usr/home/tgl/pgsql/src/test/kerberos/tmp_check/krb5.keytab. 
# Running: /usr/local/sbin/krb5kdc -P /usr/home/tgl/pgsql/src/test/kerberos/tmp_check/krb5kdc.pid
# setting up PostgreSQL instance
# Checking port 56616
# Found port 56616
Name: node
Data directory: /usr/home/tgl/pgsql/src/test/kerberos/tmp_check/t_001_auth_node_data/pgdata
Backup directory: /usr/home/tgl/pgsql/src/test/kerberos/tmp_check/t_001_auth_node_data/backup
Archive directory: /usr/home/tgl/pgsql/src/test/kerberos/tmp_check/t_001_auth_node_data/archives
Connection string: port=56616 host=/tmp/bB3RP5P9de
Log file: /usr/home/tgl/pgsql/src/test/kerberos/tmp_check/log/001_auth_node.log
# Running: initdb -D /usr/home/tgl/pgsql/src/test/kerberos/tmp_check/t_001_auth_node_data/pgdata -A trust -N
The files belonging to this database system will be owned by user "tgl".
This user must also own the server process.

The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".

Data page checksums are disabled.

creating directory /usr/home/tgl/pgsql/src/test/kerberos/tmp_check/t_001_auth_node_data/pgdata ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... America/New_York
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok

Sync to disk skipped.
The data directory might become corrupt if the operating system crashes.

Success. You can now start the database server using:

    pg_ctl -D /usr/home/tgl/pgsql/src/test/kerberos/tmp_check/t_001_auth_node_data/pgdata -l logfile start

# Running: /usr/home/tgl/pgsql/src/test/kerberos/../../../src/test/regress/pg_regress --config-auth
/usr/home/tgl/pgsql/src/test/kerberos/tmp_check/t_001_auth_node_data/pgdata
### Starting node "node"
# Running: pg_ctl -D /usr/home/tgl/pgsql/src/test/kerberos/tmp_check/t_001_auth_node_data/pgdata -l
/usr/home/tgl/pgsql/src/test/kerberos/tmp_check/log/001_auth_node.log-o --cluster-name=node start 
waiting for server to start.... done
server started
# Postmaster PID for node "node" is 1810
# running tests
### Restarting node "node"
# Running: pg_ctl -D /usr/home/tgl/pgsql/src/test/kerberos/tmp_check/t_001_auth_node_data/pgdata -l
/usr/home/tgl/pgsql/src/test/kerberos/tmp_check/log/001_auth_node.logrestart 
waiting for server to shut down.... done
server stopped
waiting for server to start.... done
server started
# Postmaster PID for node "node" is 1821
ok 1 - fails without ticket
# Running: /usr/local/bin/kinit test1
Password for test1@EXAMPLE.COM:
ok 2 - fails without mapping
### Restarting node "node"
# Running: pg_ctl -D /usr/home/tgl/pgsql/src/test/kerberos/tmp_check/t_001_auth_node_data/pgdata -l
/usr/home/tgl/pgsql/src/test/kerberos/tmp_check/log/001_auth_node.logrestart 
waiting for server to shut down.... done
server stopped
waiting for server to start.... done
server started
# Postmaster PID for node "node" is 1837
ok 3 - succeeds with mapping with default gssencmode and host hba
ok 4 - succeeds with GSS-encrypted access preferred with host hba
ok 5 - succeeds with GSS-encrypted access required with host hba
not ok 6 - transporting 100K lines works

#   Failed test 'transporting 100K lines works'
#   at t/001_auth.pl line 215.
#          got: '3'
#     expected: '0'
not ok 7 - transporting 100K lines works

#   Failed test 'transporting 100K lines works'
#   at t/001_auth.pl line 216.
#                   ''
#     doesn't match '(?^s:^1\n.*\n1024\n.*\n9999\n.*\n100000$)'
not ok 8 - transporting 100K lines works

#   Failed test 'transporting 100K lines works'
#   at t/001_auth.pl line 217.
#          got: 'psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle
# psql:<stdin>:1: message type 0x44 arrived from server while idle'
#     expected: ''
### Restarting node "node"
# Running: pg_ctl -D /usr/home/tgl/pgsql/src/test/kerberos/tmp_check/t_001_auth_node_data/pgdata -l
/usr/home/tgl/pgsql/src/test/kerberos/tmp_check/log/001_auth_node.logrestart 
waiting for server to shut down.... done
server stopped
waiting for server to start.... done
server started
# Postmaster PID for node "node" is 1854
ok 9 - succeeds with GSS-encrypted access preferred and hostgssenc hba
ok 10 - succeeds with GSS-encrypted access required and hostgssenc hba
ok 11 - fails with GSS encryption disabled and hostgssenc hba
### Restarting node "node"
# Running: pg_ctl -D /usr/home/tgl/pgsql/src/test/kerberos/tmp_check/t_001_auth_node_data/pgdata -l
/usr/home/tgl/pgsql/src/test/kerberos/tmp_check/log/001_auth_node.logrestart 
waiting for server to shut down.... done
server stopped
waiting for server to start.... done
server started
# Postmaster PID for node "node" is 1869
ok 12 - succeeds with GSS-encrypted access preferred and hostnogssenc hba, but no encryption
ok 13 - fails with GSS-encrypted access required and hostnogssenc hba
ok 14 - succeeds with GSS encryption disabled and hostnogssenc hba
### Restarting node "node"
# Running: pg_ctl -D /usr/home/tgl/pgsql/src/test/kerberos/tmp_check/t_001_auth_node_data/pgdata -l
/usr/home/tgl/pgsql/src/test/kerberos/tmp_check/log/001_auth_node.logrestart 
waiting for server to shut down.... done
server stopped
waiting for server to start.... done
server started
# Postmaster PID for node "node" is 1885
ok 15 - succeeds with include_realm=0 and defaults
### Stopping node "node" using mode immediate
# Running: pg_ctl -D /usr/home/tgl/pgsql/src/test/kerberos/tmp_check/t_001_auth_node_data/pgdata -m immediate stop
waiting for server to shut down.... done
server stopped
# No postmaster PID for node "node"
# Looks like you failed 3 tests of 15.