Re: User defined data types in Logical Replication - Mailing list pgsql-hackers

From Kyotaro HORIGUCHI
Subject Re: User defined data types in Logical Replication
Date
Msg-id 20180319.161139.248923135.horiguchi.kyotaro@lab.ntt.co.jp
Whole thread Raw
In response to Re: User defined data types in Logical Replication  (Masahiko Sawada <sawada.mshk@gmail.com>)
List pgsql-hackers
At Mon, 19 Mar 2018 12:50:55 +0900, Masahiko Sawada <sawada.mshk@gmail.com> wrote in
<CAD21AoD=kiDVxfZ2aT_Oeg7+5etkxg0eqmsRE-gcUbptKNir6g@mail.gmail.com>
> On Fri, Mar 16, 2018 at 10:24 AM, Alvaro Herrera
> <alvherre@alvh.no-ip.org> wrote:
> > Masahiko Sawada wrote:
> >> On Thu, Mar 15, 2018 at 9:41 AM, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> >
> >> > I think this is a worthwhile test, but IMO it should be improved a bit
> >> > before we include it.  Also, we can come up with a better name for the
> >> > test surely, not just refer to this particular bug. Maybe "typemap".
> >>
> >> It might be useful if we have the facility of TAP test to check the
> >> log message and regexp-match the message to the expected string.
> >
> > Something similar to PostgresNode::issues_sql_like() perhaps?
> >
> 
> Yeah, I didn't know that but I think it's a good idea. Unlike
> issues_sql_like() we don't issue anything to the subscriber. So maybe
> we need truncate logfile before insertion and verify logfile of
> particular period. The test code would be like follows.
> 
> $node_subscriber->safe_psql('postgres', 'CREATE SUBSCRIPTION...");
> truncate $node_subscriber->logfile, 0;
> $node_publisher->safe_psql('postgres', 'INSERT .. ')
> my $log = TestLib::slurp_file($node_subscriber->logfile);
> 
> # Verify logs
> like($log, qr/processing remote data for replication target relation
> "public.test" column "b", remote type dummyint, local type dummyint/,
> 'callback function of datatype conversion1');
> like($log, qr/processing remote data for replication target relation
> "public.test" column "a", remote type dummytext, local type
> dummytext/, 'callback function of datatype conversion2');
> 
> Thoughts?

FWIW something like is in a currently proposed patch.

https://www.postgresql.org/message-id/20180129.194023.228030941.horiguchi.kyotaro@lab.ntt.co.jp

0003 contains the following function.

+# find $pat in logfile of $node after $off-th byte
+sub find_in_log
+{
+    my ($node, $pat, $off) = @_;
+
+    $off = 0 unless defined $off;
+    my $log = TestLib::slurp_file($node->logfile);
+    return 0 if (length($log) <= $off);
+
+    $log = substr($log, $off);
+
+    return $log =~ m/$pat/;
+}

It is used as the follows.

+$logstart = get_log_size($node_standby);
+$node_standby->start;
+
+my $failed = 0;
+for (my $i = 0 ; $i < 10000 ; $i++)
+{
+    if (find_in_log($node_standby,
+                    "requested WAL segment [0-9A-F]+ has already been removed",
+                    $logstart))
+    {
+        $failed = 1;
+        last;
+    }
+    usleep(100_000);
+}
+ok($failed, 'check replication has been broken');

regards,

-- 
Kyotaro Horiguchi
NTT Open Source Software Center



pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: file cloning in pg_upgrade and CREATE DATABASE
Next
From: Michael Paquier
Date:
Subject: Re: file cloning in pg_upgrade and CREATE DATABASE