Re: bogus: logical replication rows/cols combinations - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: bogus: logical replication rows/cols combinations
Date
Msg-id CAA4eK1KQimd=0gLS1DHTK2HrZA3kPdaY3sDY9Jko8o-nXRQw9A@mail.gmail.com
Whole thread Raw
In response to Re: bogus: logical replication rows/cols combinations  (Tomas Vondra <tomas.vondra@enterprisedb.com>)
Responses Re: bogus: logical replication rows/cols combinations  (Alvaro Herrera <alvherre@alvh.no-ip.org>)
List pgsql-hackers
On Mon, May 2, 2022 at 3:53 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
> On 5/2/22 12:17, Alvaro Herrera wrote:
> > On 2022-May-02, Tomas Vondra wrote:
> >> On 5/2/22 07:31, Amit Kapila wrote:
> >
> >>> Yeah, or don't allow to define such publications in the first place so
> >>> that different subscriptions can't combine them but I guess that might
> >>> forbid some useful cases as well where publication may not get
> >>> combined with other publications.
> >>
> >> But how would you check that? You don't know which publications will be
> >> combined by a subscription until you create the subscription, right?
> >

Yeah, I was thinking to check for all publications where the same
relation is published but as mentioned that may not be a very good
option as that would unnecessarily block many valid cases.

> > ... and I think this poses a problem: if the publisher has multiple
> > publications and the subscriber later uses those to create a combined
> > subscription, we can check at CREATE SUBSCRIPTION time that they can be
> > combined correctly.  But if the publisher decides to change the
> > publications changing the rules and they are no longer consistent, can
> > we throw an error at ALTER PUBLICATION point?  If the publisher can
> > detect that they are being used together by some subscription, then
> > maybe we can check consistency in the publication side and everything is
> > all right.  But I'm not sure that the publisher knows who is subscribed
> > to what, so this might not be an option.
> >
>
> AFAIK we don't track that (publication/subscription mapping). The
> publications are listed in publication_names parameter of the
> START_REPLICATION command.
>

We don't do that currently but we can as mentioned in my previous
email [1]. Let me write the relevant part again. We need to expose all
publications for a walsender, and then we can find the exact set of
publications where the current publication is used with other
publications and we can check only those publications. So, if we have
three walsenders (walsnd1: pub1, pub2; walsnd2 pub2; walsnd3: pub2,
pub3) in the system and we are currently altering publication pub1
then we need to check only pub3 for any conflicting conditions.

I think it is possible to expose a list of publications for each
walsender as it is stored in each walsenders
LogicalDecodingContext->output_plugin_private. AFAIK, each walsender
can have one such LogicalDecodingContext and we can probably share it
via shared memory?


[1] -
https://www.postgresql.org/message-id/CAA4eK1LGX-ig%3D%3DQyL%2B%3D%3DnKvcAS3qFU7%3DNiKL77ukUT-Q_4XncQ%40mail.gmail.com

-- 
With Regards,
Amit Kapila.



pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: bogus: logical replication rows/cols combinations
Next
From: Amit Kapila
Date:
Subject: Re: bogus: logical replication rows/cols combinations