Re: Adding Support for Copy callback functionality on COPY TO api - Mailing list pgsql-hackers

From Bharath Rupireddy
Subject Re: Adding Support for Copy callback functionality on COPY TO api
Date
Msg-id CALj2ACXnEd6t1DQkaW7=QLyrvsYtd=+OeYMfxTwM+ZmV6EUC_A@mail.gmail.com
Whole thread Raw
In response to Re: Adding Support for Copy callback functionality on COPY TO api  (Nathan Bossart <nathandbossart@gmail.com>)
Responses Re: Adding Support for Copy callback functionality on COPY TO api  (Nathan Bossart <nathandbossart@gmail.com>)
List pgsql-hackers
On Sun, Oct 9, 2022 at 2:44 AM Nathan Bossart <nathandbossart@gmail.com> wrote:
>
> Sorry for the noise.  There was an extra #include in v4 that I've removed
> in v5.

IIUC, COPY TO callback helps move a table's data out of postgres
server. Just wondering, how is it different from existing solutions
like COPY TO ... PROGRAM/FILE, logical replication, pg_dump etc. that
can move a table's data out? I understandb that the COPY FROM callback
was needed for logical replication 7c4f52409. Mentioning a concrete
use-case helps here.

I'm not quite sure if we need a separate module to just tell how to
use this new callback. I strongly feel that it's not necessary. It
unnecessarily creates extra code (actual code is 25 LOC with v1 patch
but 150 LOC with v5 patch) and can cause maintenance burden. These
callback APIs are simple enough to understand for those who know
BeginCopyTo() or BeginCopyFrom() and especially for those who know how
to write extensions. These are not APIs that an end-user uses. The
best would be to document both COPY FROM and COPY TO callbacks,
perhaps with a pseudo code specifying just the essence [1], and their
possible usages somewhere here
https://www.postgresql.org/docs/devel/sql-copy.html.

The order of below NOTICE messages isn't guaranteed and it can change
depending on platforms. Previously, we've had to suppress such
messages in the test output 6adc5376d.

+SELECT test_copy_to_callback('public.test'::pg_catalog.regclass);
+NOTICE:  COPY TO callback called with data "1    2    3" and length 5
+NOTICE:  COPY TO callback called with data "12    34    56" and length 8
+NOTICE:  COPY TO callback called with data "123    456    789" and length 11
+ test_copy_to_callback

[1]
+    Relation    rel = table_open(PG_GETARG_OID(0), AccessShareLock);
+    CopyToState cstate;
+
+    cstate = BeginCopyTo(NULL, rel, NULL, RelationGetRelid(rel), NULL, NULL,
+                         to_cb, NIL, NIL);
+    (void) DoCopyTo(cstate);
+    EndCopyTo(cstate);
+
+    table_close(rel, AccessShareLock);

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com



pgsql-hackers by date:

Previous
From: "Drouvot, Bertrand"
Date:
Subject: Re: Patch proposal: make use of regular expressions for the username in pg_hba.conf
Next
From: Bharath Rupireddy
Date:
Subject: Re: ps command does not show walsender's connected db