Re: adding status for COPY progress report - Mailing list pgsql-hackers

From Zhihong Yu
Subject Re: adding status for COPY progress report
Date
Msg-id CALNJ-vQeNybotsiTc_FhL36bhN=o0y+95SVEC_A19X4Z80Uv4Q@mail.gmail.com
Whole thread Raw
In response to Re: adding status for COPY progress report  (Matthias van de Meent <boekewurm+postgres@gmail.com>)
Responses Re: adding status for COPY progress report
List pgsql-hackers


On Wed, May 25, 2022 at 3:55 AM Matthias van de Meent <boekewurm+postgres@gmail.com> wrote:
On Wed, 25 May 2022 at 10:15, Zhihong Yu <zyu@yugabyte.com> wrote:
>
> Hi,
> Patch v3 follows advice from Matthias (status field has been dropped).

Could you argue why you think that this should be added to the
pg_stat_progress_copy view? Again, the progress reporting subsystem is
built to "report the progress of certain commands during command
execution". Why do you think we need to go further than that and allow
some commands to retain their report even after they've finished
executing?

Of note: The contents of >st_progress_param are only defined and
guaranteed to be consistent when the reporting command is running.
Even if no other progress-reporting command is running other commands
or processes in that backend may call functions that update the fields
with somewhat arbitrary values when no progress-reporting command is
actively running, thus corrupting the information for the progress
reporting view.

Could you please provide some insights on why you think that we should
change the progress reporting guts to accomodate something that it was
not built for?


Kind regards,

Matthias van de Meent
Hi, Matthias:
When I first followed the procedure in https://paquier.xyz/postgresql-2/postgres-14-monitoring-copy/ , I didn't see the output from the view.
This was because the example used 10 rows where the COPY command finishes quickly.
I had to increase the row count in order to see output from the system view.

With my patch, the user would be able to see the result of COPY command even if the duration for command execution is very short.

I made a slight change in patch v4. With patch v3, we would see the following:

   relid |  command  | status_yb | type | bytes_processed | bytes_total | tuples_processed | tuples_excluded
  -------+-----------+-----------+------+-----------------+-------------+------------------+-----------------
   -     | COPY FROM | PASS      | PIPE |               6 |           0 |                1 |               0

It would be desirable to see the relation for the COPY command.

With the updated patch, I think the interference from other commands in progress reporting has been prevented (see logic inside pg_stat_get_progress_info).

Cheers
Attachment

pgsql-hackers by date:

Previous
From: Japin Li
Date:
Subject: Invalid memory alloc request size for repeat()
Next
From: "David G. Johnston"
Date:
Subject: Re: Invalid memory alloc request size for repeat()