Re: Add on_error and log_verbosity options to file_fdw - Mailing list pgsql-hackers

From Michael Paquier
Subject Re: Add on_error and log_verbosity options to file_fdw
Date
Msg-id Zp7x8Y-SpbkZtpE-@paquier.xyz
Whole thread Raw
In response to Re: Add on_error and log_verbosity options to file_fdw  (Masahiko Sawada <sawada.mshk@gmail.com>)
Responses Re: Add on_error and log_verbosity options to file_fdw
List pgsql-hackers
On Mon, Jul 22, 2024 at 03:07:46PM -0700, Masahiko Sawada wrote:
> I'm slightly concerned that users might not want to see the NOTICE
> message for every scan. Unlike COPY FROM, scanning a file via file_fdw
> could be frequent. An alternative idea of place to write the
> information of the number of malformed rows would be the EXPLAIN
> command as follow:

Yeah, I also have some concerns regarding the noise that this could
produce if called on a foreign table on a regular basis.  The verbose
mode is disabled by default so I don't see why we should not allow it
if the relation owner wants to show it.

Perhaps we should first do a silence mode for log_verbosity to skip
the NOTICE produced at the end of the COPY FROM summarizing the whole?
It would be confusing to have different defaults between COPY and
file_fdw, but having the option to silence that completely is also
appealing from the user point of view.

>                            QUERY PLAN
> ----------------------------------------------------------------
>  Foreign Scan on public.test  (cost=0.00..1.10 rows=1 width=12)
>    Output: a, b, c
>    Foreign File: test.csv
>    Foreign File Size: 12 b
>    Skipped Rows: 10

Interesting idea linked to the idea of pushing the error state to
something else than the logs.  Sounds like a separate feature.

> IIUC we have to execute ALTER FOREIGN TABLE to change the
> log_verbosity value and which requires to be the owner. Which seems
> not to be user-friendly.

I am not sure about allowing scans to force an option to be a
different thing at runtime vs what's been defined in the relation
itself with CREATE/ALTER.
--
Michael

Attachment

pgsql-hackers by date:

Previous
From: Peter Smith
Date:
Subject: Re: Pgoutput not capturing the generated columns
Next
From: Michael Paquier
Date:
Subject: Re: Add privileges test for pg_stat_statements to improve coverage