Re: File content logging during execution of COPY queries (was: Better logging of COPY queries if log_statement='all') - Mailing list pgsql-hackers

From Stephen Frost
Subject Re: File content logging during execution of COPY queries (was: Better logging of COPY queries if log_statement='all')
Date
Msg-id 20161020125004.GL13284@tamriel.snowman.net
Whole thread Raw
In response to File content logging during execution of COPY queries (was: Better logging of COPY queries if log_statement='all')  (Aleksander Alekseev <a.alekseev@postgrespro.ru>)
List pgsql-hackers
Aleksander,

* Aleksander Alekseev (a.alekseev@postgrespro.ru) wrote:
> > The idea is to record application workload in real environment and write
> > a benchmark based on this record. Then using this benchmark we could try
> > different OS/DBMS configuration (or maybe hardware), find an extremum,
> > then change configuration in production environment.
> >
> > It's not always possible to change an application or even database (e.g.
> > to use triggers) for this purpose. For instance, if DBMS is provided as
> > a service.
> >
> > Currently PostgreSQL allows to record all workload _except_ COPY
> > queries. Considering how easily it could be done I think it's wrong.
> > Basically the only real question here is how it should look like in
> > postgresql.conf.
>
> OK, how about introducing a new boolean parameter named log_copy?
> Corresponding patch is attached.

The parameter would be better as 'log_copy_data', I believe.  The actual
COPY command is already logged with just 'log_statement = all', of
course.

Also..

> diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
> index 8c25b45..84a7542 100644
> --- a/doc/src/sgml/config.sgml
> +++ b/doc/src/sgml/config.sgml
> @@ -5205,6 +5205,20 @@ FROM pg_stat_activity;
>        </listitem>
>       </varlistentry>
>
> +     <varlistentry id="guc-log-copy" xreflabel="log_copy">
> +      <term><varname>log_copy</varname> (<type>boolean</type>)
> +      <indexterm>
> +       <primary><varname>log_copy</> configuration parameter</primary>
> +      </indexterm>
> +      </term>
> +      <listitem>
> +       <para>
> +        Controls whether file content is logged during execution of
> +        COPY queries.  The default is <literal>off</>.
> +       </para>
> +      </listitem>
> +     </varlistentry>

"file" isn't accurate here and I don't know that it actually makes sense
to log "COPY TO" data- we don't log the results of SELECT statements,
after all, and the use-case you outlined above (which I generally agree
is one we should consider) doesn't have any need for the data of "COPY
TO" statements to be in the log, it seems to me.

Can you elaborate on why we would want to log the data sent to the
client with a COPY TO command.  If there is a reason, why wouldn't we
want to support that for SELECT and ... RETURNING commands too?

Thanks!

Stephen

pgsql-hackers by date:

Previous
From: "Constantin S. Pan"
Date:
Subject: Re: Fun fact about autovacuum and orphan temp tables
Next
From: "Joshua D. Drake"
Date:
Subject: Remove autovacuum GUC?