Thread: Separate the result of \watch for each query execution (psql)

I need a way to separate the results of \watch for each query execution.

There is only a blank line between the results of \watch.
However, there is also a blank line after the title, which complicates
the rules.

My suggestion is to insert a "form feed(\f)" (preferably a newline)
before the result and output it.
Then, the output will be as follows.

    # select now(); \watch 1
    ^L                                          <--- add
    Thu Feb 17 11:52:05 2022 (every 1s)

    now
    -------------------------------
    2022-02-17 11:52:05.69394 + 09
    (1 row)

    ^L                                           <--- add
    Thu Feb 17 11:52:06 2022 (every 1s)

    now
    ------------------------------
    2022-02-17 11:52:06.96906 + 09
    (1 row)

(^L is usually not displayed. It is visualized by passing through a
filter such as `less`.)

This is possible with the following simple patch.

```
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index d65b9a124f..ee9442d0a6 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -646,10 +646,12 @@ PSQLexecWatch(const char *query, const
printQueryOpt *opt, FILE *printQueryFout)
  switch (PQresultStatus(res))
  {
  case PGRES_TUPLES_OK:
+ fprintf(fout, "\f\n");
  printQuery(res, opt, fout, false, pset.logfile);
  break;

  case PGRES_COMMAND_OK:
+ fprintf(fout, "\f\n");
  fprintf(fout, "%s\n%s\n\n", opt->title, PQcmdStatus(res));
  break;

```

I am developing a terminal pager ov (https://github.com/noborus/ov).
It's a generic pager, but it has features suitable for use with psql.

I found that \watch supports PAGER in PostgreSQL 15. That's great.

ov can be received and displayed, but I want to display it from the
beginning every time it is executed (like pspg).

The current output is a bit difficult to clear and display for each result.

Thank you for your reply.

2022年2月21日(月) 14:52 Pavel Stehule <pavel.stehule@gmail.com>:

>
> Hi
>
> po 21. 2. 2022 v 6:19 odesílatel Noboru Saito <noborusai@gmail.com> napsal:
>>
>> I need a way to separate the results of \watch for each query execution.
>>
>> There is only a blank line between the results of \watch.
>> However, there is also a blank line after the title, which complicates
>> the rules.
>>
>> My suggestion is to insert a "form feed(\f)" (preferably a newline)
>> before the result and output it.
>> Then, the output will be as follows.
>>
>>     # select now(); \watch 1
>>     ^L                                          <--- add
>>     Thu Feb 17 11:52:05 2022 (every 1s)
>>
>>     now
>>     -------------------------------
>>     2022-02-17 11:52:05.69394 + 09
>>     (1 row)
>>
>>     ^L                                           <--- add
>>     Thu Feb 17 11:52:06 2022 (every 1s)
>>
>>     now
>>     ------------------------------
>>     2022-02-17 11:52:06.96906 + 09
>>     (1 row)
>>
>> (^L is usually not displayed. It is visualized by passing through a
>> filter such as `less`.)
>>
>> This is possible with the following simple patch.
>>
>> ```
>> diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
>> index d65b9a124f..ee9442d0a6 100644
>> --- a/src/bin/psql/common.c
>> +++ b/src/bin/psql/common.c
>> @@ -646,10 +646,12 @@ PSQLexecWatch(const char *query, const
>> printQueryOpt *opt, FILE *printQueryFout)
>>   switch (PQresultStatus(res))
>>   {
>>   case PGRES_TUPLES_OK:
>> + fprintf(fout, "\f\n");
>>   printQuery(res, opt, fout, false, pset.logfile);
>>   break;
>>
>>   case PGRES_COMMAND_OK:
>> + fprintf(fout, "\f\n");
>>   fprintf(fout, "%s\n%s\n\n", opt->title, PQcmdStatus(res));
>>   break;
>>
>> ```
>>
>> I am developing a terminal pager ov (https://github.com/noborus/ov).
>> It's a generic pager, but it has features suitable for use with psql.
>>
>> I found that \watch supports PAGER in PostgreSQL 15. That's great.
>>
>> ov can be received and displayed, but I want to display it from the
>> beginning every time it is executed (like pspg).
>>
>> The current output is a bit difficult to clear and display for each result.
>
>
> I strongly agree. It was a lot of work to find a workable solution for pspg. Special chars that starting result and
maybeother, that ending result can significantly increase robustness and can reduce code. I think it can be better to
useform feed at the end of form - like it is semantic of form feed. You know, at this moment, the result is complete.
https://en.wikipedia.org/wiki/Page_break

It's easier to print a form feed before the result, but it's okay at the end.

> I don't think using it by default can be the best. Lot of people don't use specialized pagers, but it can be set by
\pset.Form feed should be used on end 
>
> \pset formfeed [on, off]

I think it's a good idea to be able to switch with \pset.

> Ascii has few nice characters, that we can use
>
> 1 .. SOX - start of header
> 2 .. STX - start of text
> 3 .. ETX - end of text
>
> Using it, it can reduce some heuristic in pspg, that is not fully 100% when border is not 2.
>
> But implementation of formfeed support can be a very good start. Any mark that can be used for synchronization can
helpa lot. 
>
> Regards
>
> Pavel

Thank you.

Hi,

Pavel Stehule <pavel.stehule@gmail.com>:
> > I strongly agree. It was a lot of work to find a workable solution for pspg. Special chars that starting result and
maybeother, that ending result can significantly increase robustness and can reduce code. I think it can be better to
useform feed at the end of form - like it is semantic of form feed. You know, at this moment, the result is complete.
https://en.wikipedia.org/wiki/Page_break
>
> It's easier to print a form feed before the result, but it's okay at the end.
>
> > I don't think using it by default can be the best. Lot of people don't use specialized pagers, but it can be set by
\pset.Form feed should be used on end 
> >
> > \pset formfeed [on, off]
>
> I think it's a good idea to be able to switch with \pset.

I have created a patch that allows you to turn it on and off in \pset.
The attached patch adds the following features.


Formfeed can be turned on with the command line option or \pset.
Formfeed (\f\n) is output after the query execution result by \watch.

I think the considerations are as follows.

* Is formfeed output after the result, not before?
* Is the formfeed output only "\f\n"?
* Is the formfeed output only executed by \watch?
* Is the name "formfeed" appropriate?

If the formfeed is output before the query result,
it will be better if the screen is reset when the formfeed is read.

Furthermore, if the terminal clear string can be set instead of the
formfeed character,
the \watch output can be fixedly displayed without a pager.

(I noticed that if I set it in the title, it would behave similarly in
the current version.
    # \C '\033[2J;\033[0;0H'
    # SELECT now();\watch 1
)

Also, it may be good to output formfeed when outputting a file with
`\o result.txt`
other than \watch.

Hi,

2022年2月25日(金) 13:42 Pavel Stehule <pavel.stehule@gmail.com>:
>
>
>
> pá 25. 2. 2022 v 5:23 odesílatel Noboru Saito <noborusai@gmail.com> napsal:
>>
>> Hi,
>>
>> Pavel Stehule <pavel.stehule@gmail.com>:
>> > > I strongly agree. It was a lot of work to find a workable solution for pspg. Special chars that starting result
andmaybe other, that ending result can significantly increase robustness and can reduce code. I think it can be better
touse form feed at the end of form - like it is semantic of form feed. You know, at this moment, the result is
complete.https://en.wikipedia.org/wiki/Page_break 
>> >
>> > It's easier to print a form feed before the result, but it's okay at the end.
>> >
>> > > I don't think using it by default can be the best. Lot of people don't use specialized pagers, but it can be set
by\pset. Form feed should be used on end 
>> > >
>> > > \pset formfeed [on, off]
>> >
>> > I think it's a good idea to be able to switch with \pset.
>>
>> I have created a patch that allows you to turn it on and off in \pset.
>> The attached patch adds the following features.
>>
>>
>> Formfeed can be turned on with the command line option or \pset.
>> Formfeed (\f\n) is output after the query execution result by \watch.
>>
>> I think the considerations are as follows.
>>
>> * Is formfeed output after the result, not before?
>
>
> We are talking about the first iteration. In the second and other iteration this question has no sense. You know the
startingpoint. You don't know the endpoint. So I think so using formfeed on the end is good idea. 

Yes. However, as you know, in the case of \ watch, there is a
difference in waiting before and after the specified time.

>> * Is the formfeed output only "\f\n"?
>
>
> yes
>
>>
>> * Is the formfeed output only executed by \watch?
>
>
> This is a good question. I think the implementation for \watch is a good start. But it can help with normal results
too.In pspg it can be very useful for incremental load or for streaming mode. But if it will be used everywhere, then
itshould be used just for some specified pagers. 
>
>
>>
>> * Is the name "formfeed" appropriate?
>
>
> If it will do work of formfeed, then the formfeed is good name.
>
>>
>>
>> If the formfeed is output before the query result,
>> it will be better if the screen is reset when the formfeed is read.
>
>
> I think, so you propose another feature - reset terminal sequence - it can be a good idea. Not for pspg, but
generally,why not. 

Yes, it is. I was wondering if I could just add formfeed or add more features.

Thank you.
Following your advice, I will register this for the commitfest
 (I am registering for the commitfest for the first time. Thank you
for your cooperation).

Noboru Saito <noborusai@gmail.com> writes:
> I have created a patch that allows you to turn it on and off in \pset.
> The attached patch adds the following features.
> Formfeed can be turned on with the command line option or \pset.
> Formfeed (\f\n) is output after the query execution result by \watch.

Hmm ... I grant your use-case for this, but I think the patch
is too narrow-minded, because it supposes that the only string
anybody could wish to output between \watch commands is "\f\n".
Once you open the floodgates of inserting formatting there,
ISTM that people might want other things.

Also, I'm not that thrilled with treating this as a \pset option,
because it has nothing to do with formatting of normal query
results.  (IMV anyway, perhaps others will disagree.)

How about instead of defining fixed semantics, we invent a psql
special variable that can contain a string to be output between
\watch commands?  It looks like you could then set it through
a command like

\set WATCH_SEPARATOR '\f\n'

(not wedded to that variable name, it's just the first idea
that came to mind)

Personally I'd not bother with inventing a specialized command-line
option to set it, either.  There's already -v and friends.

> * Is formfeed output after the result, not before?

Or we could invent WATCH_BEFORE and WATCH_AFTER ...

            regards, tom lane

Tom Lane <tgl@sss.pgh.pa.us>:
> Noboru Saito <noborusai@gmail.com> writes:
> > I have created a patch that allows you to turn it on and off in \pset.
> > The attached patch adds the following features.
> > Formfeed can be turned on with the command line option or \pset.
> > Formfeed (\f\n) is output after the query execution result by \watch.
>
> Hmm ... I grant your use-case for this, but I think the patch
> is too narrow-minded, because it supposes that the only string
> anybody could wish to output between \watch commands is "\f\n".
> Once you open the floodgates of inserting formatting there,
> ISTM that people might want other things.
>
> Also, I'm not that thrilled with treating this as a \pset option,
> because it has nothing to do with formatting of normal query
> results.  (IMV anyway, perhaps others will disagree.)
>
> How about instead of defining fixed semantics, we invent a psql
> special variable that can contain a string to be output between
> \watch commands?  It looks like you could then set it through
> a command like

I understand because it's a problem I was worried about.
However, we need a machine-readable string for PAGER.
It didn't exist before, so newcomers will match it to a fixed string.
The de facto standard is required even if it can be changed.

> \set WATCH_SEPARATOR '\f\n'
>
> (not wedded to that variable name, it's just the first idea
> that came to mind)

I think it's better not to include "WATCH" in the name,
as it may be used in the future other than \watch.

> Personally I'd not bother with inventing a specialized command-line
> option to set it, either.  There's already -v and friends.
>
> > * Is formfeed output after the result, not before?
>
> Or we could invent WATCH_BEFORE and WATCH_AFTER ...
>
>                         regards, tom lane

It may be good to add it in the future.
For now, I'm fine with either one.

Thank you.

Hi,

On 2022-02-25 13:23:31 +0900, Noboru Saito wrote:
> I have created a patch that allows you to turn it on and off in \pset.

The patch unfortunately causes tests to fail:
https://cirrus-ci.com/task/5932406812180480
diff -U3 /tmp/cirrus-ci-build/src/test/regress/expected/psql.out
/tmp/cirrus-ci-build/src/test/regress/results/psql.out
--- /tmp/cirrus-ci-build/src/test/regress/expected/psql.out    2022-03-21 09:55:55.875784000 +0000
+++ /tmp/cirrus-ci-build/src/test/regress/results/psql.out    2022-03-21 09:59:51.950512000 +0000
@@ -304,6 +304,7 @@
 fieldsep_zero            off
 footer                   on
 format                   aligned
+formfeed                 off
 linestyle                ascii
 null                     ''
 numericlocale            off

Greetings,

Andres Freund

On Mon, Mar 21, 2022 at 9:15 PM Andres Freund <andres@anarazel.de> wrote:
> On 2022-02-25 13:23:31 +0900, Noboru Saito wrote:
> > I have created a patch that allows you to turn it on and off in \pset.
>
> The patch unfortunately causes tests to fail:

It doesn't seem like the originally proposed design here will be
accepted. Perhaps a patch implementing what Tom proposed or some
variant of it will get accepted, but the only patch we have is now
more than 1 month old and has not been updated. I'm accordingly
marking this CommitFest entry "Returned with Feedback." If Noboru
Saito decides to update the patch with some new design, he can change
the status and move the patch forward to a future CommitFest.

-- 
Robert Haas
EDB: http://www.enterprisedb.com