Thread: Proposed patch - psql wraps at window width
I have moved this discussion to hackers in hopes of getting more
feedback, and moved the patch to a static URL:
ftp://momjian.us/pub/postgresql/mypatches/wrap
This patch adds a new '\pset format wrapped' mode that wraps long values
to fit the table on the user's screen, or in '\pset columns' columns in
an output to file or pipe. The documentation additions are at the top
of the patch.
Sample:
\pset format wrappedOutput format is wrapped.
\pset columns 70Target column width for "wrap" format is 70.
SELECT 1, 2, repeat('a', 80), repeat('b', 80), E'a\nb\nc', 1FROM generate_series(1,2)\g ?column? | ?column? | repeat
| repeat | ?column? | ?column? ----------+----------+------------+-------------+----------+---------- 1 |
2 | aaaaaaaaaa | bbbbbbbbbbb | a | 1 ; aaaaaaaaaa ; bbbbbbbbbbb : b
; aaaaaaaaaa ; bbbbbbbbbbb : c ; aaaaaaaaaa ; bbbbbbbbbbb
; aaaaaaaaaa ; bbbbbbbbbbb ;
aaaaaaaaaa; bbbbbbbbbbb ; aaaaaaaaaa ; bbbbbbbbbbb
; aaaaaaaaaa ; bbb 1 | 2 | aaaaaaaaaa | bbbbbbbbbbb | a
| 1 ; aaaaaaaaaa ; bbbbbbbbbbb : b ; aaaaaaaaaa ;
bbbbbbbbbbb: c ; aaaaaaaaaa ; bbbbbbbbbbb
; aaaaaaaaaa ; bbbbbbbbbbb ; aaaaaaaaaa ; bbbbbbbbbbb
; aaaaaaaaaa ; bbbbbbbbbbb ; aaaaaaaaaa ; bbb
(2 rows)
You will notice:
o I have pulled up newline values to appear in the same rows as the wrapped texto Colons are used on the left for
newlinevalueso Semicolons are used on the left for wrapped valueso There are no vertical bars for values that don't
extend to the wrapped or newline rows. This is how our newline display has always worked so it was copied by the
wrappedcodeo The left column has no indicator of wrapping or newlines because there is no left border
We could use dashes to indicated wrapped values, but we don't. It would
be nice if someone could do some multi-byte testing of this,
particularly for characters that have a display width greater than one.
I think this patch is ready for application.
Should 'wrapped' be the default for certain operations, like \df?
'wrapped' mode is really good for a table that would fit the screen
width except for a few wide values.
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Thu, Apr 24, 2008 at 8:30 AM, Bruce Momjianwrote:
> I have moved this discussion to hackers in hopes of getting more
> feedback, and moved the patch to a static URL:
>
Hi Bruce,
This is a very cool feature! Looking through the patch I did have a
few thoughts.
This is definitely going to introduce merge conflicts with my
printTable API patch. That's not a problem, just a "note to self"
that when/if this patch goes in I'll have to submit a fresh version of
my patch.
This psql message seemed a bit strange:
+ if (!quiet)
+ printf(_("Target column width for \"wrap\" format is %d.\n"),
popt->topt.columns);
To me, this message sounds like you're setting the width of a single
column, when in fact you're setting the target *total* width of the
table. I think this message would be more clear if it read "Target
output width ..." or "Target table width ...". Also, as far as the
user is concerned the format is referred to as "wrapped", not "wrap".
Cheers,
BJ
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: http://getfiregpg.org
iD8DBQFID72J5YBsbHkuyV0RAr45AKDETJTapg6c836Cr1RS8uk3gUUH0ACdH8mQ
M8ikE3VO+3H2/rt8AvhVoew=
=Jfnk
-----END PGP SIGNATURE-----
Brendan Jurd wrote:
> This is a very cool feature! Looking through the patch I did have a
> few thoughts.
>
> This is definitely going to introduce merge conflicts with my
> printTable API patch. That's not a problem, just a "note to self"
> that when/if this patch goes in I'll have to submit a fresh version of
> my patch.
>
> This psql message seemed a bit strange:
>
> + if (!quiet)
> + printf(_("Target column width for \"wrap\" format is %d.\n"),
> popt->topt.columns);
>
> To me, this message sounds like you're setting the width of a single
> column, when in fact you're setting the target *total* width of the
> table. I think this message would be more clear if it read "Target
> output width ..." or "Target table width ...". Also, as far as the
> user is concerned the format is referred to as "wrapped", not "wrap".
Good point. I have updated the text to be:
test=> \pset columns 70Target width of file and pipe output for "wrap" format is 70.
Patch updated at the same URL.
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
"Bruce Momjian" <bruce@momjian.us> writes: > Brendan Jurd wrote: >> To me, this message sounds like you're setting the width of a single >> column, when in fact you're setting the target *total* width of the >> table. I think this message would be more clear if it read "Target >> output width ..." or "Target table width ...". Also, as far as the >> user is concerned the format is referred to as "wrapped", not "wrap". > > Good point. I have updated the text to be: > > test=> \pset columns 70 > Target width of file and pipe output for "wrap" format is 70. I think "file and pipe output" is short-sighted. There are lots more cases this is necessary including SSH sessions and emacs shell buffers, etc. And as I pointed out there are often cases where the user may want to override the terminal width in any case. Earlier I suggested -- and nobody refuted -- that we should follow the precedents of ls and man and other tools which need to find the terminal width: Explicitly set width takes precedence always, if it's not explicitly set then you use the ioctl, and if that fails then you use the COLUMNS environment variable. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's 24x7 Postgres support!
Gregory Stark wrote: > "Bruce Momjian" <bruce@momjian.us> writes: > > > Brendan Jurd wrote: > >> To me, this message sounds like you're setting the width of a single > >> column, when in fact you're setting the target *total* width of the > >> table. I think this message would be more clear if it read "Target > >> output width ..." or "Target table width ...". Also, as far as the > >> user is concerned the format is referred to as "wrapped", not "wrap". > > > > Good point. I have updated the text to be: > > > > test=> \pset columns 70 > > Target width of file and pipe output for "wrap" format is 70. > > I think "file and pipe output" is short-sighted. There are lots more cases > this is necessary including SSH sessions and emacs shell buffers, etc. And as > I pointed out there are often cases where the user may want to override the > terminal width in any case. > > Earlier I suggested -- and nobody refuted -- that we should follow the > precedents of ls and man and other tools which need to find the terminal > width: Explicitly set width takes precedence always, if it's not explicitly > set then you use the ioctl, and if that fails then you use the COLUMNS > environment variable. Yes, I like that better. Patch updated, same URL: ftp://momjian.us/pub/postgresql/mypatches/wrap -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
"Bruce Momjian" <bruce@momjian.us> writes:
> Gregory Stark wrote:
>
>> Earlier I suggested -- and nobody refuted -- that we should follow the
>> precedents of ls and man and other tools which need to find the terminal
>> width: Explicitly set width takes precedence always, if it's not explicitly
>> set then you use the ioctl, and if that fails then you use the COLUMNS
>> environment variable.
>
> Yes, I like that better. Patch updated, same URL:
>
> ftp://momjian.us/pub/postgresql/mypatches/wrap
I think it should just be:
if (opt->format == PRINT_WRAP){ /* Get terminal width -- explicit setting takes precedence */ output_columns =
opt->columns;
#ifdef TIOCGWINSZ if (output_columns == 0 && isatty(fout)) { struct winsize screen_size;
if (ioctl(fileno(fout), TIOCGWINSZ, &screen_size) != -1) output_columns = screen_size.ws_col; }
#endif
if (output_columns == 0) { const char *columns_env = getenv("COLUMNS");
if (columns_env) output_columns = atoi(columns_env); }
if (output_columns == 0) output_columns = 79;}
The differences this makes are that:
a) if you do -o /dev/tty (perhaps on some kind of cronjob) it will use the ioctl.
b) If you dump to a file it will still respect COLUMNS. This might be a bit weird since bash sets COLUMNS so your file
widthwill be based on the size of your terminal. But people also do things like COLUMNS=120 psql -o f ...
-- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's 24x7 Postgres support!
Gregory Stark wrote:
> "Bruce Momjian" <bruce@momjian.us> writes:
> > Gregory Stark wrote:
> >> Earlier I suggested -- and nobody refuted -- that we should follow the
> >> precedents of ls and man and other tools which need to find the terminal
> >> width: Explicitly set width takes precedence always, if it's not explicitly
> >> set then you use the ioctl, and if that fails then you use the COLUMNS
> >> environment variable.
> >
> > Yes, I like that better. Patch updated, same URL:
> >
> > ftp://momjian.us/pub/postgresql/mypatches/wrap
>
> I think it should just be:
>
> if (opt->format == PRINT_WRAP)
> {
> /* Get terminal width -- explicit setting takes precedence */
> output_columns = opt->columns;
>
> #ifdef TIOCGWINSZ
> if (output_columns == 0 && isatty(fout))
> {
> struct winsize screen_size;
>
> if (ioctl(fileno(fout), TIOCGWINSZ, &screen_size) != -1)
> output_columns = screen_size.ws_col;
> }
> #endif
>
> if (output_columns == 0)
> {
> const char *columns_env = getenv("COLUMNS");
>
> if (columns_env)
> output_columns = atoi(columns_env);
> }
>
> if (output_columns == 0)
> output_columns = 79;
> }
>
>
> The differences this makes are that:
>
> a) if you do -o /dev/tty (perhaps on some kind of cronjob) it will use the
> ioctl.
Uh, if you do that I am not sure what the user would want. I duplicated
what we do with PAGER and unless there is a clear mandate I think we
should keep the wrapping detection consistent with that; we have gotten
no complaints. Pager will not work for -o /dev/tty either.
> b) If you dump to a file it will still respect COLUMNS. This might be a bit
> weird since bash sets COLUMNS so your file width will be based on the size
> of your terminal. But people also do things like COLUMNS=120 psql -o f ...
No, that will make the regression tests fail and it is hard to say why
you would want a file wrap width to match your screen width.
Your final change is to assume a width of 79 if no columns are detected.
I also don't think that is a good idea, and if we want to do that we
would need to discuss that too.
I don't want to over-design this.
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Bruce Momjian escribió: > > b) If you dump to a file it will still respect COLUMNS. This might be a bit > > weird since bash sets COLUMNS so your file width will be based on the size > > of your terminal. But people also do things like COLUMNS=120 psql -o f ... > > No, that will make the regression tests fail and it is hard to say why > you would want a file wrap width to match your screen width. What this means is that the regression tests should not use the wrapped mode. -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Am Donnerstag, 24. April 2008 schrieb Gregory Stark: > b) If you dump to a file it will still respect COLUMNS. This might be a bit > weird since bash sets COLUMNS so your file width will be based on the > size of your terminal. But people also do things like COLUMNS=120 psql -o f > ... Well, the feature is labeled "psql wraps at window width". If the output isn't on a window, then it shouldn't wrap. I certainly don't want the output to a file to depend on the size of the window at the time. Perhaps having a variable inside psql to control this explicitly could be useful for the case you describe.
Peter Eisentraut wrote: > Am Donnerstag, 24. April 2008 schrieb Gregory Stark: > > b) If you dump to a file it will still respect COLUMNS. This might be a bit > > ? ?weird since bash sets COLUMNS so your file width will be based on the > > size of your terminal. But people also do things like COLUMNS=120 psql -o f > > ... > > Well, the feature is labeled "psql wraps at window width". If the output > isn't on a window, then it shouldn't wrap. I certainly don't want the output > to a file to depend on the size of the window at the time. > > Perhaps having a variable inside psql to control this explicitly could be > useful for the case you describe. \pset columns will wrap to the specified width for file output. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
"Bruce Momjian" <bruce@momjian.us> writes: > Uh, if you do that I am not sure what the user would want. I duplicated > what we do with PAGER and unless there is a clear mandate I think we > should keep the wrapping detection consistent with that; we have gotten > no complaints. Pager will not work for -o /dev/tty either. I've explained before why I think the pager case is not analogous. In any case a pager *can't* work if you do -o /dev/tty. >> b) If you dump to a file it will still respect COLUMNS. This might be a bit >> weird since bash sets COLUMNS so your file width will be based on the size >> of your terminal. But people also do things like COLUMNS=120 psql -o f ... > > No, that will make the regression tests fail and it is hard to say why > you would want a file wrap width to match your screen width. Well you don't know where the pipe is going, it could be, for example, piped to a pager. I think the point is to keep the logic simple and not put in special cases based on assumptions of what users might do. If you have simple logic which doesn't do what the user expects in a corner case they understand and we can tell them to override it with \pset. If you have logic which does what they want normally but does something different sometimes based on criteria which they consider irrelevant then they get angry. > Your final change is to assume a width of 79 if no columns are detected. > I also don't think that is a good idea, and if we want to do that we > would need to discuss that too. Well what width would you use if you have no better info? > I don't want to over-design this. I think your design is more complicated than mine. I've *removed* some of the special cases from your logic. Mine is very straightforward: explicit user setting takes precedence always, otherwise we try to use the ioctl and if that fails fall back to COLUMNS. Yours is "explicit setting takes precedence otherwise if you're on a terminal and haven't redirected the output then ( we try the terminal if that fails then we fall back to COLUMNS ) otherwise we ignore the ioctl and COLUMNS and uh, I don't know what happens. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's Slony Replication support!
Bruce Momjian <bruce@momjian.us> writes:
> Peter Eisentraut wrote:
>> Well, the feature is labeled "psql wraps at window width". If the output
>> isn't on a window, then it shouldn't wrap.
> \pset columns will wrap to the specified width for file output.
I agree with Peter: that's a seriously bad idea.
regards, tom lane
Gregory Stark wrote: > >> b) If you dump to a file it will still respect COLUMNS. This might be a bit > >> weird since bash sets COLUMNS so your file width will be based on the size > >> of your terminal. But people also do things like COLUMNS=120 psql -o f ... > > > > No, that will make the regression tests fail and it is hard to say why > > you would want a file wrap width to match your screen width. > > Well you don't know where the pipe is going, it could be, for example, piped > to a pager. > > I think the point is to keep the logic simple and not put in special cases > based on assumptions of what users might do. If you have simple logic which > doesn't do what the user expects in a corner case they understand and we can > tell them to override it with \pset. If you have logic which does what they > want normally but does something different sometimes based on criteria which > they consider irrelevant then they get angry. They can always do: test=> \pset columns `echo $COLUMNS`Target width for "wrap" format is 127. My point is that we should do what most people _expect_, and the majority of people here have stated they don't think wrap should modify the file output _by_ _default_. People who want a specific width for files should be setting their desired width themselves, hence no need for the '79' default. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > Peter Eisentraut wrote: > >> Well, the feature is labeled "psql wraps at window width". If the output > >> isn't on a window, then it shouldn't wrap. > > > \pset columns will wrap to the specified width for file output. > > I agree with Peter: that's a seriously bad idea. Uh, I am confused. Are you saying \pset columns should not control file output? And if it doesn't how does someone get file output in a specified width? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > Tom Lane wrote: > > Bruce Momjian <bruce@momjian.us> writes: > > > Peter Eisentraut wrote: > > >> Well, the feature is labeled "psql wraps at window width". If the output > > >> isn't on a window, then it shouldn't wrap. > > > > > \pset columns will wrap to the specified width for file output. > > > > I agree with Peter: that's a seriously bad idea. > > Uh, I am confused. Are you saying \pset columns should not control file > output? And if it doesn't how does someone get file output in a > specified width? For reference Peter's comment is here: http://archives.postgresql.org/pgsql-hackers/2008-04/msg01633.php and the patch is here: ftp://momjian.us/pub/postgresql/mypatches/wrap with documentation. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
"Bruce Momjian" <bruce@momjian.us> writes: > My point is that we should do what most people _expect_, and the > majority of people here have stated they don't think wrap should modify > the file output _by_ _default_. > > People who want a specific width for files should be setting their > desired width themselves, hence no need for the '79' default. Consider that this means that psql -Pformat=wrapped -e 'select * from tab' and psql -Pformat=wrapped -e 'select * from tab' | more will generate different output. That's bunk. This is what I mean about trying to guess about what users expect. If you come up with complicated logic trying to anticipate every case and get it wrong users get angry. If you just have simple rules which always apply then users understand they they need to override them for their corner cases. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Get trained by Bruce Momjian - ask me about EnterpriseDB'sPostgreSQL training!
Gregory Stark wrote:
> "Bruce Momjian" <bruce@momjian.us> writes:
>
> > My point is that we should do what most people _expect_, and the
> > majority of people here have stated they don't think wrap should modify
> > the file output _by_ _default_.
> >
> > People who want a specific width for files should be setting their
> > desired width themselves, hence no need for the '79' default.
>
> Consider that this means that
>
> psql -Pformat=wrapped -e 'select * from tab'
>
> and
>
> psql -Pformat=wrapped -e 'select * from tab' | more
>
> will generate different output. That's bunk.
'ls' and 'ls | more' generate different outputs, and I have never heard
anyone call that "bunk".
Also, this does work:
psql -Pformat=wrapped -Pcolumns=70 -c "select repeat('a', 100)" test | more
If you want non-terminal output to wrap, you have to specify the width
--- that seems only reasonable because the file case really needs to
have the width specified.
> This is what I mean about trying to guess about what users expect. If you come
> up with complicated logic trying to anticipate every case and get it wrong
> users get angry.
>
> If you just have simple rules which always apply then users understand they
> they need to override them for their corner cases.
You are going to need to find community members who support your
analysis if you want to make any headway in changing the patch.
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Bruce Momjian <bruce@momjian.us> writes:
> You are going to need to find community members who support your
> analysis if you want to make any headway in changing the patch.
Let's turn that around, shall we? I think at this point it's *you*
that are standing alone and need to find someone who agrees with
your approach.
regards, tom lane
Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > You are going to need to find community members who support your > > analysis if you want to make any headway in changing the patch. > > Let's turn that around, shall we? I think at this point it's *you* > that are standing alone and need to find someone who agrees with > your approach. I am confused exactly what people want changed in the patch. Some want no control over wrapping in file output, and others want $COLUMN to control column file output. The only person I am seeing code from is Greg Stark, but I think most don't want his changes. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
bruce wrote: > Tom Lane wrote: > > Bruce Momjian <bruce@momjian.us> writes: > > > You are going to need to find community members who support your > > > analysis if you want to make any headway in changing the patch. > > > > Let's turn that around, shall we? I think at this point it's *you* > > that are standing alone and need to find someone who agrees with > > your approach. > > I am confused exactly what people want changed in the patch. Some want > no control over wrapping in file output, and others want $COLUMN to > control column file output. The only person I am seeing code from is > Greg Stark, but I think most don't want his changes. Here is an email showing two people who want some way of getting wrapped output into a file: http://archives.postgresql.org/pgsql-patches/2008-04/msg00344.php I think the API in the patch is the best I am going to do to keep everyone happy --- 'wrapped' doesn't affect file/pipe output unless you also tell it the width you want. Most interactive users are going to set 'wrapped' and never set the width so it is automatically determined. Some want wrapped to not affect file/pipe at all, while others (Greg only?) want $COLUMNS to affect file/pipe output. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
* Bruce Momjian <bruce@momjian.us> [080424 14:37]: > I am confused exactly what people want changed in the patch. Some want > no control over wrapping in file output, and others want $COLUMN to > control column file output. The only person I am seeing code from is > Greg Stark, but I think most don't want his changes. If I had to vote as as user, Greg's suggestion made the most sense to me. It was a simple approach that I could easily understand, and easily envision using with and without a pager. But basically, I pretty much *always* use a pager, having settled on comfortable $LESS. That said, I really don't see myself using the wrapped format so maybe my vote shouldn't count. a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
"Bruce Momjian" <bruce@momjian.us> writes:
> 'ls' and 'ls | more' generate different outputs, and I have never heard
> anyone call that "bunk".
The analogue of that would be making psql default to wrapped mode if isatty is
true and normal mode if it's false. I wouldn't be entirely against that but I
don't really care much either way.
Note that there's still -C and -1 to override that default. And if you specify
-C or isatty returns true it *always* uses the same deterministic logic to
determine the width: -w first, then ioctl, then COLUMNS.
> Also, this does work:
>
> psql -Pformat=wrapped -Pcolumns=70 -c "select repeat('a', 100)" test | more
>
> If you want non-terminal output to wrap, you have to specify the width
> --- that seems only reasonable because the file case really needs to
> have the width specified.
No it's not reasonable. I promise you users will report this as a bug.
This isn't anything new. Offhand I could only think of two precedents, ls and
man, but I'm sure there are others. They both use the same basic logic. And
not just GNU, FreeBSD and Solaris document the same behaviour.
I'm puzzled what you think should happen for the above. You think it should
just ignore the user's -Pformat=wrapped ?
-- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's RemoteDBA services!
Aidan Van Dyk wrote: -- Start of PGP signed section. > * Bruce Momjian <bruce@momjian.us> [080424 14:37]: > > > I am confused exactly what people want changed in the patch. Some want > > no control over wrapping in file output, and others want $COLUMN to > > control column file output. The only person I am seeing code from is > > Greg Stark, but I think most don't want his changes. > > If I had to vote as as user, Greg's suggestion made the most sense to > me. It was a simple approach that I could easily understand, and easily > envision using with and without a pager. > > But basically, I pretty much *always* use a pager, having settled on > comfortable $LESS. > > That said, I really don't see myself using the wrapped format so maybe > my vote shouldn't count. Greg's suggestion is to use $COLUMNS if the width can't be determined because the output is not directly to the screen. $COLUMNS is updated by many shells. The arguments I have heard are: o wrapped never affects file/pipe outputo wrapped to file/pipe controlled only by \pset columnso wrapped to file/pipecontrolled by $COLUMNSo wrapped to file/pipe controlled by \pset columns or $COLUMNS -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Gregory Stark wrote:
> "Bruce Momjian" <bruce@momjian.us> writes:
>
> > 'ls' and 'ls | more' generate different outputs, and I have never heard
> > anyone call that "bunk".
>
> The analogue of that would be making psql default to wrapped mode if isatty is
> true and normal mode if it's false. I wouldn't be entirely against that but I
> don't really care much either way.
Yea, we have to discuss any default changes once we are done.
> Note that there's still -C and -1 to override that default. And if you specify
> -C or isatty returns true it *always* uses the same deterministic logic to
> determine the width: -w first, then ioctl, then COLUMNS.
Interesting. That is a powerful argument. I see if I do:
ls -C > /tmp/x
/tmp/x is wrapped, but if I make the window wider (on Ubuntu), the file
ouput is not wider. It seems to default to 72 columns as a target, even
though $COLUMNS is updated. This seems to indicate that 'ls' doesn't
check the terminal width at all when doing output.
What that would translate into for psql is that the interactive behavior
is as posted in the patch (try ioctl, then try $COLUMNS), but for
file/pipe, wrap is to 72 but can be overridden with \pset columns.
> > Also, this does work:
> >
> > psql -Pformat=wrapped -Pcolumns=70 -c "select repeat('a', 100)" test | more
> >
> > If you want non-terminal output to wrap, you have to specify the width
> > --- that seems only reasonable because the file case really needs to
> > have the width specified.
>
> No it's not reasonable. I promise you users will report this as a bug.
>
> This isn't anything new. Offhand I could only think of two precedents, ls and
> man, but I'm sure there are others. They both use the same basic logic. And
> not just GNU, FreeBSD and Solaris document the same behaviour.
>
> I'm puzzled what you think should happen for the above. You think it should
> just ignore the user's -Pformat=wrapped ?
Well, I have two people who don't want wrap to ever affect file/pipe
output, and now have two who want wrapped to try to affect file/pipe,
even if it has to read $COLUMNS. Obviously someone is going to be
unhappy in the end.
The only distinction I can think of is that 'ls' uses per-command flags,
while psql \pset format is for all commands, but you are kind of right
it is like 'ls'.
I can modify the patch for further review. I need feedback.
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
"Bruce Momjian" <bruce@momjian.us> writes: > I think the API in the patch is the best I am going to do to keep > everyone happy --- 'wrapped' doesn't affect file/pipe output unless you > also tell it the width you want. Most interactive users are going to > set 'wrapped' and never set the width so it is automatically determined. Aaah, I think that's the key to where we're going wrong. Trying to use the columns logic to encode two separate decisions. "wrapped or not wrapped" and "how wide". I think this also clarifies Tom's objection. He's worried about people configuring psql for interactive use and being surprised when their automated scripts fail to parse the resulting output. We do need a way to specifically request wrapped format, but if we want a way to say "wrapped format only on a terminal" then we should have a mode for that too. But once we're in wrapped format we should stick to it and always follow the same logic to determine the width. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's Slony Replication support!
Bruce Momjian wrote: > 'ls' and 'ls | more' generate different outputs, and I have never heard > anyone call that "bunk". bunk
Bruce Momjian wrote: > Greg's suggestion is to use $COLUMNS if the width can't be determined > because the output is not directly to the screen. $COLUMNS is updated > by many shells. I think it is best not to look at $COLUMNS at all. If the output is to a terminal, then use ioctl to query the terminal. And provide a \pset command to set a width explicitly, which can apply in all cases.
Peter Eisentraut wrote: > Bruce Momjian wrote: > > Greg's suggestion is to use $COLUMNS if the width can't be determined > > because the output is not directly to the screen. ?$COLUMNS is updated > > by many shells. > > I think it is best not to look at $COLUMNS at all. If the output is to a > terminal, then use ioctl to query the terminal. And provide a \pset command > to set a width explicitly, which can apply in all cases. Yes, that is pretty much what we have now, except we try for $COLUMNS if ioctl() fails for interactive use. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Gregory Stark wrote: > "Bruce Momjian" <bruce@momjian.us> writes: > > > I think the API in the patch is the best I am going to do to keep > > everyone happy --- 'wrapped' doesn't affect file/pipe output unless you > > also tell it the width you want. Most interactive users are going to > > set 'wrapped' and never set the width so it is automatically determined. > > Aaah, I think that's the key to where we're going wrong. > > Trying to use the columns logic to encode two separate decisions. "wrapped or > not wrapped" and "how wide". Well, they kind of fit because there is no good way to know the width they would want for file/pipe output (no the terminal width isn't very helpful in most cases). Bottom line is we are going to need a way to specify the width for wrapped file/pipe (COLUMNS is not something that is easily set), and \pset columns seems to allow both the setting of the width and saying we want wrapping for file/pipe. I am not excited about 'wrapped-interactive' and 'wrapped-all' formats. Do you have some other idea in mind? > I think this also clarifies Tom's objection. He's worried about people > configuring psql for interactive use and being surprised when their automated > scripts fail to parse the resulting output. > > We do need a way to specifically request wrapped format, but if we want a way > to say "wrapped format only on a terminal" then we should have a mode for that > too. > > But once we're in wrapped format we should stick to it and always follow the > same logic to determine the width. I can't think of any cases where we have one setting for interactive and another for all uses. I do think we might be adding an 'auto' format the does aligned/wrapped/expanded based on the table width, but only for interactive use. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian escribió: > Peter Eisentraut wrote: > > > > I think it is best not to look at $COLUMNS at all. If the output is to a > > terminal, then use ioctl to query the terminal. And provide a \pset command > > to set a width explicitly, which can apply in all cases. > > Yes, that is pretty much what we have now, except we try for $COLUMNS if > ioctl() fails for interactive use. On what platforms does ioctl() fail? -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Alvaro Herrera wrote: > Bruce Momjian escribi?: > > Peter Eisentraut wrote: > > > > > > I think it is best not to look at $COLUMNS at all. If the output is to a > > > terminal, then use ioctl to query the terminal. And provide a \pset command > > > to set a width explicitly, which can apply in all cases. > > > > Yes, that is pretty much what we have now, except we try for $COLUMNS if > > ioctl() fails for interactive use. > > On what platforms does ioctl() fail? I don't think Win32 supports it. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
"Alvaro Herrera" <alvherre@commandprompt.com> writes: > On what platforms does ioctl() fail? On ssh for example. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's RemoteDBA services!
"Bruce Momjian" <bruce@momjian.us> writes: > I am not excited about 'wrapped-interactive' and 'wrapped-all' formats. > Do you have some other idea in mind? Some other idea for the name of it? Not particularly. I like your later suggestion of "auto". Perhaps just "wrapped" and "auto"? > I can't think of any cases where we have one setting for interactive and > another for all uses. In general I don't like such things but it seems to be what you're trying to do. And also what Tom seems to be yearning for when he says that it shouldn't affect file output. And if we do it we should do it properly and not hobble both modes. Note that basing it on the window size ioctl doesn't actually dtrt either. What you really want is for it to depend on isatty(). If isatty(fout) is true then you want to try to take the interactive default even if the ioctl fails. Trying to join the two decisions means that nobody will be happy. If you ssh in you won't get wrapped format, if you redirect to a file and specify wrapped format explicitly you'll be frustrated that you're still not getting it. And if you do want those things so you \pset columns you'll find it starts doing it even when you -f file.dmp -- which doesn't seem bad to me but it means it isn't satisfying your desires. > I do think we might be adding an 'auto' format the does > aligned/wrapped/expanded based on the table width, but only for > interactive use. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's On-Demand Production Tuning
Gregory Stark <stark@enterprisedb.com> writes:
> "Alvaro Herrera" <alvherre@commandprompt.com> writes:
>> On what platforms does ioctl() fail?
> On ssh for example.
That'd certainly be a showstopper if true, but it seems to be okay for
me. ssh'ing from an xterm window, and running psql on the remote side,
I can see with gdb that ioctl(TIOCGWINSZ) succeeds and gives a result
that correctly tracks window resizes --- indeed there are several bits
of psql that I'd never have committed if they didn't work in this
environment, because it's been my everyday work environment for years.
Maybe you are used to an ancient ssh version?
regards, tom lane
Gregory Stark wrote: > "Bruce Momjian" <bruce@momjian.us> writes: > > > I am not excited about 'wrapped-interactive' and 'wrapped-all' formats. > > Do you have some other idea in mind? > > Some other idea for the name of it? Not particularly. I like your later > suggestion of "auto". Perhaps just "wrapped" and "auto"? Well, I was going to bring up changes to the default after the patch was applied but I will bring it up now. I think there is some real attractivness to having long values wrap to fit on your screen in interactive mode. In fact, it is hard to justify our current interactive behavior of a row just overflowing the screen width and moving to the next line. We have gotten used to it, but it is certainly not very user-friendly. We have discussed having a formatting mode where aligned output switches to expanded output when the row is too wide. One idea would be to create an 'auto' mode that would display in aligned, or wrapped if that doesn't fit, or expanded if that doesn't fit. However, this idea really doesn't help us with the file/pipe idea. One crazy idea would be for formatting to take two values, like: \pset formatting wrapped aligned where the first is for interactive use and the second is for file/pipe output. Now, if they only specify one value, what does that control? > > I can't think of any cases where we have one setting for interactive and > > another for all uses. > > In general I don't like such things but it seems to be what you're trying to > do. And also what Tom seems to be yearning for when he says that it shouldn't > affect file output. And if we do it we should do it properly and not hobble > both modes. > > Note that basing it on the window size ioctl doesn't actually dtrt either. > What you really want is for it to depend on isatty(). If isatty(fout) is true > then you want to try to take the interactive default even if the ioctl fails. Yes, we do that now with $COLUMNS. > Trying to join the two decisions means that nobody will be happy. If you ssh > in you won't get wrapped format, if you redirect to a file and specify wrapped > format explicitly you'll be frustrated that you're still not getting it. And > if you do want those things so you \pset columns you'll find it starts doing > it even when you -f file.dmp -- which doesn't seem bad to me but it means it > isn't satisfying your desires. I think we need to honor $COLUMNS if ioctl() fails. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > > Trying to join the two decisions means that nobody will be happy. If you ssh > > in you won't get wrapped format, if you redirect to a file and specify wrapped > > format explicitly you'll be frustrated that you're still not getting it. And > > if you do want those things so you \pset columns you'll find it starts doing > > it even when you -f file.dmp -- which doesn't seem bad to me but it means it > > isn't satisfying your desires. > > I think we need to honor $COLUMNS if ioctl() fails. Clarification, honor $COLUMNS if ioctl() fails but only for interactive mode. If we can't make everyone happy we could just not add the patch. It sounds crazy but we have had cases in the past where we couldn't get concensus and we did nothing. Of course nothing is usually worse than either option but groups behave in strange ways sometimes. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > interactive mode. In fact, it is hard to justify our current > interactive behavior of a row just overflowing the screen width and > moving to the next line. We have gotten used to it, but it is certainly > not very user-friendly. +1 > > We have discussed having a formatting mode where aligned output switches > to expanded output when the row is too wide. One idea would be to > create an 'auto' mode that would display in aligned, or wrapped if that > doesn't fit, or expanded if that doesn't fit. > > However, this idea really doesn't help us with the file/pipe idea. One > crazy idea would be for formatting to take two values, like: > > \pset formatting wrapped aligned IMO we should never wrap file or pipe output. Once it leaves psql, it should be up to whatever you pushed it through (file/pipe/script) to handle formatting. Sincerely, Joshua D. Drake
Joshua D. Drake wrote: > Bruce Momjian wrote: > > We have discussed having a formatting mode where aligned output switches > > to expanded output when the row is too wide. One idea would be to > > create an 'auto' mode that would display in aligned, or wrapped if that > > doesn't fit, or expanded if that doesn't fit. > > > > However, this idea really doesn't help us with the file/pipe idea. One > > crazy idea would be for formatting to take two values, like: > > > > \pset formatting wrapped aligned > > IMO we should never wrap file or pipe output. Once it leaves psql, it > should be up to whatever you pushed it through (file/pipe/script) to > handle formatting. Yes, I understand that argument, but the way we wrap values is something an external tool is going to have great trouble duplicating. For example, if I want to send wide psql output in email, right now I just send it wide or use \x. With 'wrapped' I can set it to 72 columns and get something I can email to people. I could copy it from my screen, but if the output is more than a screen full it is much harder to capture. (Yea, I could use 'script', but that isn't for the novice, and I have no idea how that would be done on Win32.) We could go with wrap only handling interactive and then see what feedback we get from the field. For short output people can cut/paste into files if they want wrapped output. That would eliminate the need for \pset columns. We could just do ioctl() and check $COLUMNS. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > For example, if I want to send wide psql output in email, right now I > just send it wide or use \x. With 'wrapped' I can set it to 72 columns > and get something I can email to people. I could copy it from my > screen, but if the output is more than a screen full it is much harder > to capture. (Yea, I could use 'script', but that isn't for the novice, > and I have no idea how that would be done on Win32.) I think your idea of novice is a little different than mine. A novice isn't going to use this feature anyway, and by the time they start to consider it (if they ever do), they will have already wrapped it in Perl or C#. Sincerely, Joshua D. Drake
"Tom Lane" <tgl@sss.pgh.pa.us> writes: > Gregory Stark <stark@enterprisedb.com> writes: >> "Alvaro Herrera" <alvherre@commandprompt.com> writes: >>> On what platforms does ioctl() fail? > >> On ssh for example. > > That'd certainly be a showstopper if true, but it seems to be okay for > me. ssh'ing from an xterm window, and running psql on the remote side, > I can see with gdb that ioctl(TIOCGWINSZ) succeeds and gives a result > that correctly tracks window resizes --- indeed there are several bits > of psql that I'd never have committed if they didn't work in this > environment, because it's been my everyday work environment for years. Hum. It appears you're right. I did run a test earlier with ssh and strace where I saw an error from the ioctl. I have a sneaking suspicion I ran it within my usual emacs shell session without realizing. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's RemoteDBA services!
* Bruce Momjian <bruce@momjian.us> [080424 23:14]: > Well, I was going to bring up changes to the default after the patch was > applied but I will bring it up now. I think there is some real > attractivness to having long values wrap to fit on your screen in > interactive mode. In fact, it is hard to justify our current > interactive behavior of a row just overflowing the screen width and > moving to the next line. We have gotten used to it, but it is certainly > not very user-friendly. -1. Personally, I don't ever want to see that wrapped format. <opinion valid="maybe not"> That wrapped format is more user-unfriendly than the long lines, which are at least easily understandable. </opinion> So add a wrapped format, because obviously some people want it, but in a similar vien to the making psql have no banner by default, I think making wrapped format the default will cause much user consternation. In Bruce's sample, even with the rules "right there", I still had to look at the query to try and figure out what each data piece actually was. But I have to admit, trying to get a generic multi-column "wrapped" format is a hard task. I couldn't come up with a scheme for showing it easily myself on my (discarded) attempt to reply to the sample output. Anything that *doesn't* have a wrapped line end with \ and then continue with a > is going to feel awkward to me, and I'ld rather just have the long lines, because having a one-off, complicated display format in psql, even though it could technically define exactly what the data is it's trying to display, if it's different from everything else means I'm going to avoid it as much as possible. a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
Bruce Momjian wrote: > We have discussed having a formatting mode where aligned output switches > to expanded output when the row is too wide. One idea would be to > create an 'auto' mode that would display in aligned, or wrapped if that > doesn't fit, or expanded if that doesn't fit. I haven't heard any new ideas of how to resolve this issue in the past few hours so I will throw out two new ideas. Have a 'format=auto' mode that does aligned/wrapped/expanded, but only for screen output --- file/pipe would still use aligned. And have 'format=wrapped' affect file/pipe by requiring the user to specify the width, or use a default of 72. Another idea is to require the user to specify the file/pipe output width when they define format=wrapped, e.g. format=wrapped:75. If they don't specify the width, 'wrapped' doesn't affect file/pipe. (Perhaps the width controls screen width too.) (That is effectively what \pset columns does, but this is more integrated.) I have heard why people want 'wrapped' to affect file/pipe output, but I have not heard a clear explanation of why people don't want that. I personally think that having the screen width affect the width of file/pipe is odd, but if the user specifies the width, it seems fine to me. What is the objection? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian escribió: > Have a 'format=auto' mode that does aligned/wrapped/expanded, but only > for screen output --- file/pipe would still use aligned. And have > 'format=wrapped' affect file/pipe by requiring the user to specify the > width, or use a default of 72. I have a different question. Why are we mixing file and pipe output? I think the use cases are different and perhaps we should use different defaults. For example, most people I've seen writing shell scripts involving psql output have to pass the -A flag all the time. Perhaps we could change the default output to unaligned for pipes? -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
On Fri, 25 Apr 2008 10:45:01 -0400 Alvaro Herrera <alvherre@commandprompt.com> wrote: > I have a different question. Why are we mixing file and pipe > output? I think the use cases are different and perhaps we should > use different defaults. > > For example, most people I've seen writing shell scripts involving > psql output have to pass the -A flag all the time. Perhaps we could > change the default output to unaligned for pipes? and -t Sincerely, Joshua D. Drake -- The PostgreSQL Company since 1997: http://www.commandprompt.com/ PostgreSQL Community Conference: http://www.postgresqlconference.org/ United States PostgreSQL Association: http://www.postgresql.us/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
Alvaro Herrera wrote: > Bruce Momjian escribi?: > > > Have a 'format=auto' mode that does aligned/wrapped/expanded, but only > > for screen output --- file/pipe would still use aligned. And have > > 'format=wrapped' affect file/pipe by requiring the user to specify the > > width, or use a default of 72. > > I have a different question. Why are we mixing file and pipe output? I > think the use cases are different and perhaps we should use different > defaults. > > For example, most people I've seen writing shell scripts involving psql > output have to pass the -A flag all the time. Perhaps we could change > the default output to unaligned for pipes? Yes, that would make sense to me. I think the people wanting wrapped to control file/pipe don't want it as the default, but want _some_ way of getting wrapped output into a file. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > Alvaro Herrera wrote: > > Bruce Momjian escribi?: > > > > > Have a 'format=auto' mode that does aligned/wrapped/expanded, but only > > > for screen output --- file/pipe would still use aligned. And have > > > 'format=wrapped' affect file/pipe by requiring the user to specify the > > > width, or use a default of 72. > > > > I have a different question. Why are we mixing file and pipe output? I > > think the use cases are different and perhaps we should use different > > defaults. > > > > For example, most people I've seen writing shell scripts involving psql > > output have to pass the -A flag all the time. Perhaps we could change > > the default output to unaligned for pipes? > > Yes, that would make sense to me. > > I think the people wanting wrapped to control file/pipe don't want it as > the default, but want _some_ way of getting wrapped output into a file. Let me add that the patch as it was posted does not have wrapping affecting file/pipe output unless you also specify a column width with \pset. This seemed the most logical and tried to satisfy the most people. You can see this in the documentation changes of the patch: ftp://momjian.us/pub/postgresql/mypatches/wrap I have now clarified the documentation in the patch. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
"Bruce Momjian" <bruce@momjian.us> writes: >> I think the people wanting wrapped to control file/pipe don't want it as >> the default, but want _some_ way of getting wrapped output into a file. > > Let me add that the patch as it was posted does not have wrapping > affecting file/pipe output unless you also specify a column width with > \pset. This seemed the most logical and tried to satisfy the most > people. You can see this in the documentation changes of the patch: If you specify format=wrapped and get something other than wrapped it's a bug and people will undoubtedly report it as such. If you want a way to specify "wrapped on a terminal but not to a non-terminal" then you should make that explicit and separate from the column-width determination. I'm done with this thread now. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's 24x7 Postgres support!
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sat, Apr 26, 2008 at 4:46 AM, Gregory Stark wrote: > If you specify format=wrapped and get something other than wrapped it's a bug > and people will undoubtedly report it as such. > Agree. If I tell psql that I want wrapped output and it gives me something else when I output to a file, I'm going to be confused. In particular, I often work up a query in psql, making revisions with \e, and when I've got the results I want, I do a \g out.txt to dump the query output into a file. I expect that the contents of out.txt will be exactly the same as the query output I've just been crafting in psql. If I want to output the query in a machine-readable form to a file, I just use \a to switch to unaligned output. Aside: \a should probably be changed to toggle between "unaligned" and "aligned"/"wrapped", whichever of the two you had selected last. Cheers, BJ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) Comment: http://getfiregpg.org iD8DBQFIEixE5YBsbHkuyV0RArhoAKDUH/Svt84xPFn7BGRkq7zEWtonpwCg/pOS 66uQ6mKWvxnZLICfAcJzdLw= =ibbj -----END PGP SIGNATURE-----
Brendan Jurd wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Sat, Apr 26, 2008 at 4:46 AM, Gregory Stark wrote: > > If you specify format=wrapped and get something other than wrapped it's a bug > > and people will undoubtedly report it as such. > > > > Agree. If I tell psql that I want wrapped output and it gives me > something else when I output to a file, I'm going to be confused. > > In particular, I often work up a query in psql, making revisions with > \e, and when I've got the results I want, I do a \g out.txt to dump > the query output into a file. > > I expect that the contents of out.txt will be exactly the same as the > query output I've just been crafting in psql. Obviously you have expections of how wrapping should behave. Please name me an application that has a wrapped mode that has the output to a file wrap based on the screen width? It isn't 'ls -C'. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian escribió: > Obviously you have expections of how wrapping should behave. Please > name me an application that has a wrapped mode that has the output to a > file wrap based on the screen width? It isn't 'ls -C'. Why would we need to imitate what other apps do? What we need to investigate is use cases, and how do we cater for each one, making it easy for the most common while at the same time making it not impossible for the most obscure. There is no point in doing things in a certain way just because others do the same. Are you going to argue that we need to make the server crash from time to time because other systems do that too? We came up with dollar quoting which is a completely novel idea AFAIK. Why can't we come up with other useful, novel designs? -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Alvaro Herrera <alvherre@commandprompt.com> writes:
> There is no point in doing things in a certain way just because others
> do the same. Are you going to argue that we need to make the server
> crash from time to time because other systems do that too?
> We came up with dollar quoting which is a completely novel idea AFAIK.
> Why can't we come up with other useful, novel designs?
I don't think there's a need to invent a totally new way of dealing with
window wrapping. If we did, it'd be highly unlikely to preserve the
principle of least surprise.
regards, tom lane
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sat, Apr 26, 2008 at 5:21 AM, Bruce Momjianwrote: > Obviously you have expections of how wrapping should behave. Please > name me an application that has a wrapped mode that has the output to a > file wrap based on the screen width? It isn't 'ls -C'. > Trying to draw analogies between psql and other command-line tools is always going to be a bit of a stretch. Name me any application at all that tries to produce both human- and machine- readable representations of arbitrary tabular data. All I'm saying is, if the user has exlicitly told psql that he wants wrapped, human-readable output, we should give it to him regardless of where he wants to put it. If the user hasn't specified any format at all, then it's fine to play guessing games and try to select the best format automatically for him, based on factors like the destination. But IMO once the user makes a determination about the output format, that's the end of the story. You toe that line. Cheers, BJ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) Comment: http://getfiregpg.org iD8DBQFIEju85YBsbHkuyV0RAgNTAJ4ghWB1SlvuQhsH3ltrbVM5LoMBwACgoTdT B6UDw4oG5tlwTlcLcJA4xic= =2lmV -----END PGP SIGNATURE-----
>>> On Fri, Apr 25, 2008 at 3:14 PM, in message <37ed240d0804251314y72461b8fy8d56606f362f91c6@mail.gmail.com>, "Brendan Jurd" <direvus@gmail.com> wrote: > If the user hasn't specified any format at all, then it's fine to play > guessing games and try to select the best format automatically for > him, based on factors like the destination. But IMO once the user > makes a determination about the output format, that's the end of the > story. You toe that line. I would go further, and say that it would be surprising and troublesome for psql to guess at whether I want wrapping or unaligned output. A given set of command line switches and a given set of inputs should give a consistent output format, regardless of whether it's going into a pipe or a disk file or out to the local console or back through ssh. Like a previous poster, I often use an interactive session to refine something that will be run against a list of servers with xargs or will be run from crontab. If the interactive output is big enough to cause it to go through "less", then I still want to see the format it would have if it didn't. If I save to a file from "less" or copy and paste from a ssh window when it was a few long lines, I want it to match what I will get if I run directly to disk. I consider current behavior pretty darned friendly to the way I work. Some of the suggestions in this thread sound downright nightmarish to me. I hope that wrapping never happens without an explicit command line option or a backslash command. -Kevin
Alvaro Herrera wrote: > Bruce Momjian escribi?: > > > Obviously you have expections of how wrapping should behave. Please > > name me an application that has a wrapped mode that has the output to a > > file wrap based on the screen width? It isn't 'ls -C'. > > Why would we need to imitate what other apps do? What we need to > investigate is use cases, and how do we cater for each one, making it > easy for the most common while at the same time making it not impossible > for the most obscure. > > There is no point in doing things in a certain way just because others > do the same. Are you going to argue that we need to make the server > crash from time to time because other systems do that too? > > We came up with dollar quoting which is a completely novel idea AFAIK. > Why can't we come up with other useful, novel designs? Your argument about crashing above seems like reductio ad absurdum (http://en.wikipedia.org/wiki/Proof_by_contradiction). My point was that the poster was saying he expected the file/pipe output to honor the screen width on output, so I asked him for an example of why he had that expectation --- that seems logical. Perhaps it is only to be consistent with other psql behavior. FYI, ls -C actually wraps to 72(?) unless you specify another width, so one possible behavior would be for \pset wrapped to wrap to 72 for file/pipe unless you set \pset columns. That might make the "I want it always to wrap" group happier, but not the "wrapped shouldn't affect file/pipe". I have not heard anyone explain why the later behavior is bad, especially if we default to a width of 72 rather than the screen width. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
As the originator of the "psql wraps at window width" patch, I'd like to set a matter or two straight: The ioctl() function does not fail under ssh, contrary to the assertion made several times. Nor does $COLUMNS remain static if the window size changes. $COLUMNS is not a property of a bash, you'll find it is set by the readline library. $COLUMNS is not fully cross-platform, though $MANWIDTH should be fine. Please supply counter examples if needed, but this is pretty well traveled ground. I think the original patch is fat and happy as far as interactive terminal use. But I agree it's not desirable to wrap file any sort of stream output, by default, as that would break just about any script known to mankind. Yet wrapping is a very user-friendly default for interactive terminals. This is potentially an irreconcilable inconsistency. I weigh in on the side of leaving it inconsistent, but making it really easy to force the behavior you want with something like: \pset format wrap [auto|nnn|off] But that leaves a big hole: what does the setting in .psqlrc refer to? Do we need separate controls in .psql? \pset format_terminal wrap [auto|nnn|off] \pset format_terminal html \pset format_stream wrap [auto|nnn|off] \pset format_streamhtml Where, on a stream, auto and off would have the same meaning, and \pset format would set both?
Am Samstag, 26. April 2008 schrieb Bryce Nesbitt: > But that leaves a big hole: what does the setting in .psqlrc refer to? > Do we need separate controls in .psql? > > \pset format_terminal wrap [auto|nnn|off] > \pset format_terminal html > \pset format_stream wrap [auto|nnn|off] > \pset format_stream html I think one of the weirdest behaviors in psql is that it reads the startup file in noninteractive mode. Unix shells don't do this for good reasons. If we could work out a change that moves the psql startup file behavior more in line with that of Unix shells, perhaps meaning separate startup files, then users could put all the crazy formatting settings they like in .psqlrc without affecting noninteractive output. And there would still be the possibility to format noninteractive output the way you want by explicit intervention. This is basically what your pseudoproposal above would accomplish, just on a more global scale.
Bryce Nesbitt wrote: > But I agree it's not desirable to wrap file any sort of stream output, > by default, as that would break just about any script known to mankind. > Yet wrapping is a very user-friendly default for interactive terminals. > This is potentially an irreconcilable inconsistency. > > I weigh in on the side of leaving it inconsistent, but making it really > easy to force the behavior you want with something like: > \pset format wrap [auto|nnn|off] > > But that leaves a big hole: what does the setting in .psqlrc refer to? > Do we need separate controls in .psql? > > \pset format_terminal wrap [auto|nnn|off] > \pset format_terminal html > \pset format_stream wrap [auto|nnn|off] > \pset format_stream html > > Where, on a stream, auto and off would have the same meaning, and \pset > format would set both? Hey, I can work with this idea. First, there really is no 'off' mode for wrapped because that is aligned. What we could do is to have: \pset format wrapped display affect only output to the screen, using the screen width, and: \pset format wrapped nnn affect output to the screen and file/pipes. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
[Just when I thought I was out, they pull me back in -- argh, I'm weak] "Bruce Momjian" <bruce@momjian.us> writes: > FYI, ls -C actually wraps to 72(?) unless you specify another width, I told you exactly what ls did, at least GNU ls. It uses -w if specified, if not then it uses the ioctl if that succeeds, if it fails it uses COLUMNS, and if that's unavailable it uses a constant. > one possible behavior would be for \pset wrapped to wrap to 72 for > file/pipe unless you set \pset columns. You can't use ls to justify having different rules for screen width for "file/pipe": $ COLUMNS=80 ls -C | cat distmp3.rh3280 gconfd-stark orbit-stark purpleNMN49T ssh-WdHPsk4277 $ COLUMNS=60 ls -C | cat distmp3.rh3280 orbit-stark ssh-WdHPsk4277 gconfd-stark purpleNMN49T $ COLUMNS=40 ls -C | cat distmp3.rh3280 purpleNMN49T gconfd-stark ssh-WdHPsk4277 orbit-stark $ COLUMNS=20 ls -C | cat distmp3.rh3280 gconfd-stark orbit-stark purpleNMN49T ssh-WdHPsk4277 > That might make the "I want it always to wrap" group happier, but not the > "wrapped shouldn't affect file/pipe". I have not heard anyone explain why > the later behavior is bad, especially if we default to a width of 72 rather > than the screen width. Presumably they're concerned that scripts which dump out data and then try to parse it will have trouble parsing wrapped output. In any case that should be based on whether isatty() is true, which is related to but not the same as whether the window size ioctl succeeds. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's Slony Replication support!
Bruce Momjian wrote:
> Hey, I can work with this idea. First, there really is no 'off' mode
> for wrapped because that is aligned...
Well, come to think of it, "wrapped" is not really a new output format
in the sense of "html" or "latex". It could build on aligned:
\pset format aligned [autowrap|nowrap|nnn]
But there's still the issue of wanting separate defaults for tty and
stream outputs. The last thing you want is an admin deciding on
wrapping, and then subtly breaking scripts. My personal desired
defaults are:
* Terminals autowrap.
* Streams don't wrap, except in the rare case when I want to force a
specific width (e.g. 79 for a newsgroup posting).
-Bryce
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sat, Apr 26, 2008 at 5:08 PM, Bryce Nesbittwrote: > Well, come to think of it, "wrapped" is not really a new output format in > the sense of "html" or "latex". It could build on aligned: > > \pset format aligned [autowrap|nowrap|nnn] > I agree that wrapped is more a variant of aligned mode than a format mode in its own right. Expressing it that way with \pset has the nice bonus that you don't lose your preference for wrapped mode when switching between aligned and unaligned with \a. > But there's still the issue of wanting separate defaults for tty and stream > outputs. The last thing you want is an admin deciding on wrapping, and then > subtly breaking scripts. My personal desired defaults are: > Well, if we pursue Peter's suggestion that psql abstain from reading the startup file in noninteractive mode, then this problem goes away. An admin could opt to wrap in his interactive sessions without it ever affecting the behaviour of scripts ... which is exactly what you would want. Cheers, BJ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.7 (GNU/Linux) Comment: http://getfiregpg.org iD8DBQFIEs3x5YBsbHkuyV0RAiPsAJ0QIhWmq4s622dTZNP4MknxWTm30wCfaMTP kY9qEW0GB3rJb3Xq5F92geY= =GbOb -----END PGP SIGNATURE-----
Gregory Stark wrote: > > [Just when I thought I was out, they pull me back in -- argh, I'm weak] > > "Bruce Momjian" <bruce@momjian.us> writes: > > > FYI, ls -C actually wraps to 72(?) unless you specify another width, > > I told you exactly what ls did, at least GNU ls. It uses -w if specified, if > not then it uses the ioctl if that succeeds, if it fails it uses COLUMNS, and > if that's unavailable it uses a constant. > $ COLUMNS=40 ls -C | cat > distmp3.rh3280 purpleNMN49T > gconfd-stark ssh-WdHPsk4277 > orbit-stark I don't see that behavior here on Ubuntu 7.10: $ COLUMNNS=120 ls -C |catarchive cd initrd lost+found proc srv usrbasement.usr dev initrd.img media root sys varbin etc laptop mnt rtmp tmp vmlinuzboot home lib opt sbin u win$ ls --versionls (GNU coreutils) 5.97 That is not a 120 width. 'ls' seems to ignore columns for pipe output. > > That might make the "I want it always to wrap" group happier, but not the > > "wrapped shouldn't affect file/pipe". I have not heard anyone explain why > > the later behavior is bad, especially if we default to a width of 72 rather > > than the screen width. > > Presumably they're concerned that scripts which dump out data and then try to > parse it will have trouble parsing wrapped output. In any case that should be > based on whether isatty() is true, which is related to but not the same as > whether the window size ioctl succeeds. Right now we honor $COLUMNS only when isatty() is true. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > Hey, I can work with this idea. First, there really is no 'off' mode > for wrapped because that is aligned. What we could do is to have: > > \pset format wrapped display > > affect only output to the screen, using the screen width, and: > > \pset format wrapped nnn > > affect output to the screen and file/pipes. A new idea would be for a wrap value of zero to be special: \pset format wrapped 0 to wrap to screen width for terminal and file/pipe output. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > Gregory Stark wrote: > I don't see that behavior here on Ubuntu 7.10: > > $ COLUMNNS=120 ls -C |cat > archive cd initrd lost+found proc srv usr > basement.usr dev initrd.img media root sys var > bin etc laptop mnt rtmp tmp vmlinuz > boot home lib opt sbin u win > $ ls --version > ls (GNU coreutils) 5.97 > > That is not a 120 width. 'ls' seems to ignore columns for pipe output. Oops, Alvaro pointed out I typo'ed the variable name COLUMNS as COLUMNNS. I see now that 'ls -C' does honor columns. See my later posting about '\pset wrapped 0' as a special case where we could honor the ioctl/COLUMNS case. My real confusion is this: $ echo $COLUMNS146 $ ls -C|lessarchive cd initrd lost+found proc srv usrbasement.usr dev initrd.img media root sys varbin etc laptop mnt rtmp tmp vmlinuzboot home lib opt sbin u win $ COLUMNS=120 ls -C|lessarchive bin cd etc initrd laptop lost+found mnt proc rtmp srv tmp usr vmlinuzbasement.usr boot dev home initrd.img lib media opt root sbin sys u var win Why does the first 'ls' not honor columns while the second does? How does 'ls' detect that the COLUMNS=120 is somehow different from the default COLUMNS value? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
* Bruce Momjian <bruce@momjian.us> [080426 09:44]: > Why does the first 'ls' not honor columns while the second does? How > does 'ls' detect that the COLUMNS=120 is somehow different from the > default COLUMNS value? I would hazard a guess that COLUMNS isn't "exported" from your shell environment in the first case. In the other cases, the explicit:VAR=... command the shell is told to set VAR explicitly before starting command, in addition to any exported vars. a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
Bruce Momjian wrote:
> Oops, Alvaro pointed out I typo'ed the variable name COLUMNS as
> COLUMNNS. I see now that 'ls -C' does honor columns. See my later
> posting about '\pset wrapped 0' as a special case where we could honor
> the ioctl/COLUMNS case.
>
> My real confusion is this:
>
> $ echo $COLUMNS
> 146
>
> $ ls -C|less
> archive cd initrd lost+found proc srv usr
> basement.usr dev initrd.img media root sys var
> bin etc laptop mnt rtmp tmp vmlinuz
> boot home lib opt sbin u win
>
> $ COLUMNS=120 ls -C|less
> archive bin cd etc initrd laptop lost+found mnt proc rtmp srv tmp usr vmlinuz
> basement.usr boot dev home initrd.img lib media opt root sbin sys u var win
>
> Why does the first 'ls' not honor columns while the second does? How
> does 'ls' detect that the COLUMNS=120 is somehow different from the
> default COLUMNS value?
Ah, I see now. $COLUMNS isn't exported to subshells, hence the previous
comment that readline needs to be called before it has a value. It
seems psql does have COLUMNS set if readline is defined, which means we
can't detect of $COLUMNS was passed to psql or was detected. More
interesting, it doesn't seem psql sets $COLUMNS in batch mode:
psql -c '\echo `echo $COLUMNS`' test{blank line}
COLUMNS=120 sql -c '\echo `echo $COLUMNS`' test120
so we could argue that COLUMNS should be honored but again this would
affect \g filename. The issue with 'ls' is that it knows it isn't going
to be getting new commands from the user that change where its output is
going, while psql can.
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Bruce Momjian <bruce@momjian.us> writes:
> I don't see that behavior here on Ubuntu 7.10:
> $ COLUMNNS=120 ls -C |cat
> archive cd initrd lost+found proc srv usr
> basement.usr dev initrd.img media root sys var
> bin etc laptop mnt rtmp tmp vmlinuz
> boot home lib opt sbin u win
> $ ls --version
> ls (GNU coreutils) 5.97
> That is not a 120 width. 'ls' seems to ignore columns for pipe output.
Well, it's *certainly* gonna ignore "COLUMNNS".
regards, tom lane
Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
>
>> I don't see that behavior here on Ubuntu 7.10:
>> $ COLUMNNS=120 ls -C |cat
>> archive cd initrd lost+found proc srv usr
>> basement.usr dev initrd.img media root sys var
>> bin etc laptop mnt rtmp tmp vmlinuz
>> boot home lib opt sbin u win
>> $ ls --version
>> ls (GNU coreutils) 5.97
>>
> Well, it's *certainly* gonna ignore "COLUMNNS".
>
> regards, tom lane
>
I'm having trouble seeing the relevance, either way. First many shells
don't set $COLUMNS at all (readline does it for psql). And most shells
that set $COLUMNS don't export it. So most people get different output
from:
# ls -C# ls -C | cat
Unless they are in the habit of doing:
# COLUMNS=$COLUMNS ls -C |cat
I think it's too weird to default pipes to whatever the terminal width
happens to be. So that leaves you with an explicit command to set the
width for pipes.
-Bryce
Echo $MANWIDTH
"Bryce Nesbitt" <bryce2@obviously.com> writes: > Unless they are in the habit of doing: > > # COLUMNS=$COLUMNS ls -C |cat Some of us are actually in the habit of doing that because it's easier to use the standard interface than remembering the different command-line option for each command. I quite often do precisely that with dpkg, for example. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's 24x7 Postgres support!
Gregory Stark wrote: > "Bryce Nesbitt" <bryce2@obviously.com> writes: > > > Unless they are in the habit of doing: > > > > # COLUMNS=$COLUMNS ls -C |cat > > Some of us are actually in the habit of doing that because it's easier to use > the standard interface than remembering the different command-line option for > each command. I quite often do precisely that with dpkg, for example. Yes, this is true, but it assume the application is not going to set $COLUMNS itself, like psql does in interactive mode: test=> \echo `echo $COLUMNS`127 $ sql -c '\echo `echo $COLUMNS`' test(empty) Now, we could get fancy and honor $COLUMNS only in non-interactive mode, but that seems confusing. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
"Bruce Momjian" <bruce@momjian.us> writes: > Gregory Stark wrote: >> "Bryce Nesbitt" <bryce2@obviously.com> writes: >> >> > Unless they are in the habit of doing: >> > >> > # COLUMNS=$COLUMNS ls -C |cat >> >> Some of us are actually in the habit of doing that because it's easier to use >> the standard interface than remembering the different command-line option for >> each command. I quite often do precisely that with dpkg, for example. > > Yes, this is true, but it assume the application is not going to set > $COLUMNS itself, like psql does in interactive mode: > > test=> \echo `echo $COLUMNS` > 127 > > $ sql -c '\echo `echo $COLUMNS`' test > (empty) > > Now, we could get fancy and honor $COLUMNS only in non-interactive mode, > but that seems confusing. We could always read COLUMNS early on before readline is initialized and stash the value away in a variable. But... We would only look at COLUMNS if the ioctl for window size failed. Does psql/readline do anything to COLUMNS in that case? -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's RemoteDBA services!
Gregory Stark wrote: > > Now, we could get fancy and honor $COLUMNS only in non-interactive mode, > > but that seems confusing. > > We could always read COLUMNS early on before readline is initialized and stash > the value away in a variable. But... > > We would only look at COLUMNS if the ioctl for window size failed. Does > psql/readline do anything to COLUMNS in that case? We do look at COLUMNS if the ioctl() fails, but not for file/pipe output. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
"Bruce Momjian" <bruce@momjian.us> writes: > Gregory Stark wrote: >> > Now, we could get fancy and honor $COLUMNS only in non-interactive mode, >> > but that seems confusing. >> >> We could always read COLUMNS early on before readline is initialized and stash >> the value away in a variable. But... >> >> We would only look at COLUMNS if the ioctl for window size failed. Does >> psql/readline do anything to COLUMNS in that case? > > We do look at COLUMNS if the ioctl() fails, but not for file/pipe > output. Yeah, it looks like your most recent patch still has the bug that if the user specifies wrapped there are some complicated rules creating cases where it will ignore the user's request and use un-wrapped output instead. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Get trained by Bruce Momjian - ask me about EnterpriseDB'sPostgreSQL training!
Am Dienstag, 29. April 2008 schrieb Bruce Momjian: > We do look at COLUMNS if the ioctl() fails, but not for file/pipe > output. This is quite a useless complication. Readline uses exactly the same ioctl() call to determine the columns, so if ioctl() were to fail, then COLUMNS would be unset or wrong as well.
"Peter Eisentraut" <peter_e@gmx.net> writes: > Am Dienstag, 29. April 2008 schrieb Bruce Momjian: >> We do look at COLUMNS if the ioctl() fails, but not for file/pipe >> output. > > This is quite a useless complication. Readline uses exactly the same ioctl() > call to determine the columns, so if ioctl() were to fail, then COLUMNS would > be unset or wrong as well. COLUMNS is just a regular environment variable. The user is free to set it and many people have dotfiles, aliases, or scripts which do just that. Consider, for example, someone with a cron job which runs several commands such as "ls -C", "dpkg -l", and of course "psql -Pformat=wrapped" to generate various reports and wants it all formatted to 72 columns. They would normally just set COLUMNS=72 and run their commands and get an email all formatted to 72 columns. But your point is valid, that's why I'm not too worried about cases where COLUMNS is set to the desired width but readline interferes with it. In those cases we would be using the ioctl value anyways. It would probably still be a good idea to getenv(COLUMNS) early on before readline is initialized though. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Get trained by Bruce Momjian - ask me about EnterpriseDB'sPostgreSQL training!
Peter Eisentraut wrote: > Am Dienstag, 29. April 2008 schrieb Bruce Momjian: > > We do look at COLUMNS if the ioctl() fails, but not for file/pipe > > output. > > This is quite a useless complication. Readline uses exactly the same ioctl() > call to determine the columns, so if ioctl() were to fail, then COLUMNS would > be unset or wrong as well. I was thinking about Win32 or binaries that don't have readline. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Gregory Stark wrote: > "Bruce Momjian" <bruce@momjian.us> writes: > > > Gregory Stark wrote: > >> > Now, we could get fancy and honor $COLUMNS only in non-interactive mode, > >> > but that seems confusing. > >> > >> We could always read COLUMNS early on before readline is initialized and stash > >> the value away in a variable. But... > >> > >> We would only look at COLUMNS if the ioctl for window size failed. Does > >> psql/readline do anything to COLUMNS in that case? > > > > We do look at COLUMNS if the ioctl() fails, but not for file/pipe > > output. > > Yeah, it looks like your most recent patch still has the bug that if the user > specifies wrapped there are some complicated rules creating cases where it > will ignore the user's request and use un-wrapped output instead. Can you be more specific? You mean if the headings don't fit? Yea, that is true. I am thinking of adding a \pset auto format to \x in those cases, but that if for later. Also, I thiink you could do in your .psqlrc: \pset columns `echo $COLUMNS` to get the behavior you want. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
"Bruce Momjian" <bruce@momjian.us> writes: > Gregory Stark wrote: >> "Bruce Momjian" <bruce@momjian.us> writes: >> >> > We do look at COLUMNS if the ioctl() fails, but not for file/pipe >> > output. >> >> Yeah, it looks like your most recent patch still has the bug that if the user >> specifies wrapped there are some complicated rules creating cases where it >> will ignore the user's request and use un-wrapped output instead. > > Can you be more specific? You mean if the headings don't fit? Yea, > that is true. I am thinking of adding a \pset auto format to \x in > those cases, but that if for later. [No I wasn't thinking of that, that's an interesting case too though I think we might need to think a bit harder about cases that wrap poorly. If you have long column headings we could wrap those too. But what if you have enough space for just a few characters per column and you have long text fields in those columns?] I just meant the same thing I've been on about all week. Currently the decision about whether to use wrapped mode is tied up with the decision on what width to use and the result is that we ignore -Pformat=wrapped according to some arcane set of rules. The cases where we ignore the user's selected format are quite complex and not accurately described in the documentation. They're also not accurately described by your "not for file/pipe output" description either. An accurate description would appear to be something like: <quote>Wrapped</quote> is like <literal>aligned</> but wraps to a targetwidth of <literal>\pset columns</> or the width ofthe screen (unless screensize determination fails or output has been redirected using -o or \o inwhich case it is ignoredand psql uses normal aligned mode unless \psetcolumns is used). It's confusing and inconsistent. I think it's better to pick a simple set of general principles and code to that. Trying to code to presumed use cases often ends up with code which handles corner cases poorly or inconsistently. I think the behaviour should be simply: format=autoisatty(fout) ? format := wrapped : format := aligned format=wrappedcolumns := \pset columns || ioctl(fout) || getenv(COLUMNS) || 79 [Note in the above that the ioctl is on fout, not stdout!] That would be easy to explain in the documentation as two simple consistent rules. And as a bonus it would be consistent with other programs which use these variables. So the description I would code to is simply: "Wrapped" is like aligned but wraps to \pset columns or an automaticallydetermined screen size. The screen size is determinedautomatically if outputis to a terminal which supports that, if that fails then by checking theCOLUMNS environmentvariable, and if that's unset then by defaulting to 79. "Auto" selects "wrapped" format when output is a terminal and "aligned"format otherwise. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's Slony Replication support!
Am Dienstag, 29. April 2008 schrieb Bruce Momjian: > Peter Eisentraut wrote: > > Am Dienstag, 29. April 2008 schrieb Bruce Momjian: > > > We do look at COLUMNS if the ioctl() fails, but not for file/pipe > > > output. > > > > This is quite a useless complication. Readline uses exactly the same > > ioctl() call to determine the columns, so if ioctl() were to fail, then > > COLUMNS would be unset or wrong as well. > > I was thinking about Win32 or binaries that don't have readline. These rules don't seem very consistent. You are mixing platform dependencies, build options, theoretical, unproven failures of kernel calls, none of which have anything to do with each other. For example, if readline weren't installed, then there would be no one who sets COLUMNS, so why look at it? If you want to allow users to set COLUMNS manually (possibly useful, see Greg Stark's arguments), then it should have priority over ioctl(), not the other way around.
"Bruce Momjian" <bruce@momjian.us> writes: > Peter Eisentraut wrote: >> Am Dienstag, 29. April 2008 schrieb Bruce Momjian: >> > Peter Eisentraut wrote: >> > > Am Dienstag, 29. April 2008 schrieb Bruce Momjian: >> > > > We do look at COLUMNS if the ioctl() fails, but not for file/pipe >> > > > output. >> > > >> > > This is quite a useless complication. Readline uses exactly the same >> > > ioctl() call to determine the columns, so if ioctl() were to fail, then >> > > COLUMNS would be unset or wrong as well. >> > >> > I was thinking about Win32 or binaries that don't have readline. >> >> These rules don't seem very consistent. You are mixing platform dependencies, >> build options, theoretical, unproven failures of kernel calls, none of which >> have anything to do with each other. For example, if readline weren't >> installed, then there would be no one who sets COLUMNS, so why look at it? >> If you want to allow users to set COLUMNS manually (possibly useful, see Greg >> Stark's arguments), then it should have priority over ioctl(), not the other >> way around. > > OK, two people like it, no one has objected. :-) I will work on making > those changes. Thanks. Uh, precisely what changes are you referring to now? "These rules don't seem very consistent" doesn't sound like "no one has objected" to me. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's PostGIS support!
This patch is blocking other work -- for instance, the PrintTable API
and two patches that depend on it.
Could we get the main hunks committed soon, with the policy bits
stripped out? That way, discussion on the behavior can continue until
we reach an agreement, and others can work on other patches.
("Policy bits stripped out" could mean selecting the wrapped behavior
only explicitly and selecting the column width only on a \pset variable.
All the hard bits about when to use $COLUMNS and the ioctl(), which are
the controversial parts, can be deferred and are pretty localized
changes AFAICS.)
--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Peter Eisentraut wrote: > Am Dienstag, 29. April 2008 schrieb Bruce Momjian: > > Peter Eisentraut wrote: > > > Am Dienstag, 29. April 2008 schrieb Bruce Momjian: > > > > We do look at COLUMNS if the ioctl() fails, but not for file/pipe > > > > output. > > > > > > This is quite a useless complication. Readline uses exactly the same > > > ioctl() call to determine the columns, so if ioctl() were to fail, then > > > COLUMNS would be unset or wrong as well. > > > > I was thinking about Win32 or binaries that don't have readline. > > These rules don't seem very consistent. You are mixing platform dependencies, > build options, theoretical, unproven failures of kernel calls, none of which > have anything to do with each other. For example, if readline weren't > installed, then there would be no one who sets COLUMNS, so why look at it? > If you want to allow users to set COLUMNS manually (possibly useful, see Greg > Stark's arguments), then it should have priority over ioctl(), not the other > way around. OK, two people like it, no one has objected. :-) I will work on making those changes. Thanks. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Alvaro Herrera <alvherre@commandprompt.com> writes:
> This patch is blocking other work -- for instance, the PrintTable API
> and two patches that depend on it.
> Could we get the main hunks committed soon, with the policy bits
> stripped out? That way, discussion on the behavior can continue until
> we reach an agreement, and others can work on other patches.
This patch seems sufficiently controversial that "commit now" is the
very last thing that should happen to it.
I suggest committing the PrintTable stuff and not worrying about whether
that breaks the wrap patch.
regards, tom lane
Tom Lane wrote: > This patch seems sufficiently controversial that "commit now" is the > very last thing that should happen to it. > > I suggest committing the PrintTable stuff and not worrying about whether > that breaks the wrap patch. > > regards, tom lane\ > AFIK, the only thing that's controversial about the patch is how to turn it on and off -- the actual core code appears to be inflaming no passions. And it's the core code that presents merge issues. Could it be committed with a hidden enable syntax, for the interim? Or even no enable syntax, just to have it in the code base? \pset format wrap-beta-test on
Bryce Nesbitt <bryce2@obviously.com> writes:
> AFIK, the only thing that's controversial about the patch is how to turn
> it on and off -- the actual core code appears to be inflaming no
> passions. And it's the core code that presents merge issues.
Well, personally I haven't read the core code yet, since it's not commit
fest yet ;-). I don't know whether there are any issues there, but
it wouldn't surprise me given the number of issues in the control code.
regards, tom lane
Gregory Stark wrote: > [No I wasn't thinking of that, that's an interesting case too though I think > we might need to think a bit harder about cases that wrap poorly. If you have > long column headings we could wrap those too. But what if you have enough > space for just a few characters per column and you have long text fields in > those columns?] I've been using this patch for months. After the first week, I switched to the behavior above (the wrap code gives up at a certain point). It is a lot better, try it for yourself. If you try to cram too much stuff onto a line it's going to become an even more unreadable mess of very very tall columns. Wrapping column headings only gets you so far, delaying the inevitable a little bit. A patch to squeeze out extra header space, or abbreviate headers would be an advance. -Bryce \pset format aligned autowrap \pset format aligned 80
Gregory Stark wrote: > [No I wasn't thinking of that, that's an interesting case too though I think > we might need to think a bit harder about cases that wrap poorly. If you have > long column headings we could wrap those too. But what if you have enough > space for just a few characters per column and you have long text fields in > those columns?] I've been using this patch for months. After the first week, I switched to the behavior above (the wrap code gives up at a certain point). It is a lot better, try it for yourself. If you try to cram too much stuff onto a line it's going to become an even more unreadable mess of very very tall columns. Wrapping column headings only gets you so far, delaying the inevitable a little bit. A patch to squeeze out extra header space, or abbreviate headers would be an advance. -Bryce \pset format aligned autowrap \pset format aligned 80
Tom Lane wrote: > Well, personally I haven't read the core code yet, since it's not commit fest yet ;-). I don't know whether there areany issues there, but it wouldn't surprise me given the number of issues in the control code. > > regards, tom lane > I'm biased because I wrote the patch. However -- I think the core is commitable (it won't break anything else, or have complex dependencies or bloat). The issue of $COLUMNS was identified as unresolved at the time the patch was submitted, and I think it's the only bit in serious question.
Bryce Nesbitt wrote: > Tom Lane wrote: > > Well, personally I haven't read the core code yet, since it's not commit fest yet ;-). I don't know whether there areany issues there, but it wouldn't surprise me given the number of issues in the control code. > > > > regards, tom lane > > > > I'm biased because I wrote the patch. However -- I think the core is > commitable (it won't break anything else, or have complex dependencies > or bloat). > > The issue of $COLUMNS was identified as unresolved at the time the patch > was submitted, and I think it's the only bit in serious question. This code is complicated. There is no need to rush and have to revisit it later. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian wrote: > Peter Eisentraut wrote: > > Am Dienstag, 29. April 2008 schrieb Bruce Momjian: > > > Peter Eisentraut wrote: > > > > Am Dienstag, 29. April 2008 schrieb Bruce Momjian: > > > > > We do look at COLUMNS if the ioctl() fails, but not for file/pipe > > > > > output. > > > > > > > > This is quite a useless complication. Readline uses exactly the same > > > > ioctl() call to determine the columns, so if ioctl() were to fail, then > > > > COLUMNS would be unset or wrong as well. > > > > > > I was thinking about Win32 or binaries that don't have readline. > > > > These rules don't seem very consistent. You are mixing platform dependencies, > > build options, theoretical, unproven failures of kernel calls, none of which > > have anything to do with each other. For example, if readline weren't > > installed, then there would be no one who sets COLUMNS, so why look at it? > > If you want to allow users to set COLUMNS manually (possibly useful, see Greg > > Stark's arguments), then it should have priority over ioctl(), not the other > > way around. > > OK, two people like it, no one has objected. :-) I will work on making > those changes. Thanks. OK, so COLUMNS should take precedence. I assume this is going to require us to read the COLUMNS enviroment variable in psql _before_ readline sets it, and that COLUMNS will only affect screen output, like ioctl(). Is that consistent? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
> OK, so COLUMNS should take precedence. I assume this is going to
> require us to read the COLUMNS enviroment variable in psql _before_
> readline sets it, and that COLUMNS will only affect screen output, like
> ioctl(). Is that consistent?
>
This whole thing is confusing enough at the point, I think a complete
proposal needs to be articulated. It is hard to comment on a fragment of
an idea.
The main cases to cover are: (1) how to specify wrap for tty's (2) how
to specify wrap for pipes (3) how to get wraped on platforms that don't
have the ioctl (presumably windows without cygwin) (4) how to set up
different defaults for tty's and pipes (e.g. wrap interactive tty's, but
leave output aligned for scripts).
And perhaps, as a bonus comment on (5) the idea of having psql NOT
source .psqlrc
I hope at some point someone will actually try the actual core wrapping
code, and comment on it.
-Bryce
"Bruce Momjian" <bruce@momjian.us> writes: > OK, so COLUMNS should take precedence. I assume this is going to > require us to read the COLUMNS enviroment variable in psql _before_ > readline sets it, and that COLUMNS will only affect screen output, like > ioctl(). Is that consistent? What are you talking about? "screen output"? Are you even getting my emails? Please confirm that you receive this email. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's On-Demand Production Tuning
Bryce Nesbitt wrote:
>
> > OK, so COLUMNS should take precedence. I assume this is going to
> > require us to read the COLUMNS environment variable in psql _before_
> > readline sets it, and that COLUMNS will only affect screen output, like
> > ioctl(). Is that consistent?
> >
> This whole thing is confusing enough at the point, I think a complete
> proposal needs to be articulated. It is hard to comment on a fragment of
> an idea.
>
> The main cases to cover are: (1) how to specify wrap for tty's
In order of precedence:
\pset columns$COLUMNSiotcl()
> (2) how to specify wrap for pipes
\pset columns
> (3) how to get wrapped on platforms that don't
> have the ioctl (presumably windows without cygwin)
\pset columns$COLUMNS
> (4) how to set up
> different defaults for tty's and pipes (e.g. wrap interactive tty's, but
> leave output aligned for scripts).
>
> And perhaps, as a bonus comment on (5) the idea of having psql NOT
> source .psqlrc
-X --no-psqlrc
> I hope at some point someone will actually try the actual core wrapping
> code, and comment on it.
I tested it and it worked well once I modified it.
Updated patch with clearer documentation that matches the above
behavior:
ftp://momjian.us/pub/postgresql/mypatches/wrap
FYI, I looked into 'ls -C' hanlding a little more and ls (GNU coreutils)
5.97 honors COLUMNS _only_ in file/pipe output, not for screen output.
What the C code does is to read COLUMNS, then overwrite that value with
ioctl() if it works.
Do we want to follow that behavior? ls has a '-w' option to specify the
width, like our \pset columns. However, the manual page seems backwards:
-w, --width=COLS assume screen width instead of current value
The GNU 'ls' manual does not mention COLUMNS.
BSD 'ls' used COLUMNS only when outputing to the screen, and ioctl
fails. However, the BSD manual says:
COLUMNS If this variable contains a string representing a decimal integer, it is used as the column position
widthfor displaying multiple-text-column output. The ls utility calculates how many pathname text columns to
displaybased on the width pro- vided. (See -C.)
Again, I think the manual is wrong.
So it seem GNU ls and BSD ls are inconsistent, which I think means we
should design our API the best we can, rather than rely on how others
interpret COLUMNS.
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Gregory Stark wrote: > "Bruce Momjian" <bruce@momjian.us> writes: > > > OK, so COLUMNS should take precedence. I assume this is going to > > require us to read the COLUMNS enviroment variable in psql _before_ > > readline sets it, and that COLUMNS will only affect screen output, like > > ioctl(). Is that consistent? > > What are you talking about? "screen output"? > > Are you even getting my emails? Please confirm that you receive this email. Yes, I am getting your emails, but I have more than you to please. Are others OK with $COLUMNS controlling screen output and file/pipe, or perhaps COLUMNS controlling only file/pipe, as GNU ls does? I have heard a few people say they only want \pset columns to control file/pipe. I have outlined the GNU ls behavior in a email I just sent. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian <bruce@momjian.us> writes:
> Are others OK with $COLUMNS controlling screen output and file/pipe, or
> perhaps COLUMNS controlling only file/pipe, as GNU ls does? I have
> heard a few people say they only want \pset columns to control
> file/pipe.
I agree with the latter. Anyone who is setting COLUMNS is going to set
it to reflect the *screen* width they want; it has zero to do with
what should happen for output going somewhere else than the screen.
regards, tom lane
Bruce Momjian wrote: > Gregory Stark wrote: > > "Bruce Momjian" <bruce@momjian.us> writes: > > > > > OK, so COLUMNS should take precedence. I assume this is going to > > > require us to read the COLUMNS enviroment variable in psql _before_ > > > readline sets it, and that COLUMNS will only affect screen output, like > > > ioctl(). Is that consistent? > > > > What are you talking about? "screen output"? > > > > Are you even getting my emails? Please confirm that you receive this email. > > Yes, I am getting your emails, but I have more than you to please. > > Are others OK with $COLUMNS controlling screen output and file/pipe, or > perhaps COLUMNS controlling only file/pipe, as GNU ls does? I have > heard a few people say they only want \pset columns to control > file/pipe. > > I have outlined the GNU ls behavior in a email I just sent. I talked to Greg on IM and he had a new idea, related to my 'auto' idea of using aligned/wrapped/expanded mode automatically. His idea is that 'auto' mode (aligned/wrapped/expanded) is only for output to the screen, and 'wrapped' is always wrapped. 'Wrapped' would be affected by \pset columns, COLUMNS, or the terminal width for output to screen, file, or pipe. If no width could be derrmined for wrapped, it would default to 72. This gives us 'wrapped' that always wraps (if it can fit the headings), and most people will use 'auto' in their psqlrc, and it affects only terminal output. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > Are others OK with $COLUMNS controlling screen output and file/pipe, or > > perhaps COLUMNS controlling only file/pipe, as GNU ls does? I have > > heard a few people say they only want \pset columns to control > > file/pipe. > > I agree with the latter. Anyone who is setting COLUMNS is going to set > it to reflect the *screen* width they want; it has zero to do with > what should happen for output going somewhere else than the screen. OK, that's what I thought, and what I think Peter wants, and what the posted patch does. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
"Bruce Momjian" <bruce@momjian.us> writes: > FYI, I looked into 'ls -C' hanlding a little more and ls (GNU coreutils) > 5.97 honors COLUMNS _only_ in file/pipe output, not for screen output. > What the C code does is to read COLUMNS, then overwrite that value with > ioctl() if it works. Well saying "file/pipe" and "screen output" is making a lot of assumptions. The pipe may still be ending up on the screen if it's being piped to a pager. And a "file" could still be a terminal. Also, not all terminals support the ioctl(). What ls does is use the same logic for all cases. When the ioctl works that takes precedence regardless of whether it's "file/pipe" or "screen output". When it fails it uses COLUMNS instead if set. We really must stop thinking in terms of specific use cases. You will tie yourselves in knots if you try to enumerate all the types of terminals, places people redirect output, and uses people might make for the output. And it will result in very complex, hard to explain behaviour which will surprise and disappoint users. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's 24x7 Postgres support!
"Tom Lane" <tgl@sss.pgh.pa.us> writes: > Bruce Momjian <bruce@momjian.us> writes: >> Are others OK with $COLUMNS controlling screen output and file/pipe, or >> perhaps COLUMNS controlling only file/pipe, as GNU ls does? I have >> heard a few people say they only want \pset columns to control >> file/pipe. > > I agree with the latter. Anyone who is setting COLUMNS is going to set > it to reflect the *screen* width they want; it has zero to do with > what should happen for output going somewhere else than the screen. That's just not true. Consider someone writing a cron job script who wants their output formatted to 72 columns. They would set COLUMNS to direct all the programs in the cron job script and be annoyed if some of them ignored it. Consider also an application like pgadmin which might want to run a script in a window and control how wide the output is. I think you need a stronger reason to disregard the user's explicit settings than assuming that you know what the user's planning to do with the resulting data. Just because the output has been redirected doesn't mean it wasn't redirected to something like "more" or "head" or whatever with the result *still* showing up on the screen. It's one thing to change the default behaviour. It's another to disregard user-requested options. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Ask me about EnterpriseDB's 24x7 Postgres support!
Gregory Stark wrote: > "Tom Lane" <tgl@sss.pgh.pa.us> writes: > > > Bruce Momjian <bruce@momjian.us> writes: > >> Are others OK with $COLUMNS controlling screen output and file/pipe, or > >> perhaps COLUMNS controlling only file/pipe, as GNU ls does? I have > >> heard a few people say they only want \pset columns to control > >> file/pipe. > > > > I agree with the latter. Anyone who is setting COLUMNS is going to set > > it to reflect the *screen* width they want; it has zero to do with > > what should happen for output going somewhere else than the screen. > > That's just not true. Consider someone writing a cron job script who wants > their output formatted to 72 columns. They would set COLUMNS to direct all the > programs in the cron job script and be annoyed if some of them ignored it. > Consider also an application like pgadmin which might want to run a script in > a window and control how wide the output is. Well, if they are surprised, they are easily surprised. :-) What logic is there that GNU ls honors COLUMNS only in non-terminal output? And the use of COLUMNS isn't even documented in the GNU ls manual page. And BSD ls honors COLUMNS only for terminal output when the ioctl fails(). It is hard to see that there is some major expectation of how COLUMNS should behave that would make our usage "surprising". If the user wants to set the wrap width for all output, they have to use \pset columns. Also, because we run on some platforms that don't have that ioctl(), we are using COLUMNS as a way of providing the width only for screen output, like iotcl() does. > I think you need a stronger reason to disregard the user's explicit settings > than assuming that you know what the user's planning to do with the resulting > data. Just because the output has been redirected doesn't mean it wasn't > redirected to something like "more" or "head" or whatever with the result > *still* showing up on the screen. > > It's one thing to change the default behaviour. It's another to disregard > user-requested options. We are at least correctly documenting our behavior, which is more than the two 'ls' versions I saw did. If users want to set something, they better consult the documentation to find out how to do it. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Gregory Stark wrote: > "Bruce Momjian" <bruce@momjian.us> writes: > > > FYI, I looked into 'ls -C' hanlding a little more and ls (GNU coreutils) > > 5.97 honors COLUMNS _only_ in file/pipe output, not for screen output. > > What the C code does is to read COLUMNS, then overwrite that value with > > ioctl() if it works. > > Well saying "file/pipe" and "screen output" is making a lot of assumptions. > The pipe may still be ending up on the screen if it's being piped to a pager. > And a "file" could still be a terminal. Also, not all terminals support the > ioctl(). True, but I can't think of a cleaner way to specify it, and I haven't see wording that is clearer. I think most people understand this distinction. If they don't they will learn soon enough when they try it. > What ls does is use the same logic for all cases. When the ioctl works that > takes precedence regardless of whether it's "file/pipe" or "screen output". > When it fails it uses COLUMNS instead if set. > > We really must stop thinking in terms of specific use cases. You will tie > yourselves in knots if you try to enumerate all the types of terminals, places > people redirect output, and uses people might make for the output. And it will > result in very complex, hard to explain behaviour which will surprise and > disappoint users. I think we need to focus on usability. Using consistent logic is nice, but if it doesn't do what most people would want, then we have to adjust. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
* Bruce Momjian <bruce@momjian.us> [080506 10:56]: > What logic is there that GNU ls honors COLUMNS only in non-terminal > output? And the use of COLUMNS isn't even documented in the GNU ls > manual page. And BSD ls honors COLUMNS only for terminal output when > the ioctl fails(). It is hard to see that there is some major > expectation of how COLUMNS should behave that would make our usage > "surprising". > > If the user wants to set the wrap width for all output, they have to use > \pset columns. > > Also, because we run on some platforms that don't have that ioctl(), we > are using COLUMNS as a way of providing the width only for screen > output, like iotcl() does. I have to admit to using the COLUMNS=... <command> trick myself. I do have COLUMNS exported in my terminal, and often to stuff like: ls -C | less and I expect it to wrap at $COLUMNS (my terminal width) in my pager. And since the GNU coreutils is pretty consistent in this regard, I often export COLUMNS=<xxx> in scripts for cron jobs, reports, etc, to get output that formats nicely for emails, etc. If I ever wanted the psql wrapped format, I guess *I* would hope that psql would work that way, simply because that's the way the other tools I use do it. But I can see that if the other tools work differently, then there is going to have to be some people (maybe everybody) always double-checking the psql man page to find out again how it formats. But since I'm not a user wanting the wrapped format, don't cater to my hypothetical wants. But one of the interesting things is that psql has an is *interactive* mode (something the GNU utils don't have to worry about). So *when* you choose to figure out your columns is important, and really impacts behaviour too. For instance, if I was doing a query, I often to it interactively first:SELECT [...] FROM [....] LIMIT 50; And when I'm sure I have the right values,expressions, column aliases, etc, I do:\o /tmp/outputSELECT [...] FROM [...];\o And in this case, I would expect that /tmp/output would have identical formatting to the LIMITed query I just ran interactively, not matter what setting I had for format/wrapped/auto/$COLUMNS. > We are at least correctly documenting our behavior, which is more than > the two 'ls' versions I saw did. If users want to set something, they > better consult the documentation to find out how to do it. Correctly documenting it is good, just as the version of GNU ls I have on Debian, where I see: `-1'`--format=single-column' List one file per line. This is the default for `ls' when standard output is not aterminal. `-C'`--format=vertical' List files in columns, sorted vertically. This is the default for `ls' if standard outputis a terminal. It is always the default for the `dir' program. GNU `ls' uses variable width columns to displayas many files as possible in the fewest lines. `-w'`--width=COLS' Assume the screen is COLS columns wide. The default is taken from the terminal settings if possible;otherwise the environment variable `COLUMNS' is used if it is set; otherwise the default is 80. `--color [=WHEN]' Specify whether to use color for distinguishing file types. WHEN may be omitted, or one of: * none - Do not use color at all. This is the default. * auto - Only use color if standard output is a terminal. * always - Always use color. Specifying `--color' and no WHEN is equivalent to `--color=always'. Piping a colorizedlisting through a pager like `more' or `less' usually produces unreadable results. However, using `more -f' does seem to work. That's pretty straight forward, pretty explicit, and matches the description of what Greg has been saying all along. a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
Aidan Van Dyk <aidan@highrise.ca> writes:
> But one of the interesting things is that psql has an is *interactive*
> mode (something the GNU utils don't have to worry about). So *when* you
> choose to figure out your columns is important, and really impacts
> behaviour too.
Well, COLUMNS has no hope of tracking on-the-fly changes of window size,
which is why the ioctl should take precedence over it.
> Correctly documenting it is good, just as the version of GNU ls I have
> on Debian, where I see:
> `-w'
> `--width=COLS'
> Assume the screen is COLS columns wide. The default is taken from
> the terminal settings if possible; otherwise the environment
> variable `COLUMNS' is used if it is set; otherwise the default is
> 80.
Fedora 8 has the same wording in "info ls". Possibly Bruce was reading
the man page, which is not as complete (and admits it).
Experimentation on OS X (BSD clone) shows that its "ls" believes COLUMNS
in preference to ioctl, so there's clearly scope for argument; but on
the whole I think we should follow the GNU behavior. The BSD version's
behavior is full of enormous amounts of historical cruft (and its man
page admits that) --- I suspect the behavior on this point "just grew"
instead of being carefully thought about.
regards, tom lane
Aidan Van Dyk wrote: > I have to admit to using the COLUMNS=... <command> trick myself. > > I do have COLUMNS exported in my terminal, and often to stuff like: > > ls -C | less > > and I expect it to wrap at $COLUMNS (my terminal width) in my pager. > > And since the GNU coreutils is pretty consistent in this regard, I often > export COLUMNS=<xxx> in scripts for cron jobs, reports, etc, to get > output that formats nicely for emails, etc. Interesting. > If I ever wanted the psql wrapped format, I guess *I* would hope that > psql would work that way, simply because that's the way the other tools > I use do it. But I can see that if the other tools work differently, > then there is going to have to be some people (maybe everybody) always > double-checking the psql man page to find out again how it formats. True. > But since I'm not a user wanting the wrapped format, don't cater to my > hypothetical wants. > > But one of the interesting things is that psql has an is *interactive* > mode (something the GNU utils don't have to worry about). So *when* you > choose to figure out your columns is important, and really impacts > behaviour too. > > For instance, if I was doing a query, I often to it interactively first: > SELECT [...] FROM [....] LIMIT 50; > And when I'm sure I have the right values,expressions, column aliases, > etc, I do: > \o /tmp/output > SELECT [...] FROM [...]; > \o > And in this case, I would expect that /tmp/output would have identical > formatting to the LIMITed query I just ran interactively, not matter > what setting I had for format/wrapped/auto/$COLUMNS. The only thing we could do there perhaps is to have psql wrap file output to the terminal width if outputting to a pipe/file, but only from an interactive session, but that is just too odd. If we make psql too automatic it will be hard to explain and have more surprises. > `-w' > `--width=COLS' > Assume the screen is COLS columns wide. The default is taken from > the terminal settings if possible; otherwise the environment > variable `COLUMNS' is used if it is set; otherwise the default is > 80. I just looked at coreutils-6.9 and 5.97 and neither manual has a mention of COLUMNS. Seems this is some Debian manual addition or something. I don't see it on Ubuntu 7.10 either. > That's pretty straight forward, pretty explicit, and matches the > description of what Greg has been saying all along. Agreed. The problem is I have a pretty even split on how this feature should behave so we have go to with the majority and adjust as we get feedback from the field. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Am Dienstag, 6. Mai 2008 schrieb Tom Lane: > Well, COLUMNS has no hope of tracking on-the-fly changes of window size, > which is why the ioctl should take precedence over it. Readline changes the value of COLUMNS on the fly.
Tom Lane wrote: > Aidan Van Dyk <aidan@highrise.ca> writes: > > But one of the interesting things is that psql has an is *interactive* > > mode (something the GNU utils don't have to worry about). So *when* you > > choose to figure out your columns is important, and really impacts > > behaviour too. > > Well, COLUMNS has no hope of tracking on-the-fly changes of window size, > which is why the ioctl should take precedence over it. True, but Peter wanted COLUMNS to be honored over ioctl(): http://archives.postgresql.org/pgsql-hackers/2008-04/msg01853.php > > Correctly documenting it is good, just as the version of GNU ls I have > > on Debian, where I see: > > `-w' > > `--width=COLS' > > Assume the screen is COLS columns wide. The default is taken from > > the terminal settings if possible; otherwise the environment > > variable `COLUMNS' is used if it is set; otherwise the default is > > 80. > > Fedora 8 has the same wording in "info ls". Possibly Bruce was reading > the man page, which is not as complete (and admits it). Oh, you are looking at info. That makes sense now. > Experimentation on OS X (BSD clone) shows that its "ls" believes COLUMNS > in preference to ioctl, so there's clearly scope for argument; but on > the whole I think we should follow the GNU behavior. The BSD version's > behavior is full of enormous amounts of historical cruft (and its man > page admits that) --- I suspect the behavior on this point "just grew" > instead of being carefully thought about. Well, the GNU 'ls' behavior, because it does not check for isatty(), will use COLUMNS mostly for file/pipe output because those are the cases where ioctl() fails. Is that what you want? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Bruce Momjian <bruce@momjian.us> writes:
> I just looked at coreutils-6.9 and 5.97 and neither manual has a mention
> of COLUMNS. Seems this is some Debian manual addition or something. I
> don't see it on Ubuntu 7.10 either.
You're looking in the wrong place. See "info ls".
regards, tom lane
Peter Eisentraut wrote: > Am Dienstag, 6. Mai 2008 schrieb Tom Lane: > > Well, COLUMNS has no hope of tracking on-the-fly changes of window size, > > which is why the ioctl should take precedence over it. > > Readline changes the value of COLUMNS on the fly. Yes, but my patch grabs COLUMNS before we call readline(), so we use the COLUMN value we were invoked with, rather than tracking the changes readline() makes to it. If we use COLUMNS as set by readline, there is no way to pass a COLUMNS value into psql reliably. 'ls' doesn't use readline so it is safe from changes while it is doing its output. I can pull COLUMNS as modified by readline() but I thought no one wanted that. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Peter Eisentraut <peter_e@gmx.net> writes:
> Am Dienstag, 6. Mai 2008 schrieb Tom Lane:
>> Well, COLUMNS has no hope of tracking on-the-fly changes of window size,
>> which is why the ioctl should take precedence over it.
> Readline changes the value of COLUMNS on the fly.
... from the ioctl's results, presumably, so what you are saying is that
if readline is active then this discussion is all moot.
In any case I'd be happier if our behavior on this point was not
dependent on whether readline is built in. A normal person would expect
readline to affect only input behavior, not output behavior.
regards, tom lane
* Bruce Momjian <bruce@momjian.us> [080506 11:59]: > > But one of the interesting things is that psql has an is *interactive* > > mode (something the GNU utils don't have to worry about). So *when* you > > choose to figure out your columns is important, and really impacts > > behaviour too. > > > > For instance, if I was doing a query, I often to it interactively first: > > SELECT [...] FROM [....] LIMIT 50; > > And when I'm sure I have the right values,expressions, column aliases, > > etc, I do: > > \o /tmp/output > > SELECT [...] FROM [...]; > > \o > > And in this case, I would expect that /tmp/output would have identical > > formatting to the LIMITed query I just ran interactively, not matter > > what setting I had for format/wrapped/auto/$COLUMNS. > > The only thing we could do there perhaps is to have psql wrap file > output to the terminal width if outputting to a pipe/file, but only from > an interactive session, but that is just too odd. If we make psql too > automatic it will be hard to explain and have more surprises. Yes, and *this* is the tough part. And *I* think that the control of wrapping/width should be based on psql's output fd/$COLUMNS, not necessarily the query output buffer fd, because I want the interactive output to be identical when I run the query and let psql automatically pipe it through $PAGER and when I run the query and tell psql \o it to pipe it somewhere specific. Of course, all over-ridden by some specific \pset to make it all more complicated ;-) > I just looked at coreutils-6.9 and 5.97 and neither manual has a mention > of COLUMNS. Seems this is some Debian manual addition or something. I > don't see it on Ubuntu 7.10 either. Yes, the man pages point to the Texinfo manual for full documentation:http://www.gnu.org/software/coreutils/manual/html_node/General-output-formatting.html#General-output-formatting a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
Bruce Momjian wrote:
> Updated patch with clearer documentation that matches the above
> behavior:
>
> ftp://momjian.us/pub/postgresql/mypatches/wrap
I found a bug in my patch, particularly related to wrapping to pipes.
Turns out if psql uses the pager internally:
\pset format wrapped
SELECT 1, 2, repeat('a', 80), repeat('b', 80), E'a\nb\nc\nd', 1FROM generate_series(1,50);
it does not wrap to the screen width because of our default behavior of
not wrapping pipe output by default. I had to add an is_pager boolean
parameter to print_aligned_text(). (I tried passing is_pager via
printTableOpt but it is a const.)
I have updated the ftp URL to fix this.
One item not addressed is that the 'wrapped' format wrapping could force
output off the page without the pager being used if it is _near_ a full
screen before wrapping. Of course we don't consider additional lines
that wrap by extending past the right margin for non-wrapped mode, so
perhaps it is OK.
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Patch applied (yea!).
---------------------------------------------------------------------------
Bruce Momjian wrote:
> Bruce Momjian wrote:
> > Updated patch with clearer documentation that matches the above
> > behavior:
> >
> > ftp://momjian.us/pub/postgresql/mypatches/wrap
>
> I found a bug in my patch, particularly related to wrapping to pipes.
> Turns out if psql uses the pager internally:
>
> \pset format wrapped
>
> SELECT 1, 2, repeat('a', 80), repeat('b', 80), E'a\nb\nc\nd', 1
> FROM generate_series(1,50);
>
> it does not wrap to the screen width because of our default behavior of
> not wrapping pipe output by default. I had to add an is_pager boolean
> parameter to print_aligned_text(). (I tried passing is_pager via
> printTableOpt but it is a const.)
>
> I have updated the ftp URL to fix this.
>
> One item not addressed is that the 'wrapped' format wrapping could force
> output off the page without the pager being used if it is _near_ a full
> screen before wrapping. Of course we don't consider additional lines
> that wrap by extending past the right margin for non-wrapped mode, so
> perhaps it is OK.
>
> --
> Bruce Momjian <bruce@momjian.us> http://momjian.us
> EnterpriseDB http://enterprisedb.com
>
> + If your life is a hard drive, Christ can be your backup. +
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
-- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB
http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +