Thread: pasting into psql garbles text

pasting into psql garbles text

From
Merlin Moncure
Date:
I've noticed over a wide variety of operating systems that when you
paste from an application into psql through a terminal (currently
using the default gnome terminal in ubuntu), large pastes tend to get
garbled with some of the input getting truncated.  While working on
functions, this is annoying in the extreme.

Interestingly, the one platform that tends not to suffer from this is
windows so I'm guessing this is a readline problem.  Has anybody else
noticed this? Is there a workaround?

merlin

Re: pasting into psql garbles text

From
Greg Stark
Date:
On Sat, Jun 27, 2009 at 6:19 PM, Merlin Moncure<mmoncure@gmail.com> wrote:
> I've noticed over a wide variety of operating systems that when you
> paste from an application into psql through a terminal (currently
> using the default gnome terminal in ubuntu)

You should get yourself a real terminal. Even Ubuntu hides xterm away somewhere.

Seriously, can you reproduce it in xterm? I can't, it's happily
letting me paste in a 200k query with lines over 300 bytes long.

What application are you pasting this text from? copy/paste in X
requires the application you're pasting from to be surprisingly
involved in the paste operation so perhaps it's behaving oddly.

Also, what version of readline, X, gnome-terminal, and whatever
application you're pasting from are you using?

--
greg
http://mit.edu/~gsstark/resume.pdf

Re: pasting into psql garbles text

From
"Leif B. Kristensen"
Date:
On Saturday 27. June 2009, Merlin Moncure wrote:
>I've noticed over a wide variety of operating systems that when you
>paste from an application into psql through a terminal (currently
>using the default gnome terminal in ubuntu), large pastes tend to get
>garbled with some of the input getting truncated.  While working on
>functions, this is annoying in the extreme.
>
>Interestingly, the one platform that tends not to suffer from this is
>windows so I'm guessing this is a readline problem.  Has anybody else
>noticed this? Is there a workaround?

I'm using the KDE Konsole, and I've not experienced the problem you're
describing. However, pasting large functions or multi-line queries
tends to mess up the readline history, so the next time I press arrow
up, I'll often get garbled lines and lose the prompt.
--
Leif Biberg Kristensen | Registered Linux User #338009
Me And My Database: http://solumslekt.org/blog/

Re: pasting into psql garbles text

From
Merlin Moncure
Date:
On Sat, Jun 27, 2009 at 1:38 PM, Greg Stark<gsstark@mit.edu> wrote:
> On Sat, Jun 27, 2009 at 6:19 PM, Merlin Moncure<mmoncure@gmail.com> wrote:
>> I've noticed over a wide variety of operating systems that when you
>> paste from an application into psql through a terminal (currently
>> using the default gnome terminal in ubuntu)
>
> You should get yourself a real terminal. Even Ubuntu hides xterm away somewhere.

I confirmed that xterm does not exhibit the issue.  So this is at
least partially a terminal problem.  However, xterm does not directly
accept pastes from wine clipboard apparently. it only works if I copy
it to gedit first, then into xterm.

The _only_ terminal application that exhibits this behavior is psql.
specifically, vim does not.

> What application are you pasting this text from? copy/paste in X
> requires the application you're pasting from to be surprisingly
> involved in the paste operation so perhaps it's behaving oddly.

It doesn't matter the source application...it borks regardless if I
copy from Source insight/wine (my normal editor), gedit, or another
gnome terminal.   However, only psql exhibits this issue.  For
example, when I use the buffer editor (mine is set to vim), everything
pastes in normally.

> Also, what version of readline, X, gnome-terminal, and whatever
> application you're pasting from are you using?

Source Insight/wine (or gedit, or whatever)
GNOME gnome-terminal 2.24.1.1
X.Org X Server 1.5.2
libreadline5 5.2-3build1

merlin

Re: pasting into psql garbles text

From
Greg Stark
Date:
On Sat, Jun 27, 2009 at 7:26 PM, Merlin Moncure<mmoncure@gmail.com> wrote:
>
> It doesn't matter the source application...it borks regardless if I
> copy from Source insight/wine (my normal editor), gedit, or another
> gnome terminal.

Are these all gtk apps? Try pasting from an emacs or xterm?

--
greg
http://mit.edu/~gsstark/resume.pdf

Re: pasting into psql garbles text

From
Tom Lane
Date:
Merlin Moncure <mmoncure@gmail.com> writes:
> I confirmed that xterm does not exhibit the issue.  So this is at
> least partially a terminal problem.  However, xterm does not directly
> accept pastes from wine clipboard apparently. it only works if I copy
> it to gedit first, then into xterm.

> The _only_ terminal application that exhibits this behavior is psql.
> specifically, vim does not.

FWIW, I have noticed a limitation on how much can be pasted into psql
without freezing it (not more than a few hundred characters, for me).
It's not "garbled" ... psql just gets stuck and has to be SIGTERM'd.
However, there's enough stuff in the chain that I'm not sure
psql/readline is (solely) to blame --- I'm typically working through
a remote xterm over an ssh'd X11 connection.

            regards, tom lane

Re: pasting into psql garbles text

From
Merlin Moncure
Date:
On Sat, Jun 27, 2009 at 2:56 PM, Greg Stark<gsstark@mit.edu> wrote:
> On Sat, Jun 27, 2009 at 7:26 PM, Merlin Moncure<mmoncure@gmail.com> wrote:
>>
>> It doesn't matter the source application...it borks regardless if I
>> copy from Source insight/wine (my normal editor), gedit, or another
>> gnome terminal.
>
> Are these all gtk apps? Try pasting from an emacs or xterm?

copying from xterm to gtk I can't seem to get working...I can copy
_to_ xterm, but not from it (sigh).  Anyways, that doesn't explain why
the only terminal program that gets garbled is psql.

On Sat, Jun 27, 2009 at 4:28 PM, Tom Lane<tgl@sss.pgh.pa.us> wrote:
> Merlin Moncure <mmoncure@gmail.com> writes:
>> I confirmed that xterm does not exhibit the issue.  So this is at
>> least partially a terminal problem.  However, xterm does not directly
>> accept pastes from wine clipboard apparently. it only works if I copy
>> it to gedit first, then into xterm.
>
>> The _only_ terminal application that exhibits this behavior is psql.
>> specifically, vim does not.
>
> FWIW, I have noticed a limitation on how much can be pasted into psql
> without freezing it (not more than a few hundred characters, for me).
> It's not "garbled" ... psql just gets stuck and has to be SIGTERM'd.
> However, there's enough stuff in the chain that I'm not sure
> psql/readline is (solely) to blame --- I'm typically working through
> a remote xterm over an ssh'd X11 connection.

I'm starting to feel like my problems start appearing at a very fixed
size (like you, a few hundred or so).   Do you see this in other
programs (bash, vim, etc)? or only psql?

merlin

Re: pasting into psql garbles text

From
Tom Lane
Date:
Merlin Moncure <mmoncure@gmail.com> writes:
> I'm starting to feel like my problems start appearing at a very fixed
> size (like you, a few hundred or so).   Do you see this in other
> programs (bash, vim, etc)? or only psql?

I've only noticed it in psql, but there are not that many other programs
that I tend to paste lots of input into.

(experiments...)  Hmm, and another interesting thing is that it only
seems to happen on my HPUX system, which is (intentionally) running a
pretty ancient version of readline ... 4.2a looks like.  My Fedora 10
box with readline 5.2 eats the same amount of pasted text without
indigestion.  What readline version are you using?

            regards, tom lane

Re: pasting into psql garbles text

From
Jasen Betts
Date:
On 2009-06-27, Merlin Moncure <mmoncure@gmail.com> wrote:
> I've noticed over a wide variety of operating systems that when you
> paste from an application into psql through a terminal (currently
> using the default gnome terminal in ubuntu), large pastes tend to get
> garbled with some of the input getting truncated.  While working on
> functions, this is annoying in the extreme.
>
> Interestingly, the one platform that tends not to suffer from this is
> windows so I'm guessing this is a readline problem.  Has anybody else
> noticed this? Is there a workaround?

one way to disable readline is to use cat|psql instead of psql


Re: pasting into psql garbles text

From
Роман Маширов
Date:
In my case (FreeBSD 6 and 7) psql does not freezes, but for some reason part of pasted code get corrupted when size of
thecode relatively big. It seems like some timeout used, or buffer overflows:<br /><br /> 1. locally with xterm for the
firsttime pasting function of 9K:<br /> base=> create or replace function football_recalc_match_stat(bigint) returns
voidas $$<br />     declare <br /><br /> < here i see rows 1-52 of function without 'base$>' prefix ><br /><br
/>                where football_match = match and match_event_type in (3, 12)base$>     declare <br />
base$>        match alias for $1;<br /><br /> < here i see rows 1-117 of function with 'base$>' prefix ><br
/>< here i see rows 166-till the end of the func with 'base$>' prefix><br /><br /> and got error in function
bodyat the row 118.<br /><br /><br /> 2-10 locally and via ssh to localhost with xterm -- no problem.<br /><br /><br />
11- to remote host via ssh with xterm<br /> base=> create or replace function football_recalc_match_stat(bigint)
returnsvoid as $$<br />     declare <br /><br /> < here i see rows 1-52 of function without 'base$>' prefix
><br/><br />                 where football_match = match and match_event_type in (3, 12)sovsport$>     declare
<br/> base$>         match alias for $1;<br /><br /> < here i see rows 1-23 of function with 'base$>' prefix,
lastrow truncated ><br /> base$><br /> and that's all<br /><br /><br /> 11 stable reproduced for several times
withproblems on the same rows.<br /><br /> uname -a<br /> FreeBSD  6.3-RELEASE-p2 FreeBSD 6.3-RELEASE-p2 #0: Wed Sep  3
09:41:48MSD 2008 i386<br /><br /> set | grep LANG<br /> LANG=ru_RU.UTF-8<br /><br /> pkg_info -r
postgresql-client-8.2.7<br/> Information for postgresql-client-8.2.7:<br /><br /> Depends on:<br /> Dependency:
libiconv-1.11_1<br/> Dependency: gettext-0.16.1_3<br /><br /><br /> Tom Lane wrote: <blockquote
cite="mid:19349.1246137153@sss.pgh.pa.us"type="cite"><pre wrap="">Merlin Moncure <a class="moz-txt-link-rfc2396E"
href="mailto:mmoncure@gmail.com"><mmoncure@gmail.com></a>writes: </pre><blockquote type="cite"><pre wrap="">I'm
startingto feel like my problems start appearing at a very fixed 
size (like you, a few hundred or so).   Do you see this in other
programs (bash, vim, etc)? or only psql?   </pre></blockquote><pre wrap="">
I've only noticed it in psql, but there are not that many other programs
that I tend to paste lots of input into.

(experiments...)  Hmm, and another interesting thing is that it only
seems to happen on my HPUX system, which is (intentionally) running a
pretty ancient version of readline ... 4.2a looks like.  My Fedora 10
box with readline 5.2 eats the same amount of pasted text without
indigestion.  What readline version are you using?
        regards, tom lane
 </pre></blockquote><br />

Re: pasting into psql garbles text

From
Cédric Villemain
Date:
Le samedi 27 juin 2009, Merlin Moncure a écrit :
> I've noticed over a wide variety of operating systems that when you
> paste from an application into psql through a terminal (currently
> using the default gnome terminal in ubuntu), large pastes tend to get
> garbled with some of the input getting truncated.  While working on
> functions, this is annoying in the extreme.

I had notice the same error with konsole :
http://bugs.kde.org/show_bug.cgi?id=150957



>
> Interestingly, the one platform that tends not to suffer from this is
> windows so I'm guessing this is a readline problem.  Has anybody else
> noticed this? Is there a workaround?
>
> merlin


--
----
Cédric Villemain
Administrateur de Base de Données
Cel: +33 (0)6 74 15 56 53
http://dalibo.com - http://dalibo.org

Attachment

Re: pasting into psql garbles text

From
Merlin Moncure
Date:
On Sat, Jun 27, 2009 at 5:12 PM, Tom Lane<tgl@sss.pgh.pa.us> wrote:
> Merlin Moncure <mmoncure@gmail.com> writes:
>> I'm starting to feel like my problems start appearing at a very fixed
>> size (like you, a few hundred or so).   Do you see this in other
>> programs (bash, vim, etc)? or only psql?
>
> I've only noticed it in psql, but there are not that many other programs
> that I tend to paste lots of input into.
>
> (experiments...)  Hmm, and another interesting thing is that it only
> seems to happen on my HPUX system, which is (intentionally) running a
> pretty ancient version of readline ... 4.2a looks like.  My Fedora 10
> box with readline 5.2 eats the same amount of pasted text without
> indigestion.  What readline version are you using?

I'm currently using 'ubuntu intrepid', which has very modern
everything (readline 5.2-3 build1).  I've noticed this problem since
the beginning of time now.  This is definitely a 'psql + something'
problem...I can paste into psql query buffer editor (vim) following a
/r/e with no problems, but not directly into psql itself.

Another interesting point:  the problem manifests with pasts over a
certain size, but tends to bork at a particular point.  The size of
the psql window affects this...smaller windows are more tolerant of
larger (but still not very large) pastes.

merlin

Re: pasting into psql garbles text

From
Scott Mead
Date:


On Mon, Jun 29, 2009 at 10:49 AM, Cédric Villemain <cedric.villemain@dalibo.com> wrote:
Le samedi 27 juin 2009, Merlin Moncure a écrit :

>
> Interestingly, the one platform that tends not to suffer from this is
> windows so I'm guessing this is a readline problem.  Has anybody else
> noticed this? Is there a workaround?

  Whenever I have a huge chunk of text to paste into psql, I'll drop to an editor with \e, then paste it, the close the editor (I have $EDITOR=vim).  Just a thought.

--Scott