Thread: pasting into psql garbles text
I've noticed over a wide variety of operating systems that when you paste from an application into psql through a terminal (currently using the default gnome terminal in ubuntu), large pastes tend to get garbled with some of the input getting truncated. While working on functions, this is annoying in the extreme. Interestingly, the one platform that tends not to suffer from this is windows so I'm guessing this is a readline problem. Has anybody else noticed this? Is there a workaround? merlin
On Sat, Jun 27, 2009 at 6:19 PM, Merlin Moncure<mmoncure@gmail.com> wrote: > I've noticed over a wide variety of operating systems that when you > paste from an application into psql through a terminal (currently > using the default gnome terminal in ubuntu) You should get yourself a real terminal. Even Ubuntu hides xterm away somewhere. Seriously, can you reproduce it in xterm? I can't, it's happily letting me paste in a 200k query with lines over 300 bytes long. What application are you pasting this text from? copy/paste in X requires the application you're pasting from to be surprisingly involved in the paste operation so perhaps it's behaving oddly. Also, what version of readline, X, gnome-terminal, and whatever application you're pasting from are you using? -- greg http://mit.edu/~gsstark/resume.pdf
On Saturday 27. June 2009, Merlin Moncure wrote: >I've noticed over a wide variety of operating systems that when you >paste from an application into psql through a terminal (currently >using the default gnome terminal in ubuntu), large pastes tend to get >garbled with some of the input getting truncated. While working on >functions, this is annoying in the extreme. > >Interestingly, the one platform that tends not to suffer from this is >windows so I'm guessing this is a readline problem. Has anybody else >noticed this? Is there a workaround? I'm using the KDE Konsole, and I've not experienced the problem you're describing. However, pasting large functions or multi-line queries tends to mess up the readline history, so the next time I press arrow up, I'll often get garbled lines and lose the prompt. -- Leif Biberg Kristensen | Registered Linux User #338009 Me And My Database: http://solumslekt.org/blog/
On Sat, Jun 27, 2009 at 1:38 PM, Greg Stark<gsstark@mit.edu> wrote: > On Sat, Jun 27, 2009 at 6:19 PM, Merlin Moncure<mmoncure@gmail.com> wrote: >> I've noticed over a wide variety of operating systems that when you >> paste from an application into psql through a terminal (currently >> using the default gnome terminal in ubuntu) > > You should get yourself a real terminal. Even Ubuntu hides xterm away somewhere. I confirmed that xterm does not exhibit the issue. So this is at least partially a terminal problem. However, xterm does not directly accept pastes from wine clipboard apparently. it only works if I copy it to gedit first, then into xterm. The _only_ terminal application that exhibits this behavior is psql. specifically, vim does not. > What application are you pasting this text from? copy/paste in X > requires the application you're pasting from to be surprisingly > involved in the paste operation so perhaps it's behaving oddly. It doesn't matter the source application...it borks regardless if I copy from Source insight/wine (my normal editor), gedit, or another gnome terminal. However, only psql exhibits this issue. For example, when I use the buffer editor (mine is set to vim), everything pastes in normally. > Also, what version of readline, X, gnome-terminal, and whatever > application you're pasting from are you using? Source Insight/wine (or gedit, or whatever) GNOME gnome-terminal 2.24.1.1 X.Org X Server 1.5.2 libreadline5 5.2-3build1 merlin
On Sat, Jun 27, 2009 at 7:26 PM, Merlin Moncure<mmoncure@gmail.com> wrote: > > It doesn't matter the source application...it borks regardless if I > copy from Source insight/wine (my normal editor), gedit, or another > gnome terminal. Are these all gtk apps? Try pasting from an emacs or xterm? -- greg http://mit.edu/~gsstark/resume.pdf
Merlin Moncure <mmoncure@gmail.com> writes:
> I confirmed that xterm does not exhibit the issue. So this is at
> least partially a terminal problem. However, xterm does not directly
> accept pastes from wine clipboard apparently. it only works if I copy
> it to gedit first, then into xterm.
> The _only_ terminal application that exhibits this behavior is psql.
> specifically, vim does not.
FWIW, I have noticed a limitation on how much can be pasted into psql
without freezing it (not more than a few hundred characters, for me).
It's not "garbled" ... psql just gets stuck and has to be SIGTERM'd.
However, there's enough stuff in the chain that I'm not sure
psql/readline is (solely) to blame --- I'm typically working through
a remote xterm over an ssh'd X11 connection.
regards, tom lane
On Sat, Jun 27, 2009 at 2:56 PM, Greg Stark<gsstark@mit.edu> wrote: > On Sat, Jun 27, 2009 at 7:26 PM, Merlin Moncure<mmoncure@gmail.com> wrote: >> >> It doesn't matter the source application...it borks regardless if I >> copy from Source insight/wine (my normal editor), gedit, or another >> gnome terminal. > > Are these all gtk apps? Try pasting from an emacs or xterm? copying from xterm to gtk I can't seem to get working...I can copy _to_ xterm, but not from it (sigh). Anyways, that doesn't explain why the only terminal program that gets garbled is psql. On Sat, Jun 27, 2009 at 4:28 PM, Tom Lane<tgl@sss.pgh.pa.us> wrote: > Merlin Moncure <mmoncure@gmail.com> writes: >> I confirmed that xterm does not exhibit the issue. So this is at >> least partially a terminal problem. However, xterm does not directly >> accept pastes from wine clipboard apparently. it only works if I copy >> it to gedit first, then into xterm. > >> The _only_ terminal application that exhibits this behavior is psql. >> specifically, vim does not. > > FWIW, I have noticed a limitation on how much can be pasted into psql > without freezing it (not more than a few hundred characters, for me). > It's not "garbled" ... psql just gets stuck and has to be SIGTERM'd. > However, there's enough stuff in the chain that I'm not sure > psql/readline is (solely) to blame --- I'm typically working through > a remote xterm over an ssh'd X11 connection. I'm starting to feel like my problems start appearing at a very fixed size (like you, a few hundred or so). Do you see this in other programs (bash, vim, etc)? or only psql? merlin
Merlin Moncure <mmoncure@gmail.com> writes:
> I'm starting to feel like my problems start appearing at a very fixed
> size (like you, a few hundred or so). Do you see this in other
> programs (bash, vim, etc)? or only psql?
I've only noticed it in psql, but there are not that many other programs
that I tend to paste lots of input into.
(experiments...) Hmm, and another interesting thing is that it only
seems to happen on my HPUX system, which is (intentionally) running a
pretty ancient version of readline ... 4.2a looks like. My Fedora 10
box with readline 5.2 eats the same amount of pasted text without
indigestion. What readline version are you using?
regards, tom lane
On 2009-06-27, Merlin Moncure <mmoncure@gmail.com> wrote: > I've noticed over a wide variety of operating systems that when you > paste from an application into psql through a terminal (currently > using the default gnome terminal in ubuntu), large pastes tend to get > garbled with some of the input getting truncated. While working on > functions, this is annoying in the extreme. > > Interestingly, the one platform that tends not to suffer from this is > windows so I'm guessing this is a readline problem. Has anybody else > noticed this? Is there a workaround? one way to disable readline is to use cat|psql instead of psql
In my case (FreeBSD 6 and 7) psql does not freezes, but for some reason part of pasted code get corrupted when size of
thecode relatively big. It seems like some timeout used, or buffer overflows:<br /><br /> 1. locally with xterm for the
firsttime pasting function of 9K:<br /> base=> create or replace function football_recalc_match_stat(bigint) returns
voidas $$<br /> declare <br /><br /> < here i see rows 1-52 of function without 'base$>' prefix ><br /><br
/> where football_match = match and match_event_type in (3, 12)base$> declare <br />
base$> match alias for $1;<br /><br /> < here i see rows 1-117 of function with 'base$>' prefix ><br
/>< here i see rows 166-till the end of the func with 'base$>' prefix><br /><br /> and got error in function
bodyat the row 118.<br /><br /><br /> 2-10 locally and via ssh to localhost with xterm -- no problem.<br /><br /><br />
11- to remote host via ssh with xterm<br /> base=> create or replace function football_recalc_match_stat(bigint)
returnsvoid as $$<br /> declare <br /><br /> < here i see rows 1-52 of function without 'base$>' prefix
><br/><br /> where football_match = match and match_event_type in (3, 12)sovsport$> declare
<br/> base$> match alias for $1;<br /><br /> < here i see rows 1-23 of function with 'base$>' prefix,
lastrow truncated ><br /> base$><br /> and that's all<br /><br /><br /> 11 stable reproduced for several times
withproblems on the same rows.<br /><br /> uname -a<br /> FreeBSD 6.3-RELEASE-p2 FreeBSD 6.3-RELEASE-p2 #0: Wed Sep 3
09:41:48MSD 2008 i386<br /><br /> set | grep LANG<br /> LANG=ru_RU.UTF-8<br /><br /> pkg_info -r
postgresql-client-8.2.7<br/> Information for postgresql-client-8.2.7:<br /><br /> Depends on:<br /> Dependency:
libiconv-1.11_1<br/> Dependency: gettext-0.16.1_3<br /><br /><br /> Tom Lane wrote: <blockquote
cite="mid:19349.1246137153@sss.pgh.pa.us"type="cite"><pre wrap="">Merlin Moncure <a class="moz-txt-link-rfc2396E"
href="mailto:mmoncure@gmail.com"><mmoncure@gmail.com></a>writes: </pre><blockquote type="cite"><pre wrap="">I'm
startingto feel like my problems start appearing at a very fixed
size (like you, a few hundred or so). Do you see this in other
programs (bash, vim, etc)? or only psql? </pre></blockquote><pre wrap="">
I've only noticed it in psql, but there are not that many other programs
that I tend to paste lots of input into.
(experiments...) Hmm, and another interesting thing is that it only
seems to happen on my HPUX system, which is (intentionally) running a
pretty ancient version of readline ... 4.2a looks like. My Fedora 10
box with readline 5.2 eats the same amount of pasted text without
indigestion. What readline version are you using?
regards, tom lane
</pre></blockquote><br />
Le samedi 27 juin 2009, Merlin Moncure a écrit : > I've noticed over a wide variety of operating systems that when you > paste from an application into psql through a terminal (currently > using the default gnome terminal in ubuntu), large pastes tend to get > garbled with some of the input getting truncated. While working on > functions, this is annoying in the extreme. I had notice the same error with konsole : http://bugs.kde.org/show_bug.cgi?id=150957 > > Interestingly, the one platform that tends not to suffer from this is > windows so I'm guessing this is a readline problem. Has anybody else > noticed this? Is there a workaround? > > merlin -- ---- Cédric Villemain Administrateur de Base de Données Cel: +33 (0)6 74 15 56 53 http://dalibo.com - http://dalibo.org
Attachment
On Sat, Jun 27, 2009 at 5:12 PM, Tom Lane<tgl@sss.pgh.pa.us> wrote: > Merlin Moncure <mmoncure@gmail.com> writes: >> I'm starting to feel like my problems start appearing at a very fixed >> size (like you, a few hundred or so). Do you see this in other >> programs (bash, vim, etc)? or only psql? > > I've only noticed it in psql, but there are not that many other programs > that I tend to paste lots of input into. > > (experiments...) Hmm, and another interesting thing is that it only > seems to happen on my HPUX system, which is (intentionally) running a > pretty ancient version of readline ... 4.2a looks like. My Fedora 10 > box with readline 5.2 eats the same amount of pasted text without > indigestion. What readline version are you using? I'm currently using 'ubuntu intrepid', which has very modern everything (readline 5.2-3 build1). I've noticed this problem since the beginning of time now. This is definitely a 'psql + something' problem...I can paste into psql query buffer editor (vim) following a /r/e with no problems, but not directly into psql itself. Another interesting point: the problem manifests with pasts over a certain size, but tends to bork at a particular point. The size of the psql window affects this...smaller windows are more tolerant of larger (but still not very large) pastes. merlin
On Mon, Jun 29, 2009 at 10:49 AM, Cédric Villemain <cedric.villemain@dalibo.com> wrote:
Le samedi 27 juin 2009, Merlin Moncure a écrit :>
> Interestingly, the one platform that tends not to suffer from this is
> windows so I'm guessing this is a readline problem. Has anybody else
> noticed this? Is there a workaround?
Whenever I have a huge chunk of text to paste into psql, I'll drop to an editor with \e, then paste it, the close the editor (I have $EDITOR=vim). Just a thought.
--Scott