Thread: libedit history seems to be misbehaving / broken

Hi,

I accidentally built master with libedit (instead of the readline I use
most of the time, due to missing readline-devel package). And for a
while it was working fine, but then I realized the history is not
working anymore, and is broken in a weird way :-(

I'm not terribly familiar with libedit, so can't say if it's a bug in
libedit, or if we're just using it wrong in some way. I investigated
that while on a flight from pgconf.eu, but can't look into this further.
So let me at least share the behavior I observed, and what I found.

Note: All of this is with libedit-3.1-53.20240808cvs.fc40.x86_64, which
is what I currently get in Fedora 40.

Initially, history seems to work - more or less. Except for the detail
that we always "repeat" the last command from the history. So if you do
"SELECT 1" then the first query added to the history by the *next* psql
session is going to be "SELECT 1". And this happens even if you don't do
anything in psql. So if you enter & immediately exit 10x, you get 10
copies of the last query. Which is rather weird, and it also makes it
less convenient to walk the history (ctrl-r search is fine, ofc).

But there comes the following issue. Libedit identifies the history file
by adding _HiStOrY_V2_ as the first line. But it also limits the history
to 500 lines, and after adding the 501st line, it gets confused,
attempts to enforce the 500-line limit, and removes the first line,
which is the _HiStOrY_V2_ header. At which point the history stops to
work, more or less - psql can't access the history, apparently due the
header missing. We still append new entries at the end, can access
earlier commands from the same session, but nothing from the file.

And it stays broken forever, of course :-(


regards

-- 
Tomas Vondra

Tomas Vondra <tomas@vondra.me> writes:
> I'm not terribly familiar with libedit, so can't say if it's a bug in
> libedit, or if we're just using it wrong in some way. I investigated
> that while on a flight from pgconf.eu, but can't look into this further.
> So let me at least share the behavior I observed, and what I found.
> Note: All of this is with libedit-3.1-53.20240808cvs.fc40.x86_64, which
> is what I currently get in Fedora 40.

libedit has been a hot mess for a long time.  psql's input.c has
workarounds for assorted bugs that it has or had in prior versions,
and what I suspect is that latest libedit has a different set of bugs.

FWIW, I don't observe any particular misbehavior with the very ancient
libedit that macOS ships.  On Fedora 39, I notice something related to
what you say: it seems like the "\q" ending a session gets repeated
into .psql_history by the next session.  I'm surprised that it's not
exactly the same as your results though, because it seems to be the
same source version:

$ rpm -q libedit
libedit-3.1-53.20240808cvs.fc39.x86_64

Didn't try the too-many-lines behavior, but it looks like that
is implemented totally by history_truncate_file(), so if that's
busted it's surely their fault.

            regards, tom lane

On 10/27/24 20:03, Tom Lane wrote:
> Tomas Vondra <tomas@vondra.me> writes:
>> I'm not terribly familiar with libedit, so can't say if it's a bug in
>> libedit, or if we're just using it wrong in some way. I investigated
>> that while on a flight from pgconf.eu, but can't look into this further.
>> So let me at least share the behavior I observed, and what I found.
>> Note: All of this is with libedit-3.1-53.20240808cvs.fc40.x86_64, which
>> is what I currently get in Fedora 40.
> 
> libedit has been a hot mess for a long time.  psql's input.c has
> workarounds for assorted bugs that it has or had in prior versions,
> and what I suspect is that latest libedit has a different set of bugs.
> 
> FWIW, I don't observe any particular misbehavior with the very ancient
> libedit that macOS ships.  On Fedora 39, I notice something related to
> what you say: it seems like the "\q" ending a session gets repeated
> into .psql_history by the next session.  I'm surprised that it's not
> exactly the same as your results though, because it seems to be the
> same source version:
> 
> $ rpm -q libedit
> libedit-3.1-53.20240808cvs.fc39.x86_64
> 

That's probably because I usually terminate psql by Ctrl-D, not by
typing "\q". But yeah, if I use "\q" it gets added twice.

> Didn't try the too-many-lines behavior, but it looks like that
> is implemented totally by history_truncate_file(), so if that's
> busted it's surely their fault.
> 

Sounds likely. What surprises me a bit, I haven't found any reports of
particularly similar bugs ... I'd have expected other people to hit this
too, but who knows.

-- 
Tomas Vondra