Thread: ERROR: failed to add item to the index page

Andreas Joseph Krogh <andreas@visena.com> writes:
> With master (3dbb317d32f4f084ef7badaed8ef36f5c9b85fe6) I'm getting this: 
> visena=# CREATE INDEX origo_email_part_hdrvl_value_idx ON 
> public.origo_email_part_headervalue USING btree 
> (lower(substr((header_value)::text, 0, 1000)) varchar_pattern_ops);
>  psql: ERROR: failed to add item to the index page

Hm, your example works for me on HEAD.

Usually, the first thing to suspect when you're tracking HEAD and get
bizarre failures is that you have a messed-up build.  Before spending
any time diagnosing more carefully, do "make distclean", reconfigure,
rebuild, reinstall, then see if problem is still there.

(In theory, you can avoid this sort of failure with appropriate use
of --enable-depend, but personally I don't trust that too much.
I find that with ccache + autoconf cache + parallel build, rebuilding
completely is fast enough that it's something I just do routinely
after any git pull.  I'd rather use up my remaining brain cells on
other kinds of problems...)

            regards, tom lane

Andreas Joseph Krogh <andreas@visena.com> writes:
> With master (3dbb317d32f4f084ef7badaed8ef36f5c9b85fe6) I'm getting this:
> visena=# CREATE INDEX origo_email_part_hdrvl_value_idx ON
> public.origo_email_part_headervalue USING btree
> (lower(substr((header_value)::text, 0, 1000)) varchar_pattern_ops);
>  psql: ERROR: failed to add item to the index page

Hm, your example works for me on HEAD.

Usually, the first thing to suspect when you're tracking HEAD and get
bizarre failures is that you have a messed-up build.  Before spending
any time diagnosing more carefully, do "make distclean", reconfigure,
rebuild, reinstall, then see if problem is still there.

(In theory, you can avoid this sort of failure with appropriate use
of --enable-depend, but personally I don't trust that too much.
I find that with ccache + autoconf cache + parallel build, rebuilding
completely is fast enough that it's something I just do routinely
after any git pull.  I'd rather use up my remaining brain cells on
other kinds of problems...)

regards, tom lane

Andreas Joseph Krogh <andreas@visena.com> writes:
> I built with this: make distclean && ./configure
> --prefix=$HOME/programs/postgresql-master --with-openssl --with-llvm && make -j
> 8 install-world-contrib-recurse install-world-doc-recurse

--with-llvm, eh?  Does it reproduce without that?  What platform is
this on, what LLVM version?

> I'll see if I can create a self contained example.

Please.

            regards, tom lane

Andreas Joseph Krogh <andreas@visena.com> writes:
> I built with this: make distclean && ./configure
> --prefix=$HOME/programs/postgresql-master --with-openssl --with-llvm && make -j
> 8 install-world-contrib-recurse install-world-doc-recurse

--with-llvm, eh?  Does it reproduce without that?  What platform is
this on, what LLVM version?

> I'll see if I can create a self contained example.

Please.

regards, tom lane

På tirsdag 30. april 2019 kl. 15:53:50, skrev Tom Lane <tgl@sss.pgh.pa.us>:

Andreas Joseph Krogh <andreas@visena.com> writes:
> I built with this: make distclean && ./configure
> --prefix=$HOME/programs/postgresql-master --with-openssl --with-llvm && make -j
> 8 install-world-contrib-recurse install-world-doc-recurse

--with-llvm, eh?  Does it reproduce without that?  What platform is
this on, what LLVM version?

> I'll see if I can create a self contained example.

Please.

regards, tom lane

 

Ubuntu 19.04

$ llvm-config --version
8.0.0
 

"--with-llvm" was something I had from when pg-11 was master. It might not be needed anymore?

 

I'm trying a fresh build without --with-llvm and reload of data now.

Please fix or abstain from using the MUA that produces this monstrosity
of a Subject: "Sv: Sv: Re: Sv: Re: ERROR: failed to add item to the
index page"


cheers


andrew


-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

[snip]

 

Yep, happens without --with-llvm also.

 

I'll try to load only the necessary table(s) to reproduce.

On Tue, Apr 30, 2019 at 9:44 AM Andreas Joseph Krogh
<andreas@visena.com> wrote:
> I have a 1.4GB dump (only one table) which reliably reproduces this error.
> Shall I share it off-list?

I would be quite interested in this, too, since there is a chance that
it's my bug.

-- 
Peter Geoghegan

Andreas Joseph Krogh <andreas@visena.com> writes:
> I have a 1.4GB dump (only one table) which reliably reproduces this error.
> Shall I share it off-list? --

That's awfully large :-(.  How do you have in mind to transmit it?

Maybe you could write a short script that generates dummy data
to reproduce the problem?

            regards, tom lane

On Tue, Apr 30, 2019 at 9:47 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Andreas Joseph Krogh <andreas@visena.com> writes:
> > I have a 1.4GB dump (only one table) which reliably reproduces this error.
> > Shall I share it off-list? --
>
> That's awfully large :-(.  How do you have in mind to transmit it?

I've send dumps that were larger than that by providing a Google drive
link. Something like that should work reasonably well.

-- 
Peter Geoghegan

On Tue, Apr 30, 2019 at 9:47 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Andreas Joseph Krogh <andreas@visena.com> writes:
> > I have a 1.4GB dump (only one table) which reliably reproduces this error.
> > Shall I share it off-list? --
>
> That's awfully large :-(.  How do you have in mind to transmit it?

I've send dumps that were larger than that by providing a Google drive
link. Something like that should work reasonably well.

On Tue, Apr 30, 2019 at 9:56 AM Andreas Joseph Krogh <andreas@visena.com> wrote:
> I've sent you guys a link (Google Drive) off-list.

I'll start investigating the problem right away.

Thanks
-- 
Peter Geoghegan

On Tue, Apr 30, 2019 at 9:59 AM Peter Geoghegan <pg@bowt.ie> wrote:
> I'll start investigating the problem right away.

I have found what the problem is. I simply neglected to make a
conservative assumption about suffix truncation needing to add a heap
TID to a leaf page's new high key in nbtsort.c (following commit
dd299df8189), even though I didn't make the same mistake in
nbtsplitloc.c. Not sure how I managed to make such a basic error.

Andreas' test case works fine with the attached patch. I won't push a
fix for this today.

-- 
Peter Geoghegan

On Tue, Apr 30, 2019 at 9:59 AM Peter Geoghegan <pg@bowt.ie> wrote:
> I'll start investigating the problem right away.

I have found what the problem is. I simply neglected to make a
conservative assumption about suffix truncation needing to add a heap
TID to a leaf page's new high key in nbtsort.c (following commit
dd299df8189), even though I didn't make the same mistake in
nbtsplitloc.c. Not sure how I managed to make such a basic error.

Andreas' test case works fine with the attached patch. I won't push a
fix for this today.

--
Peter Geoghegan

On Tue, Apr 30, 2019 at 11:54 AM Andreas Joseph Krogh
<andreas@visena.com> wrote:
> Nice, thanks!

Thanks for the report!

-- 
Peter Geoghegan

Hi,

On 2019-04-30 20:54:45 +0200, Andreas Joseph Krogh wrote:
> På tirsdag 30. april 2019 kl. 19:58:31, skrev Peter Geoghegan <pg@bowt.ie 
> <mailto:pg@bowt.ie>>: On Tue, Apr 30, 2019 at 9:59 AM Peter Geoghegan 
> <pg@bowt.ie> wrote:
>  > I'll start investigating the problem right away.
> 
>  I have found what the problem is. I simply neglected to make a
>  conservative assumption about suffix truncation needing to add a heap
>  TID to a leaf page's new high key in nbtsort.c (following commit
>  dd299df8189), even though I didn't make the same mistake in
>  nbtsplitloc.c. Not sure how I managed to make such a basic error.
> 
>  Andreas' test case works fine with the attached patch. I won't push a
>  fix for this today.
> 
>  --
>  Peter Geoghegan Nice, thanks! --
>  Andreas Joseph Krogh

Andreas, unfortunately your emails are pretty unreadable. Check the
quoted email, and the web archive:

https://www.postgresql.org/message-id/VisenaEmail.41.51d7719d814a1f54.16a6f98a5e9%40tc7-visena

Greetings,

Andres Freund

[...]
Andreas, unfortunately your emails are pretty unreadable. Check the
quoted email, and the web archive:

https://www.postgresql.org/message-id/VisenaEmail.41.51d7719d814a1f54.16a6f98a5e9%40tc7-visena

Greetings,

Andres Freund
 

Hi,

On 2019-04-30 21:23:21 +0200, Andreas Joseph Krogh wrote:
> På tirsdag 30. april 2019 kl. 20:59:43, skrev Andres Freund <andres@anarazel.de 
> <mailto:andres@anarazel.de>>: [...]
>  Andreas, unfortunately your emails are pretty unreadable. Check the
>  quoted email, and the web archive:
> 
>  
> https://www.postgresql.org/message-id/VisenaEmail.41.51d7719d814a1f54.16a6f98a5e9%40tc7-visena
> 
>  Greetings,
> 
>  Andres Freund

> I know that the text-version is quite unreadable, especially when quoting. My 
> MUA is web-based and uses CKEditor for composing, and it doesn't care much to 
> try to format the text/plain version (I know because I've written it, yes and 
> have yet to fix the Re: Sv: Re: Sv: subject issue...). But it has tons of 
> benefits CRM- and usage-wise so I prefer to use it. But - how use text/plain 
> these days:-) --

The standard on pg lists is to write in a manner that's usable for both
text mail readers and the archive. Doesn't terribly matter to the
occasional one-off poster on -general, but you're not that...  So please
try to write readable mails for the PG lists.

Greetings,

Andres Freund

On Tue, Apr 30, 2019 at 10:58 AM Peter Geoghegan <pg@bowt.ie> wrote:j
> I have found what the problem is. I simply neglected to make a
> conservative assumption about suffix truncation needing to add a heap
> TID to a leaf page's new high key in nbtsort.c (following commit
> dd299df8189), even though I didn't make the same mistake in
> nbtsplitloc.c. Not sure how I managed to make such a basic error.

Attached is a much more polished version of the same patch. I tried to
make clear how the "page full" test (the test that has been fixed to
take heap TID space for high key into account) is related to other
close-by code, such as the tuple space limit budget within
_bt_check_third_page(), and the code that sets up an actual call to
_bt_truncate().

I'll wait a few days before pushing this. This version doesn't feel
too far off being committable. I tested it with some of the CREATE
INDEX tests that I developed during development of the nbtree unique
keys project, including a test with tuples that are precisely at the
1/3 of a page threshold. The new definition of 1/3 of a page takes
high key heap TID overhead into account -- see _bt_check_third_page().

-- 
Peter Geoghegan

On Tue, Apr 30, 2019 at 6:28 PM Peter Geoghegan <pg@bowt.ie> wrote:
> Attached is a much more polished version of the same patch. I tried to
> make clear how the "page full" test (the test that has been fixed to
> take heap TID space for high key into account) is related to other
> close-by code, such as the tuple space limit budget within
> _bt_check_third_page(), and the code that sets up an actual call to
> _bt_truncate().

Pushed, though final version does the test a little differently. It
adds the required heap TID space to itupsz, rather than subtracting it
from pgspc. This is actually representative of the underlying logic,
and avoids unsigned underflow.

-- 
Peter Geoghegan