Postgres Pro
Downloads
Home   >   mailing lists

Thread: Error creating gin index on jsonb columns

Error creating gin index on jsonb columns

From
Glenn Zhu
Date:
We are getting an error on the following statement:

CREATE INDEX CONCURRENTLY customer_jsonb_fields_idx ON customer USING gin (jsonb_fields jsonb_path_ops);

ERROR:  invalid memory alloc request size 2013265920

Anyone know what is causing it?  It does not seem to be data corruption as when we deploy to multiple databases we got exactly the same error.

I did try to set maintenance_work_mam to 4GB and still the same error occurred.

Thank you.
-glenn
--
Glenn Zhu
SaaS Operations | ❖ Medallia, Inc.

Re: Error creating gin index on jsonb columns

From
Tom Lane
Date:
Glenn Zhu <gzhu@medallia.com> writes:
> We are getting an error on the following statement:
> CREATE INDEX CONCURRENTLY customer_jsonb_fields_idx ON customer USING gin
> (jsonb_fields jsonb_path_ops);

> ERROR:  invalid memory alloc request size 2013265920

> Anyone know what is causing it?

Sounds like a bug from here.  What PG version is this exactly?  If it's
not the latest minor releases, try updating.  If it's still there, see if
you can extract a test case that you can share.
        regards, tom lane

Re: Error creating gin index on jsonb columns

From
Glenn Zhu
Date:
Hi Tom,

Thanks for the reply.

We are currently running 9.4.4.  9.4.5 notes have two references to gin index but does not seem to address the issue.

We are however able to create same index on some other databases.  So it maybe data related or size of table related?  So far, the failed cases reported are from large tables, with number of rows between 150 to 350 millions.

What would you need in terms of test cases?

Thank you very much.
-glenn


On Fri, Oct 16, 2015 at 6:28 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Glenn Zhu <gzhu@medallia.com> writes:
> We are getting an error on the following statement:
> CREATE INDEX CONCURRENTLY customer_jsonb_fields_idx ON customer USING gin
> (jsonb_fields jsonb_path_ops);

> ERROR:  invalid memory alloc request size 2013265920

> Anyone know what is causing it?

Sounds like a bug from here.  What PG version is this exactly?  If it's
not the latest minor releases, try updating.  If it's still there, see if
you can extract a test case that you can share.

                        regards, tom lane



--
Glenn Zhu
SaaS Operations | ❖ Medallia, Inc.
  

Re: Error creating gin index on jsonb columns

From
Tom Lane
Date:
Glenn Zhu <gzhu@medallia.com> writes:
> We are currently running 9.4.4.  9.4.5 notes have two references to gin
> index but does not seem to address the issue.

> We are however able to create same index on some other databases.  So it
> maybe data related or size of table related?

I'd guess that it's triggered by a specific data item or set of data
items.  Doubt it has anything to do with table size per se.  The quoted
value is 0x78000000, which makes me think that something is
misinterpreting a plain C string as a varlena value (with a length word),
or something along that line.
        regards, tom lane

Re: Error creating gin index on jsonb columns

From
Glenn Zhu
Date:
Currently, after hitting the error, the indexes were still created but marked with status of "invalid" 

Looks like we shall see inserts to fail with the index on the column, regardless of the index status ("valid" or "invalid"), if we start to receive the "bad" values?  Maybe I shall drop all these indexes.

If it's a bug, what information would be needed to trigger a bug fix?  Is there a formal channel?

Thanks.
-glenn


On Fri, Oct 16, 2015 at 6:43 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Glenn Zhu <gzhu@medallia.com> writes:
> We are currently running 9.4.4.  9.4.5 notes have two references to gin
> index but does not seem to address the issue.

> We are however able to create same index on some other databases.  So it
> maybe data related or size of table related?

I'd guess that it's triggered by a specific data item or set of data
items.  Doubt it has anything to do with table size per se.  The quoted
value is 0x78000000, which makes me think that something is
misinterpreting a plain C string as a varlena value (with a length word),
or something along that line.

                        regards, tom lane



--
Glenn Zhu
SaaS Operations | ❖ Medallia, Inc.
  

Re: Error creating gin index on jsonb columns

From
Tom Lane
Date:
Glenn Zhu <gzhu@medallia.com> writes:
> Currently, after hitting the error, the indexes were still created but
> marked with status of "invalid"

That's just what CREATE INDEX CONCURRENTLY would do with any error.
(It might be worth checking whether a non-CONCURRENTLY build hits the
same error, though I'm betting it will.)

> If it's a bug, what information would be needed to trigger a bug fix?

A reproducible test case would move things along quite a bit; without that
we're just guessing.

> Is there a formal channel?

Well, you could move the thread to pgsql-bugs but you might as well
keep it where it is.
        regards, tom lane

Re: Error creating gin index on jsonb columns

From
Glenn Zhu
Date:
We can't test a non-concurrent index build in production - but your word is just as good.

We only see this on some of production databases.  We did not see it in QA testing.  But we will try to get a test case in QA.

Is this categorized as a bug specific to GIN indexes or a PostgreSQL bug in general?

-glenn

On Fri, Oct 16, 2015 at 7:02 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Glenn Zhu <gzhu@medallia.com> writes:
> Currently, after hitting the error, the indexes were still created but
> marked with status of "invalid"

That's just what CREATE INDEX CONCURRENTLY would do with any error.
(It might be worth checking whether a non-CONCURRENTLY build hits the
same error, though I'm betting it will.)

> If it's a bug, what information would be needed to trigger a bug fix?

A reproducible test case would move things along quite a bit; without that
we're just guessing.

> Is there a formal channel?

Well, you could move the thread to pgsql-bugs but you might as well
keep it where it is.

                        regards, tom lane



--
Glenn Zhu
SaaS Operations | ❖ Medallia, Inc.
  

Re: Error creating gin index on jsonb columns

From
Tom Lane
Date:
Glenn Zhu <gzhu@medallia.com> writes:
> Is this categorized as a bug specific to GIN indexes or a PostgreSQL bug in
> general?

My gut says it's GIN-specific, but that's really only an educated guess;
we have too little info.

What I would recommend is that you get the data onto a non-production
machine where you can play around a bit more.  One thing you could do
then is run a build with debug symbols, attach to the backend process
with gdb, and set a breakpoint at "errfinish".  Then provoke the error,
and capture a backtrace from the call to errfinish.  That would greatly
narrow things down, though it might not be enough to isolate the bug
immediately.  (If I had a test case in hand, that's exactly the first
step I would take with it, but maybe you can do it for me.)
        regards, tom lane

Re: Error creating gin index on jsonb columns

From
Glenn Zhu
Date:
We will follow your instructions and get back to you.

Thank you Tom.  Much appreciated!
-glenn

On Fri, Oct 16, 2015 at 7:16 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Glenn Zhu <gzhu@medallia.com> writes:
> Is this categorized as a bug specific to GIN indexes or a PostgreSQL bug in
> general?

My gut says it's GIN-specific, but that's really only an educated guess;
we have too little info.

What I would recommend is that you get the data onto a non-production
machine where you can play around a bit more.  One thing you could do
then is run a build with debug symbols, attach to the backend process
with gdb, and set a breakpoint at "errfinish".  Then provoke the error,
and capture a backtrace from the call to errfinish.  That would greatly
narrow things down, though it might not be enough to isolate the bug
immediately.  (If I had a test case in hand, that's exactly the first
step I would take with it, but maybe you can do it for me.)

                        regards, tom lane



--
Glenn Zhu
SaaS Operations | ❖ Medallia, Inc.