Thread: Invalid memory alloc querying field type bytea

Invalid memory alloc querying field type bytea

From
"Thompson, Shon"
Date:

I’m getting errors when I query a table that uses data type bytea.  I can query the table if I don’t include the field for data type bytea.  Here are the two error messages I get.  The first error is one generated from visual studio:

 

"ERROR: XX000: invalid memory alloc request size 1294068297"

 

This error is what I get when I use pgAdmin:

 

“ERROR:  invalid memory alloc request size 1881675850

SQL state: XX000”

 

Please let me know if you need any additional information to determine if this is a bug.  Thanks.

 

Shon

 

MGMCEmailSig

Shon Thompson

Senior Application Engineer

Information Technology Division

Office:  207-624-4391

Cell:  207-330-9501

Shon.Thompson@MaineGeneral.org

 

Attachment

Re: Invalid memory alloc querying field type bytea

From
Tom Lane
Date:
"Thompson, Shon" <Shon.Thompson@MaineGeneral.org> writes:
> I'm getting errors when I query a table that uses data type bytea.  I can query the table if I don't include the
fieldfor data type bytea.  Here are the two error messages I get.  The first error is one generated from visual studio: 
> "ERROR: XX000: invalid memory alloc request size 1294068297"

Looks like a corrupt-data problem.  I doubt it's directly the fault of
bytea; any variable-length field would have given a similar type of error,
because what this seems to be is a damaged field length word.

Do you see any clearly bogus data in other columns of the table?  That
might help isolate which row(s) have gotten damaged.  It seems somewhat
unlikely that only the bytea field header would be clobbered and not
any adjacent data.

The questions you should be asking yourself include whether your PG
version is up-to-date and whether the storage stack is reliable.

            regards, tom lane



RE: Re: Invalid memory alloc querying field type bytea

From
"Thompson, Shon"
Date:
I tried installing the latest version of postgress, and I get the same error.  Postgres returns the smaller rows
withouterrors, but cannot return the two largest bytea fields.  The length function for the bytea data of these two
rowsreturns: 
647,034,150
819,319,303

This is a windows machine.  I'd expect the OS to use paging file if there's not enough memory.  My computer doesn't
seemto be memory bound because I have 16 gig and 6 gig free when the query runs.  I'm getting the same error on these
twoversions of postgres: 
11.2 build 1914
12.3 build 1914



Shon Thompson
Office:  207-624-4391

-----Original Message-----
From: Tom Lane <tgl@sss.pgh.pa.us>
Sent: Wednesday, June 17, 2020 2:46 PM
To: Thompson, Shon <Shon.Thompson@MaineGeneral.org>
Cc: pgsql-bugs@lists.postgresql.org
Subject: <EXTERNAL> Re: Invalid memory alloc querying field type bytea

CAUTION: EXTERNAL EMAIL --- THIS EMAIL ORIGINATED OUTSIDE OF MAINEGENERAL.  DO NOT CLICK ANY LINKS OR OPEN ANY
ATTACHMENTSUNLESS YOU KNOW THE SENDER. REPORT ALL SUSPICIOUS EMAIL TO SPAM@MAINEGENERAL.ORG
<mailto:SPAM@MAINEGENERAL.ORG> 


"Thompson, Shon" <Shon.Thompson@MaineGeneral.org> writes:
> I'm getting errors when I query a table that uses data type bytea.  I can query the table if I don't include the
fieldfor data type bytea.  Here are the two error messages I get.  The first error is one generated from visual studio: 
> "ERROR: XX000: invalid memory alloc request size 1294068297"

Looks like a corrupt-data problem.  I doubt it's directly the fault of bytea; any variable-length field would have
givena similar type of error, because what this seems to be is a damaged field length word. 

Do you see any clearly bogus data in other columns of the table?  That might help isolate which row(s) have gotten
damaged. It seems somewhat unlikely that only the bytea field header would be clobbered and not any adjacent data. 

The questions you should be asking yourself include whether your PG version is up-to-date and whether the storage stack
isreliable. 

            regards, tom lane




Re: Re: Invalid memory alloc querying field type bytea

From
Joe Conway
Date:
On 7/8/20 11:25 AM, Thompson, Shon wrote:
> Postgres returns the smaller rows without errors, but cannot return the two
> largest bytea fields.  The length function for the bytea data of these two
> rows returns:

> 647,034,150
> 819,319,303

>> "ERROR: XX000: invalid memory alloc request size 1294068297"


The problem is probably byteaout rendering the bytes as (presumably) hex, which
takes ~twice as many bytes as the data, and runs afoul of MaxAllocSize.

Joe

--
Crunchy Data - http://crunchydata.com
PostgreSQL Support for Secure Enterprises
Consulting, Training, & Open Source Development


Attachment