Re: ERROR: invalid memory alloc request size 1073741824 - Mailing list pgsql-general

From Jan Wieck
Subject Re: ERROR: invalid memory alloc request size 1073741824
Date
Msg-id CAGBW59dKrpjPLkEMu_bfyJuF=DM7+d8v+M8YDMkaUEhmu3pMeQ@mail.gmail.com
Whole thread Raw
In response to Re: ERROR: invalid memory alloc request size 1073741824  (Stefan Blanke <stefan.blanke@framestore.com>)
Responses Re: ERROR: invalid memory alloc request size 1073741824
List pgsql-general


On Wed, Jan 31, 2018 at 12:32 PM, Stefan Blanke <stefan.blanke@framestore.com> wrote:
>
> I'll bet you it's not that. It's quite unlikely that would fail with
> exactly 1GB request size. It seems much more like a buffer that we keep
> to be power of 2. The question is which one.

I had dismissed corruption before writing in. It's exactly 1GB every time this has happened - and we can dump the full dataset periodically without issue.

>> I have my money on a corrupted TOAST entry. Is this happening on
>> trustworthy hardware or beige box with no ECC or RAID?

It's good quality commercial hardware in our colo - no exactly sure what.

If it is a sporadic issue and you can dump the full dataset, then I just lost my money (Tomas, you coming to PGConf in Jersey City?).

But then, if this is a plain COPY to stdout ... I am wondering what could possibly be in that path that wants to allocate 1GB. Or is this not so plain but rather a COPY ... SELECT ...?


Regards, Jan


 


Thanks for taking the time to look at this!
Stefan





On 01/30/18 22:00, Tomas Vondra wrote:


On 01/30/2018 10:43 PM, Jan Wieck wrote:


On Tue, Jan 30, 2018 at 12:35 PM, Stefan Blanke
<stefan.blanke@framestore.com <mailto:stefan.blanke@framestore.com>> wrote:

    Hello,

    We've tripped over an error when doing a "COPY.. TO STDOUT WITH
    BINARY" query.

    "ERROR:  invalid memory alloc request size 1073741824"
    (exactly 1GB)


I have my money on a corrupted TOAST entry. Is this happening on
trustworthy hardware or beige box with no ECC or RAID?


I'll bet you it's not that. It's quite unlikely that would fail with
exactly 1GB request size. It seems much more like a buffer that we keep
to be power of 2. The question is which one.


regards




--
Jan Wieck
Senior Postgres Architect

pgsql-general by date:

Previous
From: Adrian Klaver
Date:
Subject: Re: Issue with WAL logs temporary not replaying
Next
From: Tomas Vondra
Date:
Subject: Re: ERROR: invalid memory alloc request size 1073741824