allocation limit for encoding conversion - Mailing list pgsql-hackers

From Alvaro Herrera
Subject allocation limit for encoding conversion
Date
Msg-id 20190816181418.GA898@alvherre.pgsql
Whole thread Raw
Responses Re: allocation limit for encoding conversion  (Alvaro Herrera <alvherre@2ndquadrant.com>)
Re: allocation limit for encoding conversion  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
Somebody ran into issues when generating large XML output (upwards of
256 MB) and then sending via a connection with a different
client_encoding.  This occurs because we pessimistically allocate 4x as
much memory as the string needs, and we run into the 1GB palloc
limitation.  ISTM we can do better now by using huge allocations, as per
the preliminary attached patch (which probably needs an updated overflow
check rather than have it removed altogether); but at least it is able
to process this query, which it wasn't without the patch:

select query_to_xml(
    'select a, cash_words(a::text::money) from generate_series(0, 2000000) a',
    true, false, '');

-- 
Álvaro Herrera

Attachment

pgsql-hackers by date:

Previous
From: "Jonathan S. Katz"
Date:
Subject: Re: Add "password_protocol" connection parameter to libpq
Next
From: Alvaro Herrera
Date:
Subject: Re: allocation limit for encoding conversion