Thread: Improve OOM handling in pg_locale.c

Improve OOM handling in pg_locale.c

From
Michael Paquier
Date:
Hi all,

This is a follow-up of
https://www.postgresql.org/message-id/11202.1472597262%40sss.pgh.pa.us
where we are looking at improving OOM handling in the code. In short,
report an ERROR appropriately instead of crashing. As mentioned in
this message, pg_locale.c is trickier to handle because we had better
not call elog() in a code path where the backend's locale are not set
up appropriately. Attached is a patch aimed at fixing that, doing the
following:
- Copy into a temporary struct lconv the results from the call of
localeconv() as those can be overwritten when restoring back the
locales with setlocale().
- Use db_encoding_strdup to encode that correctly.
- Switch back to the backend locales
- Check for any strdup calls that returned NULL and elog()
- If no error, fill in CurrentLocaleConv and return back to caller.

I am attaching that to the next CF.
Thanks,
--
Michael

Attachment

Re: Improve OOM handling in pg_locale.c

From
Haribabu Kommi
Date:
Hi Mithun,

This is a gentle reminder.

you assigned as reviewer to the current patch in the 11-2016 commitfest.
But you haven't shared your review yet. Can you please try to share your views
about the patch. This will help us in smoother operation of commitfest.

Please Ignore if you already shared your review.

Regards,
Hari Babu
Fujitsu Australia

Re: Improve OOM handling in pg_locale.c

From
Mithun Cy
Date:
On Thu, Oct 13, 2016 at 1:40 PM, Michael Paquier <michael.paquier@gmail.com> wrote:
> I am attaching that to the next CF.

I have tested this patch. Now we error out as OOM instead of crash.
------------------------------------------------
postgres=# SELECT '12.34'::money;
ERROR:  out of memory
LINE 1: SELECT '12.34'::money;
------------------------------------------------

One thing which you might need to reconsider is removal of memory leak comments. There is still a leak if there is an error while encoding in db_encoding_strdup. Unless you want to catch those error with an TRY();....CATCH(); and then free the mem.
-	 * localeconv()'s results.  Note that if we were to fail within this
-	 * sequence before reaching "CurrentLocaleConvAllocated = true", we could
-	 * leak some memory --- but not much, so it's not worth agonizing over.

Rest all LGTM.

--
Thanks and Regards
Mithun C Y

Re: Improve OOM handling in pg_locale.c

From
Michael Paquier
Date:
On Tue, Nov 22, 2016 at 8:28 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> I could have lived with leaving the leak there, but really this wasn't
> fixing the worst problem with the code: if it did throw an error out of
> the middle of that sequence, it would leave the process setlocale'd to
> some other locale than we want.  That could lead to unwanted behavior
> in printf and other functions.  And this isn't all that unlikely: an
> encoding conversion failure is definitely possible if you have a locale
> selected that's not compatible with the database encoding.
>
> I whacked the patch around enough so that we didn't do anything except
> libc calls between setting and restoring the locale.  At that point it
> was just a matter of adding a TRY block to be able to say that we
> didn't leak any strdup'd strings, so I figured "might as well".
>
> Pushed with those changes.

Thanks. The changes you have done look good to me at short sight.
-- 
Michael



Re: Improve OOM handling in pg_locale.c

From
Tom Lane
Date:
Mithun Cy <mithun.cy@enterprisedb.com> writes:
> On Thu, Oct 13, 2016 at 1:40 PM, Michael Paquier <michael.paquier@gmail.com>
> wrote:
>> I am attaching that to the next CF.

> One thing which you might need to reconsider is removal of memory leak
> comments. There is still a leak if there is an error while encoding in
> db_encoding_strdup.
> Unless you want to catch those error with an TRY();....CATCH(); and then
> free the mem.

I could have lived with leaving the leak there, but really this wasn't
fixing the worst problem with the code: if it did throw an error out of
the middle of that sequence, it would leave the process setlocale'd to
some other locale than we want.  That could lead to unwanted behavior
in printf and other functions.  And this isn't all that unlikely: an
encoding conversion failure is definitely possible if you have a locale
selected that's not compatible with the database encoding.

I whacked the patch around enough so that we didn't do anything except
libc calls between setting and restoring the locale.  At that point it
was just a matter of adding a TRY block to be able to say that we
didn't leak any strdup'd strings, so I figured "might as well".

Pushed with those changes.
        regards, tom lane