Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception - Mailing list pgsql-performance

From James Mansion
Subject Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception
Date
Msg-id 48B78662.5010208@mansionfamily.plus.com
Whole thread Raw
In response to Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception  (david@lang.hm)
Responses Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception
List pgsql-performance
david@lang.hm wrote:
> for example if you have a process that uses 1G of ram (say firefox)
> and it needs to start a new process (say acroread to handle a pdf
> file), what it does is it forks the firefox process (each of which
> have 1G of ram allocated), and then does an exec of the acroread
> process (releasing the 1G of ram previously held by that copy of the
> firefox process)
>
Indeed, which is why we have vfork.  And, OK, vfork is busted if you
have a threaded environment, so we have posix_spawn and posix_spawnp.

It is also worth noting that the copy isn't really a full copy on any
decent modern UNIX - it is a reservation against the total swap space
available.  Most pages will be happilly shared copy-on-write and never
fully copied to the child before the exec.

I can't see how an OS can lie to processes about memory being allocated
to them and not be ridiculed as a toy, but there you go.  I don't think
Linux is the only perpetrator - doesn't AIX do this too?

The 'bests trategy' for the OOM killer is not to have one, and accept
that you need some swap space available (it doesn't have to be fast
since it won't actually be touched) to help out when fork/exec happens
in big process images.

James


pgsql-performance by date:

Previous
From: Christiaan Willemsen
Date:
Subject: Re: How to setup disk spindles for best performance
Next
From: "Rainer Mager"
Date:
Subject: Re: indexing for distinct search in timestamp based table