Thread: PostgreSQL crashing

PostgreSQL crashing

From
barai@cs.ucr.edu
Date:
PostgreSQL runs for a while but always eventually (30min - 2hrs) crashes.
I thought it was memory initially but Memtest had no errors after running
for a day.  I don't think it is a hardware issues because I can run other
applications without error.  I have ran both version 8.0.3 and 8.1.1 with
the same error.  I have posted a clipping of the crash from the system
log.  Any help would be greatly appreciated.

Dec 20 17:14:57 server4 kernel: postmaster: page allocation failure.
order:0, mode:0xd0
Dec 20 17:14:57 server4 kernel:  [<c0143271>] __alloc_pages+0x2e1/0x2f7
Dec 20 17:14:57 server4 kernel:  [<c014329f>] __get_free_pages+0x18/0x24
Dec 20 17:14:57 server4 kernel:  [<c0145bfc>] kmem_getpages+0x1c/0xbb
Dec 20 17:14:57 server4 kernel:  [<c014674a>] cache_grow+0xab/0x138
Dec 20 17:14:57 server4 kernel:  [<c014693c>] cache_alloc_refill+0x165/0x19d
Dec 20 17:14:57 server4 kernel:  [<c0146b37>] kmem_cache_alloc+0x51/0x57
Dec 20 17:14:57 server4 kernel:  [<c0142019>] mempool_alloc+0xb2/0x135
Dec 20 17:14:57 server4 kernel:  [<c011fec9>]
autoremove_wake_function+0x0/0x2d
Dec 20 17:14:57 server4 kernel:  [<c011fec9>]
autoremove_wake_function+0x0/0x2d
Dec 20 17:14:57 server4 kernel:  [<c015de0e>] bio_alloc+0x15/0x168
Dec 20 17:14:57 server4 kernel:  [<c026c48f>] sync_page_io+0x25/0xa2
Dec 20 17:14:57 server4 kernel:  [<c026d97c>] write_disk_sb+0x5a/0x86
Dec 20 17:15:01 server4 kernel:  [<c026d9ca>] sync_sbs+0x22/0x2f
Dec 20 17:15:01 server4 kernel:  [<c026da5b>] md_update_sb+0x84/0xc6
Dec 20 17:15:01 server4 kernel:  [<c02706b5>] md_write_start+0x5e/0x8c
Dec 20 17:15:01 server4 kernel:  [<f882faf7>] make_request+0x22a/0x2b3
[raid1]
Dec 20 17:15:01 server4 kernel:  [<c02232e4>]
generic_make_request+0x18e/0x19e
Dec 20 17:15:01 server4 kernel:  [<c02233be>] submit_bio+0xca/0xd2
Dec 20 17:15:01 server4 kernel:  [<c0144812>]
test_set_page_writeback+0xad/0xe1
Dec 20 17:15:01 server4 kernel:  [<c0152ad7>] swap_writepage+0x9a/0xa3
Dec 20 17:15:01 server4 kernel:  [<c01487ed>] pageout+0x8d/0xcc
Dec 20 17:15:01 server4 kernel:  [<c0148a33>] shrink_list+0x207/0x3ed
Dec 20 17:15:01 server4 kernel:  [<c0147cb4>] __pagevec_release+0x15/0x1d
Dec 20 17:15:01 server4 kernel:  [<c0148df6>] shrink_cache+0x1dd/0x34d
Dec 20 17:15:01 server4 kernel:  [<c01494b4>] shrink_zone+0xa7/0xb6
Dec 20 17:15:01 server4 kernel:  [<c014950f>] shrink_caches+0x4c/0x57
Dec 20 17:15:01 server4 kernel:  [<c0149606>] try_to_free_pages+0xc3/0x1a7
Dec 20 17:15:01 server4 kernel:  [<c014318e>] __alloc_pages+0x1fe/0x2f7
Dec 20 17:15:01 server4 kernel:  [<c014329f>] __get_free_pages+0x18/0x24
Dec 20 17:15:01 server4 kernel:  [<c0145bfc>] kmem_getpages+0x1c/0xbb
Dec 20 17:15:01 server4 kernel:  [<c014674a>] cache_grow+0xab/0x138
...


Re: PostgreSQL crashing

From
Neil Conway
Date:
barai@cs.ucr.edu wrote:
> PostgreSQL runs for a while but always eventually (30min - 2hrs) crashes.

> Dec 20 17:14:57 server4 kernel: postmaster: page allocation failure.
> order:0, mode:0xd0
> Dec 20 17:14:57 server4 kernel:  [<c0143271>] __alloc_pages+0x2e1/0x2f7

This looks like a kernel or hardware issue, not a problem with
PostgreSQL itself.

-Neil


Re: PostgreSQL crashing

From
Richard Huxton
Date:
barai@cs.ucr.edu wrote:
> PostgreSQL runs for a while but always eventually (30min - 2hrs) crashes.
> I thought it was memory initially but Memtest had no errors after running
> for a day.  I don't think it is a hardware issues because I can run other
> applications without error.  I have ran both version 8.0.3 and 8.1.1 with
> the same error.  I have posted a clipping of the crash from the system
> log.  Any help would be greatly appreciated.
>
> Dec 20 17:14:57 server4 kernel: postmaster: page allocation failure.
> order:0, mode:0xd0
> Dec 20 17:14:57 server4 kernel:  [<c0143271>] __alloc_pages+0x2e1/0x2f7
> Dec 20 17:14:57 server4 kernel:  [<c014329f>] __get_free_pages+0x18/0x24
> Dec 20 17:14:57 server4 kernel:  [<c0145bfc>] kmem_getpages+0x1c/0xbb
...
> Dec 20 17:15:01 server4 kernel:  [<c026d9ca>] sync_sbs+0x22/0x2f
> Dec 20 17:15:01 server4 kernel:  [<c026da5b>] md_update_sb+0x84/0xc6
> Dec 20 17:15:01 server4 kernel:  [<c02706b5>] md_write_start+0x5e/0x8c

Hmm - not sure what this is showing, but since the log is all kernel
activity and it goes on for four seconds after PostgreSQL errors I'm not
sure it's your crash in progress.

You probably need to check the mailing-list archives for details on how
to make sure you get a core dump. Also people will want to know what
version of PG we are looking at, what O.S. (Linux by the look of it, but
what distro/version?) and how you installed PG (from source/rpm/etc).

--
   Richard Huxton
   Archonet Ltd