Thread: PostgreSQL crash with PANIC message

PostgreSQL crash with PANIC message

From

Moreno Andreo

Date:

23 March 2016, 21:43:11

Hello,
     a customer of ours has an old PC (CoreDuo, 4 GB RAM, Windows XP)
that's running PostgreSQL. Yesterday PostgreSQL started crashing
suddenly with the following log

2016-03-23 10:40:42 CET LOG:  CreateProcess call failed: unrecognized
winsock error 10004 (error code 1450)
2016-03-23 10:40:42 CET LOG:  could not fork new process for connection:
unrecognized winsock error 10004
2016-03-23 10:40:43 CET LOG:  could not receive data from client:
unrecognized winsock error 10055
2016-03-23 10:40:43 CET LOG:  unexpected EOF on client connection
2016-03-23 10:42:48 CET LOG:  CreateProcess call failed: unrecognized
winsock error 10004 (error code 1450)
2016-03-23 10:42:48 CET LOG:  could not fork autovacuum worker process:
unrecognized winsock error 10004
2016-03-23 10:42:49 CET LOG:  CreateProcess call failed: unrecognized
winsock error 10004 (error code 1450)
2016-03-23 10:42:49 CET LOG:  could not fork autovacuum worker process:
unrecognized winsock error 10004
2016-03-23 10:42:50 CET LOG:  CreateProcess call failed: unrecognized
winsock error 10004 (error code 1450)
2016-03-23 10:42:50 CET LOG:  could not fork autovacuum worker process:
unrecognized winsock error 10004
2016-03-23 10:42:51 CET LOG:  CreateProcess call failed: unrecognized
winsock error 10004 (error code 1450)
2016-03-23 10:42:51 CET LOG:  could not fork autovacuum worker process:
unrecognized winsock error 10004
2016-03-23 10:42:52 CET LOG:  CreateProcess call failed: unrecognized
winsock error 10004 (error code 1450)
2016-03-23 10:42:52 CET LOG:  could not fork autovacuum worker process:
unrecognized winsock error 10004
2016-03-23 10:42:53 CET LOG:  CreateProcess call failed: unrecognized
winsock error 10004 (error code 1450)
2016-03-23 10:42:53 CET LOG:  could not fork autovacuum worker process:
unrecognized winsock error 10004
2016-03-23 10:42:54 CET LOG:  CreateProcess call failed: unrecognized
winsock error 10004 (error code 1450)
2016-03-23 10:42:54 CET LOG:  could not fork autovacuum worker process:
unrecognized winsock error 10004
2016-03-23 10:42:55 CET LOG:  CreateProcess call failed: unrecognized
winsock error 10004 (error code 1450)
2016-03-23 10:42:55 CET LOG:  could not fork autovacuum worker process:
unrecognized winsock error 10004
2016-03-23 10:42:56 CET LOG:  CreateProcess call failed: unrecognized
winsock error 10004 (error code 1450)
2016-03-23 10:42:56 CET LOG:  could not fork autovacuum worker process:
unrecognized winsock error 10004
2016-03-23 10:43:24 CET LOG:  could not receive data from client:
unrecognized winsock error 10055
2016-03-23 10:43:24 CET LOG:  unexpected EOF on client connection
2016-03-23 10:43:24 CET LOG:  could not receive data from client:
unrecognized winsock error 10061
2016-03-23 10:43:24 CET LOG:  unexpected EOF on client connection
2016-03-23 10:43:27 CET LOG:  could not receive data from client:
unrecognized winsock error 10055
2016-03-23 10:43:27 CET LOG:  unexpected EOF on client connection
2016-03-23 10:43:30 CET PANIC:  could not write to log file 6, segment
217 at offset 3366912, length 8192: Invalid argument

The machine has 4,5 GB of allocated memory (thus swapping).
I googled to get some references to what's above and that's results:
Winsock 10004 means Interrupted function call;
Error code 1450 should mean "Insufficient system resources exist to
complete the requested service"
Winsock 10055 means "No buffer space available"
Winsock 10061 means "Connection refused"
The only thing I couldn't get some serious reference was the PANIC
message that's in the last row.

But putting the other things together, I figured out the machine is too
obsolete and ran out of system resources, causing error 1450, which
Winsock 10055 is solely a consequence, and probably the PANIC is
generated by this mess.

Am I right?

Thanks in advance,
Moreno.

Re: PostgreSQL crash with PANIC message

From

Adrian Klaver

Date:

23 March 2016, 21:57:32

On 03/23/2016 11:43 AM, Moreno Andreo wrote:
> Hello,
>      a customer of ours has an old PC (CoreDuo, 4 GB RAM, Windows XP)
> that's running PostgreSQL. Yesterday PostgreSQL started crashing
> suddenly with the following log
>
> 2016-03-23 10:40:42 CET LOG:  CreateProcess call failed: unrecognized
> winsock error 10004 (error code 1450)
> 2016-03-23 10:40:42 CET LOG:  could not fork new process for connection:
> unrecognized winsock error 10004
> 2016-03-23 10:40:43 CET LOG:  could not receive data from client:
> unrecognized winsock error 10055
> 2016-03-23 10:40:43 CET LOG:  unexpected EOF on client connection
> 2016-03-23 10:42:48 CET LOG:  CreateProcess call failed: unrecognized
> winsock error 10004 (error code 1450)
> 2016-03-23 10:42:48 CET LOG:  could not fork autovacuum worker process:
> unrecognized winsock error 10004
> 2016-03-23 10:42:49 CET LOG:  CreateProcess call failed: unrecognized
> winsock error 10004 (error code 1450)
> 2016-03-23 10:42:49 CET LOG:  could not fork autovacuum worker process:
> unrecognized winsock error 10004
> 2016-03-23 10:42:50 CET LOG:  CreateProcess call failed: unrecognized
> winsock error 10004 (error code 1450)
> 2016-03-23 10:42:50 CET LOG:  could not fork autovacuum worker process:
> unrecognized winsock error 10004
> 2016-03-23 10:42:51 CET LOG:  CreateProcess call failed: unrecognized
> winsock error 10004 (error code 1450)
> 2016-03-23 10:42:51 CET LOG:  could not fork autovacuum worker process:
> unrecognized winsock error 10004
> 2016-03-23 10:42:52 CET LOG:  CreateProcess call failed: unrecognized
> winsock error 10004 (error code 1450)
> 2016-03-23 10:42:52 CET LOG:  could not fork autovacuum worker process:
> unrecognized winsock error 10004
> 2016-03-23 10:42:53 CET LOG:  CreateProcess call failed: unrecognized
> winsock error 10004 (error code 1450)
> 2016-03-23 10:42:53 CET LOG:  could not fork autovacuum worker process:
> unrecognized winsock error 10004
> 2016-03-23 10:42:54 CET LOG:  CreateProcess call failed: unrecognized
> winsock error 10004 (error code 1450)
> 2016-03-23 10:42:54 CET LOG:  could not fork autovacuum worker process:
> unrecognized winsock error 10004
> 2016-03-23 10:42:55 CET LOG:  CreateProcess call failed: unrecognized
> winsock error 10004 (error code 1450)
> 2016-03-23 10:42:55 CET LOG:  could not fork autovacuum worker process:
> unrecognized winsock error 10004
> 2016-03-23 10:42:56 CET LOG:  CreateProcess call failed: unrecognized
> winsock error 10004 (error code 1450)
> 2016-03-23 10:42:56 CET LOG:  could not fork autovacuum worker process:
> unrecognized winsock error 10004
> 2016-03-23 10:43:24 CET LOG:  could not receive data from client:
> unrecognized winsock error 10055
> 2016-03-23 10:43:24 CET LOG:  unexpected EOF on client connection
> 2016-03-23 10:43:24 CET LOG:  could not receive data from client:
> unrecognized winsock error 10061
> 2016-03-23 10:43:24 CET LOG:  unexpected EOF on client connection
> 2016-03-23 10:43:27 CET LOG:  could not receive data from client:
> unrecognized winsock error 10055
> 2016-03-23 10:43:27 CET LOG:  unexpected EOF on client connection
> 2016-03-23 10:43:30 CET PANIC:  could not write to log file 6, segment
> 217 at offset 3366912, length 8192: Invalid argument
>
> The machine has 4,5 GB of allocated memory (thus swapping).
> I googled to get some references to what's above and that's results:
> Winsock 10004 means Interrupted function call;
> Error code 1450 should mean "Insufficient system resources exist to
> complete the requested service"
> Winsock 10055 means "No buffer space available"
> Winsock 10061 means "Connection refused"
> The only thing I couldn't get some serious reference was the PANIC
> message that's in the last row.
>
> But putting the other things together, I figured out the machine is too
> obsolete and ran out of system resources, causing error 1450, which
> Winsock 10055 is solely a consequence, and probably the PANIC is
> generated by this mess.
>
> Am I right?

Might help to look in:

Control Panel --> Administrative Tools --> Event Viewer



>
> Thanks in advance,
> Moreno.
>
>
>
>


--
Adrian Klaver
adrian.klaver@aklaver.com

Re: PostgreSQL crash with PANIC message

From

Moreno Andreo

Date:

23 March 2016, 22:01:51

Il 23/03/2016 19:57, Adrian Klaver ha scritto:
>
> Might help to look in:
>
> Control Panel --> Administrative Tools --> Event Viewer

No help there. Nothing strange or pointing to critical resource usage.

>
>
>
>>
>> Thanks in advance,
>> Moreno.
>>
>>
>>
>>
>
>

Re: PostgreSQL crash with PANIC message

From

Adrian Klaver

Date:

23 March 2016, 23:57:08

On 03/23/2016 12:02 PM, Moreno Andreo wrote:
> Il 23/03/2016 19:57, Adrian Klaver ha scritto:
>>
>> Might help to look in:
>>
>> Control Panel --> Administrative Tools --> Event Viewer
>
> No help there. Nothing strange or pointing to critical resource usage.

The only thing I have left is the generic answer for when things act
funny on Windows:

Do you have AntiVirus software running against the Postgres data directory?

>
>>
>>
>>
>>>
>>> Thanks in advance,
>>> Moreno.
>>>
>>>
>>>
>>>
>>
>>
>
>
>


--
Adrian Klaver
adrian.klaver@aklaver.com

Re: PostgreSQL crash with PANIC message

From

Michael Paquier

Date:

24 March 2016, 07:12:04

On Thu, Mar 24, 2016 at 5:57 AM, Adrian Klaver
<adrian.klaver@aklaver.com> wrote:
> On 03/23/2016 12:02 PM, Moreno Andreo wrote:
>>
>> Il 23/03/2016 19:57, Adrian Klaver ha scritto:
>>>
>>>
>>> Might help to look in:
>>>
>>> Control Panel --> Administrative Tools --> Event Viewer
>>
>>
>> No help there. Nothing strange or pointing to critical resource usage.
>
>
> The only thing I have left is the generic answer for when things act funny
> on Windows:
>
> Do you have AntiVirus software running against the Postgres data directory?

That's likely the reason, something creeping behind is forcing
disconnections from the server.
--
Michael

Re: PostgreSQL crash with PANIC message

From

Moreno Andreo

Date:

24 March 2016, 12:46:58

Il 24/03/2016 05:12, Michael Paquier ha scritto:
> On Thu, Mar 24, 2016 at 5:57 AM, Adrian Klaver
> <adrian.klaver@aklaver.com> wrote:
>> On 03/23/2016 12:02 PM, Moreno Andreo wrote:
>>> Il 23/03/2016 19:57, Adrian Klaver ha scritto:
>>>>
>>>> Might help to look in:
>>>>
>>>> Control Panel --> Administrative Tools --> Event Viewer
>>>
>>> No help there. Nothing strange or pointing to critical resource usage.
>>
>> The only thing I have left is the generic answer for when things act funny
>> on Windows:
>>
>> Do you have AntiVirus software running against the Postgres data directory?
> That's likely the reason, something creeping behind is forcing
> disconnections from the server.
They have Kaspersky, but I don't know if an upgrade can have done this
mess. I'll have a check in Kaspersky Log
I requested a machine reboot to admins (to flush memory... since 4,5 GB
allocated was wit Postgresql service stopped, and in Event log there's
no trace of a reboot since Nov '15).
We'll see.

Thanks!
Moreno

Re: [SPAM] Re: PostgreSQL crash with PANIC message

From

Moreno Andreo

Date:

25 March 2016, 18:45:47

Il 23/03/2016 21:57, Adrian Klaver ha scritto:
> On 03/23/2016 12:02 PM, Moreno Andreo wrote:
>> Il 23/03/2016 19:57, Adrian Klaver ha scritto:
>>>
>>> Might help to look in:
>>>
>>> Control Panel --> Administrative Tools --> Event Viewer
>>
>> No help there. Nothing strange or pointing to critical resource usage.
>
> The only thing I have left is the generic answer for when things act
> funny on Windows:
>
> Do you have AntiVirus software running against the Postgres data
> directory?

Just to close the thread:

a machine restart worked perfectly (now, while users are working, the
memory allocated is about 1,5 GB), so I requested the admins to
configure a monthly restart. Hope they replace that crap ASAP.

Thanks for your time!
Moreno