Thread: Problems Restarting PostgreSQL Daemon

Problems Restarting PostgreSQL Daemon

From
Rich Shepard
Date:
   My server is rebooted infrequently, usually after a kernel upgrade and
on very rare occasions when something causes it to hang. After rebooting I
always have serious issues getting postgresql running again, even though the
startup script is part of the boot sequence. Yesterday was one of those
highly unusual hangs, and I cannot restart the service. I'd like to
understand why.

   When I run the Slackware script, '/etc/rc.d/rc.postgresql start' (script
attached), I'm shown a process ID and told the daemon is already running.
For example:

Starting PostgreSQL
15342
PostgreSQL daemon already running

   However, there is no process ID 15342, and no postgres running. I manually
removed /tmp/.s.PGSQL.5432 and its log file. Also -- apparently in error --
the .pid file. Makes no difference.

   Perhaps there's an error in the script that I'm not seeing (I didn't write
it). Regardless, if I learn why there's a problem I can fix the script and
avoid this delay and hassle restarting postgres after the daemon's been shut
down.

TIA,

Rich

--
Richard B. Shepard, Ph.D.               |  Integrity            Credibility
Applied Ecosystem Services, Inc.        |            Innovation
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863

Attachment

Re: Problems Restarting PostgreSQL Daemon

From
Tom Lane
Date:
Rich Shepard <rshepard@appl-ecosys.com> writes:
>    My server is rebooted infrequently, usually after a kernel upgrade and
> on very rare occasions when something causes it to hang. After rebooting I
> always have serious issues getting postgresql running again, even though the
> startup script is part of the boot sequence. Yesterday was one of those
> highly unusual hangs, and I cannot restart the service. I'd like to
> understand why.

>    When I run the Slackware script, '/etc/rc.d/rc.postgresql start' (script
> attached), I'm shown a process ID and told the daemon is already running.

The short answer is probably "don't use Slackware's startup script".
Some distros have PG start scripts that have had the bugs beaten out
of them, and others not so much.

>    Perhaps there's an error in the script that I'm not seeing (I didn't write
> it). Regardless, if I learn why there's a problem I can fix the script and
> avoid this delay and hassle restarting postgres after the daemon's been shut
> down.

Have you read the script to see what condition causes it to issue the
mentioned error?  I'd imagine that it's looking at some other lockfile
than you think.

            regards, tom lane

Re: Problems Restarting PostgreSQL Daemon

From
Rich Shepard
Date:
On Tue, 22 Jul 2008, Tom Lane wrote:

> The short answer is probably "don't use Slackware's startup script". Some
> distros have PG start scripts that have had the bugs beaten out of them,
> and others not so much.

   Excellent advice, Tom. I'll take it.

> Have you read the script to see what condition causes it to issue the
> mentioned error?  I'd imagine that it's looking at some other lockfile
> than you think.

   I tried following the logic, and it appears the issue now is 'invalid data
in PID file "/var/lib/pgsql/data/postmaster.pid" '. If I delete that file,
is it automatically recreated? I'm using /usr/bin/pg_ctl as user postgres.

Thanks,

Rich

--
Richard B. Shepard, Ph.D.               |  Integrity            Credibility
Applied Ecosystem Services, Inc.        |            Innovation
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863

Re: Problems Restarting PostgreSQL Daemon

From
"Jeff Soules"
Date:
>  I tried following the logic, and it appears the issue now is 'invalid data
> in PID file "/var/lib/pgsql/data/postmaster.pid" '. If I delete that file,
> is it automatically recreated?

Why not just move it and rename it?  If it's recreated, great; if not,
you still have the corrupted file on hand to try to fix, no?

On Tue, Jul 22, 2008 at 11:15 AM, Rich Shepard <rshepard@appl-ecosys.com> wrote:
> On Tue, 22 Jul 2008, Tom Lane wrote:
>
>> The short answer is probably "don't use Slackware's startup script". Some
>> distros have PG start scripts that have had the bugs beaten out of them,
>> and others not so much.
>
>  Excellent advice, Tom. I'll take it.
>
>> Have you read the script to see what condition causes it to issue the
>> mentioned error?  I'd imagine that it's looking at some other lockfile
>> than you think.
>
>  I tried following the logic, and it appears the issue now is 'invalid data
> in PID file "/var/lib/pgsql/data/postmaster.pid" '. If I delete that file,
> is it automatically recreated? I'm using /usr/bin/pg_ctl as user postgres.
>
> Thanks,
>
> Rich
>
> --
> Richard B. Shepard, Ph.D.               |  Integrity            Credibility
> Applied Ecosystem Services, Inc.        |            Innovation
> <http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863
>
> --
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
>

Re: Problems Restarting PostgreSQL Daemon

From
Tom Lane
Date:
Rich Shepard <rshepard@appl-ecosys.com> writes:
>    I tried following the logic, and it appears the issue now is 'invalid data
> in PID file "/var/lib/pgsql/data/postmaster.pid" '. If I delete that file,
> is it automatically recreated? I'm using /usr/bin/pg_ctl as user postgres.

If you're certain there's no postmaster running, it's safe to remove
postmaster.pid.  However you really shouldn't have to; the postmaster
is generally able to figure out whether a pidfile is live or not.

The "invalid data" bit is interesting though.  It looks like pg_ctl
would produce that error if the pidfile exists but is empty when it
looks.  This seems like a race condition hazard, though the odds of
hitting it are tiny.  What's in the file exactly?

            regards, tom lane

Re: Problems Restarting PostgreSQL Daemon

From
Rich Shepard
Date:
On Tue, 22 Jul 2008, Tom Lane wrote:

> If you're certain there's no postmaster running, it's safe to remove
> postmaster.pid. However you really shouldn't have to; the postmaster is
> generally able to figure out whether a pidfile is live or not.

Tom,

   I thought the postmaster knew what was current and what needed to be
replaced, but the process ID in the pidfile did not exist.

> The "invalid data" bit is interesting though.  It looks like pg_ctl would
> produce that error if the pidfile exists but is empty when it looks.  This
> seems like a race condition hazard, though the odds of hitting it are
> tiny.  What's in the file exactly?

   I deleted the .pid, but still could not get the postmaster running. Then I
'touched' the name so I had an empty file. Made no difference. While pg_ctl
tells me the server is starting, there is no /tmp/.s.PGSQL*, no pidfile, and
no postmaster process.

   In the past I've managed to start the postmaster daemon manually, but
today I seem to have it FUBARed.

Rich

--
Richard B. Shepard, Ph.D.               |  Integrity            Credibility
Applied Ecosystem Services, Inc.        |            Innovation
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863

Re: Problems Restarting PostgreSQL Daemon

From
"Andrej Ricnik-Bay"
Date:
On 23/07/2008, Rich Shepard <rshepard@appl-ecosys.com> wrote:

>   When I run the Slackware script, '/etc/rc.d/rc.postgresql start' (script
>  attached), I'm shown a process ID and told the daemon is already running.
>  For example:
Since there are no official Slackware postgres packages
I'd like to ask where that script came from :) and how you
installed postges in the first place.  Happy to communicate
of the list if you prefer that.


>  TIA,
>
>  Rich
Cheers,
Andrej

Re: Problems Restarting PostgreSQL Daemon

From
Tom Lane
Date:
Rich Shepard <rshepard@appl-ecosys.com> writes:
> On Tue, 22 Jul 2008, Tom Lane wrote:
>> The "invalid data" bit is interesting though.  It looks like pg_ctl would
>> produce that error if the pidfile exists but is empty when it looks.  This
>> seems like a race condition hazard, though the odds of hitting it are
>> tiny.  What's in the file exactly?

>    I deleted the .pid, but still could not get the postmaster running. Then I
> 'touched' the name so I had an empty file. Made no difference. While pg_ctl
> tells me the server is starting, there is no /tmp/.s.PGSQL*, no pidfile, and
> no postmaster process.

Sounds to me like the postmaster tries to start and fails.  Look into
the postmaster log.  (If the log is going to /dev/null, send it
someplace else...)

            regards, tom lane

Re: Problems Restarting PostgreSQL Daemon

From
Rich Shepard
Date:
On Wed, 23 Jul 2008, Andrej Ricnik-Bay wrote:

> Since there are no official Slackware postgres packages I'd like to ask
> where that script came from :) and how you installed postges in the first
> place.  Happy to communicate of the list if you prefer that.

Andrej,

   Unless others consider this topic to be not appropriate for the list, I
don't mind a public conversation. I thought that I attached the script to my
original message; regardless, here's the attribution:

# PostgreSQL startup script for Slackware Linux
# Copyright 2007 Adis Nezirovic <adis _at_ linux.org.ba>
# Licensed under GNU GPL v2

   I upgraded postgres manually, not creating and using a Slackware package.
It worked just fine until yesterday's reboot.

Thanks,

Rich

--
Richard B. Shepard, Ph.D.               |  Integrity            Credibility
Applied Ecosystem Services, Inc.        |            Innovation
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863

Re: Problems Restarting PostgreSQL Daemon

From
"Andrej Ricnik-Bay"
Date:
On 23/07/2008, Rich Shepard <rshepard@appl-ecosys.com> wrote:

>  Andrej,

Hi Rich,

>  Unless others consider this topic to be not appropriate for the list, I
>  don't mind a public conversation. I thought that I attached the script to
>  my original message; regardless, here's the attribution:

You did - my bad. I usually ignore attachments on mailing-lists,
and did so with yours.


>   I upgraded postgres manually, not creating and using a Slackware package.
>  It worked just fine until yesterday's reboot.

Now there's an interesting piece of information :) How long
ago did you upgrade it?
From which version of pg to which version did you upgrade,
and how did you go about it?  Chances are indeed that the
postmasters logfile (/var/log/postgres) may hold crucial
information as Tom suggested.


>  Thanks,
>
>  Rich

Cheers,
Andrej


--
Please don't top post, and don't use HTML e-Mail :}  Make your quotes concise.

http://www.american.edu/econ/notes/htmlmail.htm

Re: Problems Restarting PostgreSQL Daemon

From
Rich Shepard
Date:
On Wed, 23 Jul 2008, Andrej Ricnik-Bay wrote:

> Now there's an interesting piece of information :) How long
> ago did you upgrade it?

Andrej,

    A month ago; June 17th to be exact.

> From which version of pg to which version did you upgrade,

    From 8.1.13 to 8.3.3.

> and how did you go about it?  Chances are indeed that the postmasters
> logfile (/var/log/postgres) may hold crucial information as Tom suggested.

    Well, after digging myself into a hole, I received help here and climbed
out. It was working last week (when I made some entries into my accounting
system and viewed the local version of our web site). However, ...

    ... something broke during the reboot. From /var/log/postgresql:

FATAL:  database files are incompatible with server
DETAIL:  The database cluster was initialized with PG_CONTROL_VERSION 812,
but the server was compiled with PG_CONTROL_VERSION 833.
HINT:  It looks like you need to initdb.

    I still have the old pgsql (8.1.13) still in a non-standard directory. I
had run initdb after cleaning up the upgrade. Should I do so again?

Thanks,

Rich

--
Richard B. Shepard, Ph.D.               |  Integrity            Credibility
Applied Ecosystem Services, Inc.        |            Innovation
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863

Re: Problems Restarting PostgreSQL Daemon

From
"Joshua D. Drake"
Date:
On Tue, 2008-07-22 at 18:05 -0700, Rich Shepard wrote:
> On Wed, 23 Jul 2008, Andrej Ricnik-Bay wrote:
>
> > Now there's an interesting piece of information :) How long
> > ago did you upgrade it?
>

>     ... something broke during the reboot. From /var/log/postgresql:
>
> FATAL:  database files are incompatible with server
> DETAIL:  The database cluster was initialized with PG_CONTROL_VERSION 812,
> but the server was compiled with PG_CONTROL_VERSION 833.
> HINT:  It looks like you need to initdb.
>
>     I still have the old pgsql (8.1.13) still in a non-standard directory. I
> had run initdb after cleaning up the upgrade. Should I do so again?

It looks to me like your init script just isn't pointing to the 8.3.3
data directory. If you are unsure you can do this:

find / -name PG_VERSION

You likely have 2 or 3 of them. Find the one that says 8.3 and make sure
your start up script points there.

Joshua D. Drake



>
> Thanks,
>
> Rich
>
> --
> Richard B. Shepard, Ph.D.               |  Integrity            Credibility
> Applied Ecosystem Services, Inc.        |            Innovation
> <http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863
>
--
The PostgreSQL Company since 1997: http://www.commandprompt.com/
PostgreSQL Community Conference: http://www.postgresqlconference.org/
United States PostgreSQL Association: http://www.postgresql.us/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate




Re: Problems Restarting PostgreSQL Daemon

From
Rich Shepard
Date:
On Wed, 23 Jul 2008, Andrej Ricnik-Bay wrote:

> Now there's an interesting piece of information :) How long
> ago did you upgrade it?

Andrej,

   I found the thread in the archives for June of this year.

   Re-reading the posted results of running initdb I tried a different
approach to starting the server. Instead of using pg_ctl I used 'postgres -D
/var/lib/pgsql/data &' (while logged in as user postgres, of course.) That
cleaned up a bad shutdown (when I had to reboot the system after it hung),
fixed the missing socket, and replaced the .pid. So, it's up and running
once again.

   My question is how best to modify the startup script so the postmaster
fires up when the system is rebooted. I don't see an option to 'su' to
specify the postgres user's password so I can script this. Have you any
recommendation?

Thanks,

Rich

--
Richard B. Shepard, Ph.D.               |  Integrity            Credibility
Applied Ecosystem Services, Inc.        |            Innovation
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863

Re: Problems Restarting PostgreSQL Daemon

From
"Andrej Ricnik-Bay"
Date:
On 27/07/2008, Rich Shepard <rshepard@appl-ecosys.com> wrote:

>  Andrej,
Hi Rich,

>   I found the thread in the archives for June of this year.
>
>   Re-reading the posted results of running initdb I tried a different
>  approach to starting the server. Instead of using pg_ctl I used 'postgres
> -D
>  /var/lib/pgsql/data &' (while logged in as user postgres, of course.) That
>  cleaned up a bad shutdown (when I had to reboot the system after it hung),
>  fixed the missing socket, and replaced the .pid. So, it's up and running
>  once again.
>
>   My question is how best to modify the startup script so the postmaster
>  fires up when the system is rebooted. I don't see an option to 'su' to
>  specify the postgres user's password so I can script this. Have you any
>  recommendation?
Since Slackware doesn't use the SysV style of inits but default the
easiest way for you to achieve an automatic start-up of postgres
on reboot would be to add something like
if [ -x /etc/rc.d/rc.postgres ]; then
  /etc/rc.d/rc.postgres start
fi
to your /etc/rc.d/rc.local


>  Thanks,
>
>  Rich
Cheers,
Andrej


--
Please don't top post, and don't use HTML e-Mail :}  Make your quotes concise.

http://www.american.edu/econ/notes/htmlmail.htm

Re: Problems Restarting PostgreSQL Daemon

From
Tom Lane
Date:
Rich Shepard <rshepard@appl-ecosys.com> writes:
>    My question is how best to modify the startup script so the postmaster
> fires up when the system is rebooted. I don't see an option to 'su' to
> specify the postgres user's password so I can script this.

Startup scripts invariably run as root, so 'su' isn't going to ask
for a password...

            regards, tom lane

Re: Problems Restarting PostgreSQL Daemon

From
"Andrej Ricnik-Bay"
Date:
On 27/07/2008, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Startup scripts invariably run as root, so 'su' isn't going to ask
>  for a password...
And it's nothing to worry about because the script he's using
is suing to the postgres user anyway ...


>                         regards, tom lane
Cheerw,
Andrej


--
Please don't top post, and don't use HTML e-Mail :}  Make your quotes concise.

http://www.american.edu/econ/notes/htmlmail.htm

Re: Problems Restarting PostgreSQL Daemon

From
Rich Shepard
Date:
On Sun, 27 Jul 2008, Andrej Ricnik-Bay wrote:

> if [ -x /etc/rc.d/rc.postgres ]; then
>  /etc/rc.d/rc.postgres start
> fi
> to your /etc/rc.d/rc.local

   Well, that's the problem, Andrej. I have that script, and it worked fine
with postgres-6.x through -8.1, but failed to correctly start the postmaster
after the system reboot.

   I can try twiddling with the script; it calls pg_ctl, and that should
work, but apparently something broke last week.

Thanks,

Rich

--
Richard B. Shepard, Ph.D.               |  Integrity            Credibility
Applied Ecosystem Services, Inc.        |            Innovation
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863

Re: Problems Restarting PostgreSQL Daemon

From
Rich Shepard
Date:
On Sat, 26 Jul 2008, Tom Lane wrote:

> Startup scripts invariably run as root, so 'su' isn't going to ask for a
> password...

Tom,

   That occurred to me after I wrote the message. Think that I'll tune the
script to use a command that I know is working with 8.3.3.

Many thanks,

Rich

--
Richard B. Shepard, Ph.D.               |  Integrity            Credibility
Applied Ecosystem Services, Inc.        |            Innovation
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863

Re: Problems Restarting PostgreSQL Daemon

From
"Andrej Ricnik-Bay"
Date:
On 27/07/2008, Rich Shepard <rshepard@appl-ecosys.com> wrote:

>   Well, that's the problem, Andrej. I have that script, and it worked fine
>  with postgres-6.x through -8.1, but failed to correctly start the
> postmaster  after the system reboot.
I thought we had established that this issue was caused by
the current instance pointing at the old installs data directory?


>   I can try twiddling with the script; it calls pg_ctl, and that should
>  work, but apparently something broke last week.
That should be quite easy to tweak, really ... my current script
(slightly modified from the one in contrib/startup-scripts) is attached...
You may need to change the dirs in the script yet a bit.

>  Thanks,
>
>  Rich
Cheers,
Andrej


--
Please don't top post, and don't use HTML e-Mail :}  Make your quotes concise.

http://www.american.edu/econ/notes/htmlmail.htm

Attachment

Re: Problems Restarting PostgreSQL Daemon

From
Rich Shepard
Date:
On Sun, 27 Jul 2008, Andrej Ricnik-Bay wrote:

> I thought we had established that this issue was caused by the current
> instance pointing at the old installs data directory?

   No, that wasn't the problem.

   If I use 'postgres -D /var/lib/pgsql/data &' the postmaster starts
correctly and everything runs as intended. If I use '/etc/rc.d/rc.postgresql
start' I get error messages about the postmaster already running and an
invalid .pid.

> That should be quite easy to tweak, really ... my current script (slightly
> modified from the one in contrib/startup-scripts) is attached... You may
> need to change the dirs in the script yet a bit.

   Thank you. I think that for some reason using pg_ctl to start the
postmaster is no longer working here. As I have time, I'll look into why.

Rich

--
Richard B. Shepard, Ph.D.               |  Integrity            Credibility
Applied Ecosystem Services, Inc.        |            Innovation
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863

Re: Problems Restarting PostgreSQL Daemon

From
"Andrej Ricnik-Bay"
Date:
2008/7/28 Rich Shepard <rshepard@appl-ecosys.com>:
>  Thank you. I think that for some reason using pg_ctl to start the
> postmaster is no longer working here. As I have time, I'll look into why.
Can you do a 'locate pg_ctl|xargs ls -l' and see whether you have more than one
installed, and if so, which one comes first in the PATH?

> Rich
Cheers,
Andrej


--
Please don't top post, and don't use HTML e-Mail :} Make your quotes concise.

http://www.american.edu/econ/notes/htmlmail.htm

Re: Problems Restarting PostgreSQL Daemon

From
Rich Shepard
Date:
On Mon, 28 Jul 2008, Andrej Ricnik-Bay wrote:

> Can you do a 'locate pg_ctl|xargs ls -l' and see whether you have more
> than one installed, and if so, which one comes first in the PATH?

Andrej,

   There are two:

-rwxr-xr-x 1 root root 24320 2008-06-17 16:18 /usr/bin/pg_ctl
-rw-r--r-- 1 root root 17286 2007-11-10 13:48
/usr/doc/postgresql-8.3.3/src/sgml/ref/pg_ctl-ref.sgml
-rw-r--r-- 1 root root  3536 2008-06-17 16:18 /usr/man/man1/pg_ctl.1.gz
-rwxr-xr-x 1 root root 32316 2008-06-18 09:50 /usr3/pg813/bin/pg_ctl
-rw-r--r-- 1 root root  8888 2008-06-18 09:50 /usr3/pg813/man/man1/pg_ctl.1

and /usr3/pg813 is not in anyone's path. It's a data storage filesystem and
holds the earlier version during the upgrade. And, now that 8.3.3 is
running, I'll remove that directory.

   Postgres' path is: /bin:/usr/bin:/home/postgres/bin:/sbin:/usr/sbin.

Rich

--
Richard B. Shepard, Ph.D.               |  Integrity            Credibility
Applied Ecosystem Services, Inc.        |            Innovation
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863

why my postgresql auto crashed???

From
Yi Zhao
Date:
hi, all
when I do the command from a sql file by psql client, I got the message:
----
psql:/home/zhay/insert.sql:8: server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
psql:/home/zhay/insert.sql:8: connection to server was lost
----
my sql file is :
set search_path to lives;

insert into store_all select c.*, z.t from  (
        select b.*, y.t from (
                select a.*,  x.t from store a
                        left join (select pid,
array_to_string(array_accum(anchor), ' ') as t
from recommend group by pid)x on x.pid = a.id
        )b left join (select pid, array_to_string(array_accum(anchor), '
') as
t from tag group by pid)y on y.pid = b.id
)c left join (select pid, array_to_string(array_accum(anchor), ' ') as t
from categorie group by pid)z on z.pid = c.id;



so, I check the pg_log
------------
TRAP: FailedAssertion("!(file->curFile >= 0)", File: "buffile.c", Line:
317)
LOG:  server process (PID 4121) was terminated by signal 6: Aborted
LOG:  terminating any other active server processes
WARNING:  terminating connection because of crash of another server
process
DETAIL:  The postmaster has commanded this server process to roll back
the current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and
repeat your command.
WARNING:  terminating connection because of crash of another server
process
DETAIL:  The postmaster has commanded this server process to roll back
the current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and
repeat your command.
HINT:  In a moment you should be able to reconnect to the database and
repeat your command.
LOG:  all server processes terminated; reinitializing
LOG:  database system was interrupted; last known up at 2008-07-27
07:36:16 UTC
LOG:  database system was not properly shut down; automatic recovery in
progress
FATAL:  the database system is in recovery mode
LOG:  redo starts at D/3FB00BC0
LOG:  record with zero length at D/3FB59898
LOG:  redo done at D/3FB59868
LOG:  last completed transaction was at log time 2008-07-27
07:40:53.70866+00
LOG:  autovacuum launcher started
LOG:  database system is ready to accept connections
------------

thanks all.

regards.




Re: why my postgresql auto crashed???

From
Craig Ringer
Date:
Hi

When creating a new thread (asking a new question, etc) on a mailing
list please create a new message rather than replying to an existing
one. It helps people reading the mailing list keep track.

> so, I check the pg_log
> ------------
> TRAP: FailedAssertion("!(file->curFile >= 0)", File: "buffile.c", Line:
> 317)
> LOG:  server process (PID 4121) was terminated by signal 6: Aborted

This `postgres' backend detected a problem and terminated its self by
calling abort().

This forced the postmaster to terminate all other backends and restart
the whole server.

You omitted some very important information from your post, including:

- Your PostgreSQL version
- Your operating system
- The version of your operating system
- How you installed PostgreSQL and where you got it from
- Whether the problem is repeatable, ie does the server crash every time
you issue the problem command? Sometimes? Only ever happened once?

--
Craig Ringer

Re: why my postgresql auto crashed???

From
Yi Zhao
Date:
I'm sorry for the lack of information given in this mail;
Postgresql: 8.3.3
System:Linux 2.6.9-55.ELsmp
Install: I compile it myself

Others:
the same command, it works fine on another machine(called A):(

I EXPLAIN the sql machine A and machine B, I found that on machine B, it
never use index when command execute, but on machine A, index used:((,
so I "set enable_seqscan TO off;" on machine B, it's works beacause of
the usage of index!!!

I don't konw why.

thanks all, any help is appreciated.

On Mon, 2008-07-28 at 16:32 +0800, Craig Ringer wrote:
> Hi
>
> When creating a new thread (asking a new question, etc) on a mailing
> list please create a new message rather than replying to an existing
> one. It helps people reading the mailing list keep track.
>
> > so, I check the pg_log
> > ------------
> > TRAP: FailedAssertion("!(file->curFile >= 0)", File: "buffile.c", Line:
> > 317)
> > LOG:  server process (PID 4121) was terminated by signal 6: Aborted
>
> This `postgres' backend detected a problem and terminated its self by
> calling abort().
>
> This forced the postmaster to terminate all other backends and restart
> the whole server.
>
> You omitted some very important information from your post, including:
>
> - Your PostgreSQL version
> - Your operating system
> - The version of your operating system
> - How you installed PostgreSQL and where you got it from
> - Whether the problem is repeatable, ie does the server crash every time
> you issue the problem command? Sometimes? Only ever happened once?
>
> --
> Craig Ringer
>


Re: why my postgresql auto crashed???

From
Craig Ringer
Date:
Yi Zhao wrote:
> I'm sorry for the lack of information given in this mail;
> Postgresql: 8.3.3
> System:Linux 2.6.9-55.ELsmp
> Install: I compile it myself
>
> Others:
> the same command, it works fine on another machine(called A):(
>
> I EXPLAIN the sql machine A and machine B, I found that on machine B, it
> never use index when command execute, but on machine A, index used:((,
> so I "set enable_seqscan TO off;" on machine B, it's works beacause of
> the usage of index!!!
>
> I don't konw why.

It almost certainly means that you have a corrupt index. First, stop the
server and make a full backup of your data directory.

Then start the server back up. Do a full pg_dump if you can.

Now, try issuing a REINDEX on the problem index. If you don't know which
one, or want to make sure, use REINDEX DATABASE instead. Re-test and see
if the problem still occurs.

You should also check your RAID controller, disks, and filesystem to
make sure there's no problem in your storage system. If you are not
using a RAID controller with battery backup cache, make sure write
caching is turned off on the controller and the disks.

If you are not using a UPS, consider getting one.

By the way, a Google search (while I was trying to figure out which
distro and version of the distro you were using) turned this up:

http://www.centos.org/modules/newbb/viewtopic.php?topic_id=8779&forum=27

The crash is in ext3, and was triggered by MySQL. See:

http://bugs.centos.org/view.php?id=2077

So: check your system logs for errors from the kernel. Consider
upgrading to a less ancient kernel, too.

--
Craig Ringer

Re: why my postgresql auto crashed???

From
Tom Lane
Date:
Craig Ringer <craig@postnewspapers.com.au> writes:
> Yi Zhao wrote:
>> I don't konw why.

> It almost certainly means that you have a corrupt index.

No, because that assert is nowhere near the index code.  I think it's a
garden-variety bug, but we need a reproducible test case to fix it.

            regards, tom lane

Re: why my postgresql auto crashed???

From
Yi Zhao
Date:
yes, I thinks it is unrelated with index, beacause the problem is still
exist after reindex.

I try to change the work memory or shared memory, it's no use:(

regards,
Yi


On Mon, 2008-07-28 at 11:10 -0400, Tom Lane wrote:
> Craig Ringer <craig@postnewspapers.com.au> writes:
> > Yi Zhao wrote:
> >> I don't konw why.
>
> > It almost certainly means that you have a corrupt index.
>
> No, because that assert is nowhere near the index code.  I think it's a
> garden-variety bug, but we need a reproducible test case to fix it.
>
>             regards, tom lane
>


Re: why my postgresql auto crashed???

From
Alvaro Herrera
Date:
Yi Zhao wrote:
> yes, I thinks it is unrelated with index, beacause the problem is still
> exist after reindex.
>
> I try to change the work memory or shared memory, it's no use:(

Of course not.

Please post the schema of involved tables, and enough data in them to be
able to reproduce the problem.

--
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.