Thread: help: production db stuck in startup mode

help: production db stuck in startup mode

From
"Ed L."
Date:
I have a 7.4.6 db running on itanium hpux 11.23.  It appears a
vacuum ran out of disk space,

2005-07-20 13:47:43 [6161]   ERROR:  DBI::do failed for SQL
[VACUUM VERBOSE] to database clincomm:  DBI error:  PANIC:
could not write to file
"/users/postgresql-7.4.6/data/pg_xlog/xlogtemp.6178": No space
left on device

Pgsql was unresponsive, so we sent SIGINT, then SIGQUIT, both
with no response, so finally, sent SIGKILL, then used ipcrm to
clear the shm and semaphores.  Then restarted, but upon restart,
it's stuck in "FATAL:  the database system is starting up" mode.

Any clues???

Thanks,
Ed

Re: help: production db stuck in startup mode

From
"Ed L."
Date:
On Wednesday July 20 2005 1:13 pm, Ed L. wrote:
> I have a 7.4.6 db running on itanium hpux 11.23.  It appears a
> vacuum ran out of disk space,

Oh, and of course, we created extra diskspace before
restarting...

Ed

>
> 2005-07-20 13:47:43 [6161]   ERROR:  DBI::do failed for SQL
> [VACUUM VERBOSE] to database clincomm:  DBI error:  PANIC:
> could not write to file
> "/users/postgresql-7.4.6/data/pg_xlog/xlogtemp.6178": No space
> left on device
>
> Pgsql was unresponsive, so we sent SIGINT, then SIGQUIT, both
> with no response, so finally, sent SIGKILL, then used ipcrm to
> clear the shm and semaphores.  Then restarted, but upon
> restart, it's stuck in "FATAL:  the database system is
> starting up" mode.
>
> Any clues???
>
> Thanks,
> Ed
>
> ---------------------------(end of
> broadcast)--------------------------- TIP 3: Have you checked
> our extensive FAQ?
>
>                http://www.postgresql.org/docs/faq

Re: help: production db stuck in startup mode

From
"Ed L."
Date:
On Wednesday July 20 2005 1:19 pm, Ed L. wrote:
> On Wednesday July 20 2005 1:13 pm, Ed L. wrote:
> > I have a 7.4.6 db running on itanium hpux 11.23.  It appears
> > a vacuum ran out of disk space:
> > 2005-07-20 13:47:43 [6161]   ERROR:  DBI::do failed for SQL
> > [VACUUM VERBOSE] to database clincomm:  DBI error:  PANIC:
> > could not write to file
> > "/users/postgresql-7.4.6/data/pg_xlog/xlogtemp.6178": No
> > space left on device
> >
> > Pgsql was unresponsive, so we sent SIGINT, then SIGQUIT,
> > both with no response, so finally, sent SIGKILL, then used
> > ipcrm to clear the shm and semaphores.  Then we (cleared
> > diskspace and) restarted, but
> > upon restart, it's stuck in "FATAL:  the database system is
> > starting up" mode.

The system finally restarted, but it went through auto-recovery
and took some number of minutes to completely come up.

Autovac was in the process of doing a db-wide vacuum to prevent
xid wraparound.  We had 1-2gb of diskspace free.  What kind of
diskspace usage surge might that create for a 20gb+ db?

Ed