Thread: Error Restoring Large Database

Error Restoring Large Database

From
Mark Rae
Date:
System Configuration
---------------------
  Architecture       : Dual AMD Athlon MP 1800+

  Operating System   : Linux dpdev02 2.4.4-64GB-SMP #1 SMP

  PostgreSQL version : PostgreSQL-7.2b4 and PostgreSQL-7.2rc1

  Compiler used      : gcc version 2.95.3 20010315 (SuSE)


Please enter a FULL description of your problem:
------------------------------------------------

When trying to restore from a dump of a 9Gb database the restore fails.
(see below for info)
The same dump/restore procedure has worked succesfully for smaller
databases ~1-2Gb

The relevant error message appears to come from
src/backend/storage/freespace/freespace.c
which says

/* Shouldn't get here given the initial if-test for space available */
        elog(ERROR, "insert_fsm_page_entry: failed to insert entry!");



The database tables total 9.0Gb with associated indexes taking up
10.0Gb.
There is >25G free disk space, and there is no error in the
system logs.

I have tried other variations on the examples below, but the
restore still fails. Apparently _in the same place_ each time.
I also tried rc1 which behaves in the same way.


Please describe a way to repeat the problem.   Please try to provide a
concise reproducible example, if at all possible:
----------------------------------------------------------------------

The database is dumped using
$ pg_dump -b -Fc core4 > core4.dump

And then restored using
$ pg_restore --dbname=core4 < core4.dump

Which exits with error code 1 and the following message
pg_restore: [archiver (db)] error returned by PQputline

The logfile contains the errors
ERROR:  copy: line 26501638, insert_fsm_page_entry: failed to insert entry!
FATAL 1:  Socket command type 2 unknown


Retrying using an INSERT rather than COPY dump
$ pg_dump --inserts -b -Fc core4 > core4.dump

gives these errors when trying to restore

pg_restore: [archiver (db)] could not execute query: ERROR:  insert_fsm_page_entry: failed to insert entry!

and

ERROR:  insert_fsm_page_entry: failed to insert entry!

in the logfile



    -Mark

--
Mark Rae                                       Tel: +44(0)20 7074 4648
Inpharmatica                                   Fax: +44(0)20 7074 4700
m.rae@inpharmatica.co.uk                http://www.inpharmatica.co.uk/

Re: Error Restoring Large Database

From
Tom Lane
Date:
Mark Rae <m.rae@inpharmatica.co.uk> writes:
> ERROR:  copy: line 26501638, insert_fsm_page_entry: failed to insert entry!

Ugh.  I think I see the problem: silly oversight in this routine.
Please apply the attached patch and see if it fixes the failure.

            regards, tom lane

*** src/backend/storage/freespace/freespace.c.orig    Mon Nov  5 14:44:10 2001
--- src/backend/storage/freespace/freespace.c    Thu Jan 24 10:31:43 2002
***************
*** 8,14 ****
   * Portions Copyright (c) 1994, Regents of the University of California
   *
   * IDENTIFICATION
!  *      $Header: /cvsroot/pgsql/src/backend/storage/freespace/freespace.c,v 1.10 2001/11/05 17:46:27 momjian Exp $
   *
   *
   * NOTES:
--- 8,14 ----
   * Portions Copyright (c) 1994, Regents of the University of California
   *
   * IDENTIFICATION
!  *      $Header: /cvsroot/pgsql/src/backend/storage/freespace/freespace.c,v 1.11 2002/01/24 15:31:43 tgl Exp $
   *
   *
   * NOTES:
***************
*** 841,904 ****
  insert_fsm_page_entry(FSMRelation *fsmrel, BlockNumber page, Size spaceAvail,
                        FSMChunk *chunk, int chunkRelIndex)
  {
!     FSMChunk   *newChunk;
!     int            newChunkRelIndex;
!
!     if (fsmrel->numPages >= fsmrel->numChunks * CHUNKPAGES)
      {
!         /* No free space within chunk list, so need another chunk */
!         if ((newChunk = FreeSpaceMap->freeChunks) == NULL)
!             return false;        /* can't do it */
!         FreeSpaceMap->freeChunks = newChunk->next;
!         FreeSpaceMap->numFreeChunks--;
!         newChunk->next = NULL;
!         newChunk->numPages = 0;
!         if (fsmrel->relChunks == NULL)
!             fsmrel->relChunks = newChunk;
!         else
          {
!             FSMChunk   *priorChunk = fsmrel->relChunks;

!             while (priorChunk->next != NULL)
!                 priorChunk = priorChunk->next;
!             priorChunk->next = newChunk;
          }
!         fsmrel->numChunks++;
!         if (chunk == NULL)
!         {
!             /* Original search found that new page belongs at end */
!             chunk = newChunk;
!             chunkRelIndex = 0;
          }
-     }

!     /* Try to insert it the easy way, ie, just move down subsequent data */
!     if (chunk &&
!         push_fsm_page_entry(page, spaceAvail, chunk, chunkRelIndex))
!     {
!         fsmrel->numPages++;
!         fsmrel->nextPage++;        /* don't return same page twice running */
!         return true;
      }
-
-     /*
-      * There is space available, but evidently it's before the place where
-      * the page entry needs to go.    Compact the list and try again. This
-      * will require us to redo the search for the appropriate place.
-      */
-     compact_fsm_page_list(fsmrel);
-     if (lookup_fsm_page_entry(fsmrel, page, &newChunk, &newChunkRelIndex))
-         elog(ERROR, "insert_fsm_page_entry: entry already exists!");
-     if (newChunk &&
-         push_fsm_page_entry(page, spaceAvail, newChunk, newChunkRelIndex))
-     {
-         fsmrel->numPages++;
-         fsmrel->nextPage++;        /* don't return same page twice running */
-         return true;
-     }
-     /* Shouldn't get here given the initial if-test for space available */
-     elog(ERROR, "insert_fsm_page_entry: failed to insert entry!");
-     return false;                /* keep compiler quiet */
  }

  /*
--- 841,899 ----
  insert_fsm_page_entry(FSMRelation *fsmrel, BlockNumber page, Size spaceAvail,
                        FSMChunk *chunk, int chunkRelIndex)
  {
!     /* Outer loop handles retry after compacting rel's page list */
!     for (;;)
      {
!         if (fsmrel->numPages >= fsmrel->numChunks * CHUNKPAGES)
          {
!             /* No free space within chunk list, so need another chunk */
!             FSMChunk   *newChunk;

!             if ((newChunk = FreeSpaceMap->freeChunks) == NULL)
!                 return false;        /* can't do it */
!             FreeSpaceMap->freeChunks = newChunk->next;
!             FreeSpaceMap->numFreeChunks--;
!             newChunk->next = NULL;
!             newChunk->numPages = 0;
!             if (fsmrel->relChunks == NULL)
!                 fsmrel->relChunks = newChunk;
!             else
!             {
!                 FSMChunk   *priorChunk = fsmrel->relChunks;
!
!                 while (priorChunk->next != NULL)
!                     priorChunk = priorChunk->next;
!                 priorChunk->next = newChunk;
!             }
!             fsmrel->numChunks++;
!             if (chunk == NULL)
!             {
!                 /* Original search found that new page belongs at end */
!                 chunk = newChunk;
!                 chunkRelIndex = 0;
!             }
          }
!
!         /* Try to insert it the easy way, ie, just move down subsequent data */
!         if (chunk &&
!             push_fsm_page_entry(page, spaceAvail, chunk, chunkRelIndex))
!         {
!             fsmrel->numPages++;
!             fsmrel->nextPage++;        /* don't return same page twice running */
!             return true;
          }

!         /*
!          * There is space available, but evidently it's before the place where
!          * the page entry needs to go.    Compact the list and try again. This
!          * will require us to redo the search for the appropriate place.
!          * Furthermore, compact_fsm_page_list deletes empty end chunks, so
!          * we may need to repeat the action of grabbing a new end chunk.
!          */
!         compact_fsm_page_list(fsmrel);
!         if (lookup_fsm_page_entry(fsmrel, page, &chunk, &chunkRelIndex))
!             elog(ERROR, "insert_fsm_page_entry: entry already exists!");
      }
  }

  /*

Re: Error Restoring Large Database

From
Mark Rae
Date:
Tom Lane wrote:
>
> Mark Rae <m.rae@inpharmatica.co.uk> writes:
> > ERROR:  copy: line 26501638, insert_fsm_page_entry: failed to insert entry!
>
> Ugh.  I think I see the problem: silly oversight in this routine.
> Please apply the attached patch and see if it fixes the failure.
>

Yup, that seems to have fixed the problem.
I have tried it twice to be sure. :-)

I actually hand patched 7.2b4 as the patch was (I assume) against CVS.
I can try the current CVS version if you want to be really sure.

Thanks.

    -Mark

--
Mark Rae                                       Tel: +44(0)20 7074 4648
Inpharmatica                                   Fax: +44(0)20 7074 4700
m.rae@inpharmatica.co.uk                http://www.inpharmatica.co.uk/

Re: Error Restoring Large Database

From
Tom Lane
Date:
Mark Rae <m.rae@inpharmatica.co.uk> writes:
> Yup, that seems to have fixed the problem.

Thanks.

> I actually hand patched 7.2b4 as the patch was (I assume) against CVS.
> I can try the current CVS version if you want to be really sure.

No need.

            regards, tom lane