Re: pg_dump and large files - is this a problem? - Mailing list pgsql-hackers

Bruce Momjian wrote:
> > So I think we're wasting our time to debate whether we need to support
> > non-integral off_t ... let's just apply Bruce's version and wait to
> > see if anyone has a problem before doing more work.
>
> I am concerned about one more thing.  On BSD/OS, we have off_t of quad
> (8 byte), but we don't have fseeko, so this call looks questionable:
>
>     if (fseeko(AH->FH, tctx->dataPos, SEEK_SET) != 0)
>
> In this case, dataPos is off_t (8 bytes), while fseek only accepts long
> in that parameter (4 bytes).  When this code is hit, a file > 4 gigs
> will seek to the wrong offset, I am afraid.  Also, I don't understand
> why the compiler doesn't produce a warning.
>
> I wonder if I should add a conditional test so this code is hit only if
> HAVE_FSEEKO is defined.  There is alternative code for all the non-zero
> fseeks.

Here is a patch that I think fixes the problem I outlined above.  If
there is no fseeko(), it will not call fseek with a non-zero offset
unless sizeof(off_t) <= sizeof(long).

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
Index: src/bin/pg_dump/pg_backup_custom.c
===================================================================
RCS file: /cvsroot/pgsql-server/src/bin/pg_dump/pg_backup_custom.c,v
retrieving revision 1.22
diff -c -c -r1.22 pg_backup_custom.c
*** src/bin/pg_dump/pg_backup_custom.c    22 Oct 2002 19:15:23 -0000    1.22
--- src/bin/pg_dump/pg_backup_custom.c    22 Oct 2002 21:36:30 -0000
***************
*** 431,437 ****
      if (tctx->dataState == K_OFFSET_NO_DATA)
          return;

!     if (!ctx->hasSeek || tctx->dataState == K_OFFSET_POS_NOT_SET)
      {
          /* Skip over unnecessary blocks until we get the one we want. */

--- 431,441 ----
      if (tctx->dataState == K_OFFSET_NO_DATA)
          return;

!     if (!ctx->hasSeek || tctx->dataState == K_OFFSET_POS_NOT_SET
! #if !defined(HAVE_FSEEKO)
!         || sizeof(off_t) > sizeof(long)
! #endif
!         )
      {
          /* Skip over unnecessary blocks until we get the one we want. */

***************
*** 809,815 ****
           * be ok to just use the existing self-consistent block
           * formatting.
           */
!         if (ctx->hasSeek)
          {
              fseeko(AH->FH, tpos, SEEK_SET);
              WriteToc(AH);
--- 813,823 ----
           * be ok to just use the existing self-consistent block
           * formatting.
           */
!         if (ctx->hasSeek
! #if !defined(HAVE_FSEEKO)
!             && sizeof(off_t) <= sizeof(long)
! #endif
!             )
          {
              fseeko(AH->FH, tpos, SEEK_SET);
              WriteToc(AH);