Re: backup manifests - Mailing list pgsql-hackers

On Sat, Mar 21, 2020 at 4:00 AM Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Mon, Mar 16, 2020 at 2:03 AM Suraj Kharage
> <suraj.kharage@enterprisedb.com> wrote:
> > One more suggestion, recent commit (1933ae62) has added the PostgreSQL home page to --help output.
>
> Good catch. Fixed. I also attempted to address the compiler warning
> you mentioned in your other email.
>
> Also, I realized that the previous patch versions didn't handle the
> hex-encoded path format that we need to use for non-UTF8 filenames,
> and that there was no easy way to test that format. So, in this
> version I added an option to force all pathnames to be encoded in that
> format. I also made that option capable of suppressing the backup
> manifest altogether. Other than that, this version is pretty much the
> same as the last version, except for a few additional test cases which
> I added to get the code coverage up even a little more. It would be
> nice if someone could test whether the tests pass on Windows.
>

On my CentOS, the patch gives below compilation failure:
pg_validatebackup.c: In function ‘parse_manifest_file’:
pg_validatebackup.c:335:19: error: assignment left-hand side might be
a candidate for a format attribute [-Werror=suggest-attribute=format]
  context.error_cb = report_manifest_error;

I have tested it on Windows and found there are multiple failures.
The failures are as below:
Test Summary Report
---------------------------------------
t/002_algorithm.pl   (Wstat: 512 Tests: 5 Failed: 4)
  Failed tests:  2-5
  Non-zero exit status: 2
  Parse errors: Bad plan.  You planned 19 tests but ran 5.
t/003_corruption.pl  (Wstat: 256 Tests: 14 Failed: 7)
  Failed tests:  2, 4, 6, 8, 10, 12, 14
  Non-zero exit status: 1
  Parse errors: Bad plan.  You planned 44 tests but ran 14.
t/004_options.pl     (Wstat: 4352 Tests: 25 Failed: 17)
  Failed tests:  2, 4, 6-12, 14-17, 19-20, 22, 25
  Non-zero exit status: 17
t/005_bad_manifest.pl (Wstat: 1792 Tests: 44 Failed: 7)
  Failed tests:  18, 24, 26, 30, 32, 34, 36
  Non-zero exit status: 7
Files=6, Tests=109, 72 wallclock secs ( 0.05 usr +  0.01 sys =  0.06 CPU)
Result: FAIL

Failure Report
------------------------
t/002_algorithm.pl ..... 1/19
#   Failed test 'backup ok with algorithm "none"'
#   at t/002_algorithm.pl line 33.

#   Failed test 'backup manifest exists'
#   at t/002_algorithm.pl line 39.

t/002_algorithm.pl ..... 4/19 #   Failed test 'validate backup with
algorithm "none"'
#   at t/002_algorithm.pl line 53.

#   Failed test 'backup ok with algorithm "crc32c"'
#   at t/002_algorithm.pl line 33.
# Looks like you planned 19 tests but ran 5.
# Looks like you failed 4 tests of 5 run.
# Looks like your test exited with 2 just after 5.
t/002_algorithm.pl ..... Dubious, test returned 2 (wstat 512, 0x200)
Failed 18/19 subtests
t/003_corruption.pl .... 1/44
#   Failed test 'intact backup validated'
#   at t/003_corruption.pl line 110.

#   Failed test 'corrupt backup fails validation: extra_file: matches'
#   at t/003_corruption.pl line 117.
#                   'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     doesn't match '(?^:extra_file.*present on disk but not in the manifest)'
t/003_corruption.pl .... 5/44
#   Failed test 'intact backup validated'
#   at t/003_corruption.pl line 110.
t/003_corruption.pl .... 7/44
#   Failed test 'corrupt backup fails validation:
extra_tablespace_file: matches'
#   at t/003_corruption.pl line 117.
#                   'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     doesn't match '(?^:extra_ts_file.*present on disk but not in the
manifest)'
t/003_corruption.pl .... 9/44
#   Failed test 'intact backup validated'
#   at t/003_corruption.pl line 110.

#   Failed test 'corrupt backup fails validation: missing_file: matches'
#   at t/003_corruption.pl line 117.
#                   'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     doesn't match '(?^:pg_xact/0000.*present in the manifest but not on disk)'
t/003_corruption.pl .... 13/44
#   Failed test 'intact backup validated'
#   at t/003_corruption.pl line 110.
# Looks like you planned 44 tests but ran 14.
# Looks like you failed 7 tests of 14 run.
# Looks like your test exited with 1 just after 14.
t/003_corruption.pl .... Dubious, test returned 1 (wstat 256, 0x100)
Failed 37/44 subtests
t/004_options.pl ....... 1/25
#   Failed test '-q succeeds: exit code 0'
#   at t/004_options.pl line 25.

#   Failed test '-q succeeds: no stderr'
#   at t/004_options.pl line 27.
#          got: 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     expected: ''

#   Failed test '-q checksum mismatch: matches'
#   at t/004_options.pl line 37.
#                   'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     doesn't match '(?^:checksum mismatch for file \"PG_VERSION\")'
t/004_options.pl ....... 7/25
#   Failed test '-s skips checksumming: exit code 0'
#   at t/004_options.pl line 43.

#   Failed test '-s skips checksumming: no stderr'
#   at t/004_options.pl line 43.
#          got: 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     expected: ''

#   Failed test '-s skips checksumming: matches'
#   at t/004_options.pl line 43.
#                   ''
#     doesn't match '(?^:backup successfully verified)'

#   Failed test '-i ignores problem file: exit code 0'
#   at t/004_options.pl line 48.

#   Failed test '-i ignores problem file: no stderr'
#   at t/004_options.pl line 48.
#          got: 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     expected: ''

#   Failed test '-i ignores problem file: matches'
#   at t/004_options.pl line 48.
#                   ''
#     doesn't match '(?^:backup successfully verified)'

#   Failed test '-i does not ignore all problems: matches'
#   at t/004_options.pl line 57.
#                   'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     doesn't match '(?^:pg_xact.*is present in the manifest but not on disk)'

#   Failed test 'multiple -i options work: exit code 0'
#   at t/004_options.pl line 62.

#   Failed test 'multiple -i options work: no stderr'
#   at t/004_options.pl line 62.
#          got: 'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     expected: ''

#   Failed test 'multiple -i options work: matches'
#   at t/004_options.pl line 62.
#                   ''
#     doesn't match '(?^:backup successfully verified)'

#   Failed test 'multiple problems: missing files reported'
#   at t/004_options.pl line 71.
#                   'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     doesn't match '(?^:pg_xact.*is present in the manifest but not on disk)'

#   Failed test 'multiple problems: checksum mismatch reported'
#   at t/004_options.pl line 73.
#                   'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     doesn't match '(?^:checksum mismatch for file \"PG_VERSION\")'

#   Failed test '-e reports 1 error: missing files reported'
#   at t/004_options.pl line 80.
#                   'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     doesn't match '(?^:pg_xact.*is present in the manifest but not on disk)'

#   Failed test 'nonexistent backup directory: matches'
#   at t/004_options.pl line 86.
#                   'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     doesn't match '(?^:could not open directory)'
# Looks like you failed 17 tests of 25.
t/004_options.pl ....... Dubious, test returned 17 (wstat 4352, 0x1100)
Failed 17/25 subtests
t/005_bad_manifest.pl .. 1/44
#   Failed test 'missing pathname: matches'
#   at t/005_bad_manifest.pl line 156.
#                   'pg_validatebackup: fatal: could not parse backup
manifest: missing size
# '
#     doesn't match '(?^:could not parse backup manifest: missing pathname)'

#   Failed test 'missing size: matches'
#   at t/005_bad_manifest.pl line 156.
#                   'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     doesn't match '(?^:could not parse backup manifest: missing size)'

#   Failed test 'file size is not an integer: matches'
#   at t/005_bad_manifest.pl line 156.
#                   'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     doesn't match '(?^:could not parse backup manifest: file size is
not an integer)'

#   Failed test 'duplicate pathname in backup manifest: matches'
#   at t/005_bad_manifest.pl line 156.
#                   'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     doesn't match '(?^:fatal: duplicate pathname in backup manifest)'
t/005_bad_manifest.pl .. 31/44
#   Failed test 'checksum without algorithm: matches'
#   at t/005_bad_manifest.pl line 156.
#                   'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     doesn't match '(?^:could not parse backup manifest: checksum
without algorithm)'

#   Failed test 'unrecognized checksum algorithm: matches'
#   at t/005_bad_manifest.pl line 156.
#                   'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     doesn't match '(?^:fatal: unrecognized checksum algorithm)'

#   Failed test 'invalid checksum for file: matches'
#   at t/005_bad_manifest.pl line 156.
#                   'pg_validatebackup: fatal: could not parse backup
manifest: both pathname and encoded pathname
# '
#     doesn't match '(?^:fatal: invalid checksum for file)'
# Looks like you failed 7 tests of 44.
t/005_bad_manifest.pl .. Dubious, test returned 7 (wstat 1792, 0x700)
Failed 7/44 subtests

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com