Re: WIP/PoC for parallel backup - Mailing list pgsql-hackers

From Asif Rehman
Subject Re: WIP/PoC for parallel backup
Date
Msg-id CADM=JeiiBjvWyL5ghCgEa08xUu2zpYOQM0nZXY8R+EjBOBKyPQ@mail.gmail.com
Whole thread Raw
In response to Re: WIP/PoC for parallel backup  (Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com>)
List pgsql-hackers

rebased and updated to current master (d025cf88ba). v12 is attahced.

Also, changed the grammar for LIST_WAL_FILES and SEND_FILE to:

- LIST_WAL_FILES 'startptr' 'endptr'
- SEND_FILE 'FILE'  [NOVERIFY_CHECKSUMS]


On Wed, Apr 8, 2020 at 10:48 AM Rajkumar Raghuwanshi <rajkumar.raghuwanshi@enterprisedb.com> wrote:
Hi Asif,

Thanks for new patches.

Patches need to be rebased on head. Getting a failure while applying the 0003 patch.
edb@localhost postgresql]$ git apply  v11/0003-Parallel-Backup-Backend-Replication-commands.patch
error: patch failed: src/backend/storage/ipc/ipci.c:147
error: src/backend/storage/ipc/ipci.c: patch does not apply

I have applied v11 patches on commit - 23ba3b5ee278847e4fad913b80950edb2838fd35 to test further.

pg_basebackup has a new option "--no-estimate-size",  pg_basebackup crashes when using this option.

[edb@localhost bin]$ ./pg_basebackup -D /tmp/bkp --no-estimate-size --jobs=2
Segmentation fault (core dumped)

--stacktrace
[edb@localhost bin]$ gdb -q -c core.80438 pg_basebackup
Loaded symbols for /lib64/libselinux.so.1
Core was generated by `./pg_basebackup -D /tmp/bkp --no-estimate-size --jobs=2'.
Program terminated with signal 11, Segmentation fault.
#0  ____strtol_l_internal (nptr=0x0, endptr=0x0, base=10, group=<value optimized out>, loc=0x392158ee40) at ../stdlib/strtol_l.c:298
298  while (ISSPACE (*s))
Missing separate debuginfos, use: debuginfo-install keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-65.el6.x86_64 libcom_err-1.41.12-24.el6.x86_64 libselinux-2.0.94-7.el6.x86_64 openssl-1.0.1e-58.el6_10.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0  ____strtol_l_internal (nptr=0x0, endptr=0x0, base=10, group=<value optimized out>, loc=0x392158ee40) at ../stdlib/strtol_l.c:298
#1  0x0000003921233b30 in atoi (nptr=<value optimized out>) at atoi.c:28
#2  0x000000000040841e in main (argc=5, argv=0x7ffeaa6fb968) at pg_basebackup.c:2526

Thanks & Regards,
Rajkumar Raghuwanshi


On Tue, Apr 7, 2020 at 11:07 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Apr 7, 2020 at 1:25 PM Asif Rehman <asifr.rehman@gmail.com> wrote:
> I will, however parallel backup is already quite a large patch. So I think we should first
> agree on the current work before adding a backup manifest and progress-reporting support.

It's going to be needed for commit, but it may make sense for us to do
more review of what you've got here before we worry about it.

I'm gonna try to find some time for that as soon as I can.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


--
--
Asif Rehman
Highgo Software (Canada/China/Pakistan)
URL : www.highgo.ca

Attachment

pgsql-hackers by date:

Previous
From: Amit Kapila
Date:
Subject: Re: pg_stat_statements issue with parallel maintenance (Was Re: WALusage calculation patch)
Next
From: Julien Rouhaud
Date:
Subject: Re: pg_stat_statements issue with parallel maintenance (Was Re: WALusage calculation patch)