Re: [HACKERS] increasing the default WAL segment size - Mailing list pgsql-hackers

Thank you for your reviews Kuntal, Jim, Ashutosh

Attached in an updated 02 patch which:

1. Call RetrieveXLogSegSize(conn) in pg_receivewal.c
2. Remove the warning in Windows
3. Change PATH_MAX in pg_waldump with MAXPGPATH

Regarding the usage of the wal file size as the XLogSegSize, I agree with what Robert has said. Generally, the wal size will be of the expected wal_segment_size and to have it any other size, esspecially of a valid power2 value is extremely rare and I feel it is not a major cause of concern.

We (Prabhat and I) have started basic  testing of this feature -
2 quick issue -

1)at the time of initdb, we have set - "--wal-segsize 4"  ,so all the WAL file size should be 4 MB each  but in the postgresql.conf file , it is  mentioned

#wal_keep_segments = 0          # in logfile segments, 16MB each; 0 disables

so the comment  (16MB ) mentioned against parameter 'wal_keep_segments'  looks wrong , either we should remove this or modify it .

2)Getting "Aborted (core dumped)"  error at the time of running pg_basebackup  , (this issue is only coming on Linux32 ,not on Linux64)
 we have  double check to confirm it .

Steps to reproduce on Linux32
===================
fetch the sources
apply both the patches
 ./configure --with-zlib   --enable-debug  --enable-cassert  --enable-depend --prefix=$PWD/edbpsql --with-openssl CFLAGS="-g -O0"; make all install
Performed initdb with switch "--wal-segsize 4"
start the server
run pg_basebackup

[centos@tushar-centos bin]$ ./pg_basebackup -v -D /tmp/myslave
*** glibc detected *** ./pg_basebackup: free(): invalid pointer: 0x08da7f00 ***
======= Backtrace: =========
/lib/libc.so.6[0xae7e31]
/home/centos/pg10_10mar/postgresql/edbpsql/lib/libpq.so.5(PQclear+0x16d)[0x6266f5]
./pg_basebackup[0x8051441]
./pg_basebackup[0x804e7b5]
/lib/libc.so.6(__libc_start_main+0xe6)[0xa8dd26]
./pg_basebackup[0x804a231]
======= Memory map: ========
00153000-0017b000 r-xp 00000000 fc:01 1271       /lib/libk5crypto.so.3.1
0017b000-0017c000 r--p 00028000 fc:01 1271       /lib/libk5crypto.so.3.1
0017c000-0017d000 rw-p 00029000 fc:01 1271       /lib/libk5crypto.so.3.1
0017d000-0017e000 rw-p 00000000 00:00 0
0017e000-00180000 r-xp 00000000 fc:01 1241       /lib/libkeyutils.so.1.3
00180000-00181000 r--p 00001000 fc:01 1241       /lib/libkeyutils.so.1.3
00181000-00182000 rw-p 00002000 fc:01 1241       /lib/libkeyutils.so.1.3
002ad000-002b9000 r-xp 00000000 fc:01 1152       /lib/libnss_files-2.12.so
002b9000-002ba000 r--p 0000b000 fc:01 1152       /lib/libnss_files-2.12.so
002ba000-002bb000 rw-p 0000c000 fc:01 1152       /lib/libnss_files-2.12.so
004ad000-004b0000 r-xp 00000000 fc:01 1267       /lib/libcom_err.so.2.1
004b0000-004b1000 r--p 00002000 fc:01 1267       /lib/libcom_err.so.2.1
004b1000-004b2000 rw-p 00003000 fc:01 1267       /lib/libcom_err.so.2.1
004ec000-005c3000 r-xp 00000000 fc:01 1199       /lib/libkrb5.so.3.3
005c3000-005c9000 r--p 000d6000 fc:01 1199       /lib/libkrb5.so.3.3
005c9000-005ca000 rw-p 000dc000 fc:01 1199       /lib/libkrb5.so.3.3
00617000-00642000 r-xp 00000000 fc:01 2099439    /home/centos/pg10_10mar/postgresql/edbpsql/lib/libpq.so.5.10
00642000-00644000 rw-p 0002a000 fc:01 2099439    /home/centos/pg10_10mar/postgresql/edbpsql/lib/libpq.so.5.10
00792000-0079c000 r-xp 00000000 fc:01 1255       /lib/libkrb5support.so.0.1
0079c000-0079d000 r--p 00009000 fc:01 1255       /lib/libkrb5support.so.0.1
0079d000-0079e000 rw-p 0000a000 fc:01 1255       /lib/libkrb5support.so.0.1
007fd000-0083b000 r-xp 00000000 fc:01 1280       /lib/libgssapi_krb5.so.2.2
0083b000-0083c000 r--p 0003e000 fc:01 1280       /lib/libgssapi_krb5.so.2.2
0083c000-0083d000 rw-p 0003f000 fc:01 1280       /lib/libgssapi_krb5.so.2.2
0083f000-009ed000 r-xp 00000000 fc:01 292057     /usr/lib/libcrypto.so.1.0.1e
009ed000-009fd000 r--p 001ae000 fc:01 292057     /usr/lib/libcrypto.so.1.0.1e
009fd000-00a04000 rw-p 001be000 fc:01 292057     /usr/lib/libcrypto.so.1.0.1e
00a04000-00a07000 rw-p 00000000 00:00 0
00a51000-00a6f000 r-xp 00000000 fc:01 14109      /lib/ld-2.12.so
00a6f000-00a70000 r--p 0001d000 fc:01 14109      /lib/ld-2.12.so
00a70000-00a71000 rw-p 0001e000 fc:01 14109      /lib/ld-2.12.so
00a77000-00c08000 r-xp 00000000 fc:01 14110      /lib/libc-2.12.so
00c08000-00c0a000 r--p 00191000 fc:01 14110      /lib/libc-2.12.so
00c0a000-00c0b000 rw-p 00193000 fc:01 14110      /lib/libc-2.12.so
00c0b000-00c0e000 rw-p 00000000 00:00 0
00c10000-00c22000 r-xp 00000000 fc:01 14355      /lib/libz.so.1.2.3
00c22000-00c23000 r--p 00011000 fc:01 14355      /lib/libz.so.1.2.3
00c23000-00c24000 rw-p 00012000 fc:01 14355      /lib/libz.so.1.2.3
00c52000-00c55000 r-xp 00000000 fc:01 14375      /lib/libdl-2.12.so
00c55000-00c56000 r--p 00002000 fc:01 14375      /lib/libdl-2.12.so
00c56000-00c57000 rw-p 00003000 fc:01 14375      /lib/libdl-2.12.so
00c59000-00c70000 r-xp 00000000 fc:01 14379      /lib/libpthread-2.12.so
00c70000-00c71000 r--p 00016000 fc:01 14379      /lib/libpthread-2.12.so
00c71000-00c72000 rw-p 00017000 fc:01 14379      /lib/libpthread-2.12.so
00c72000-00c74000 rw-p 00000000 00:00 0
00d0a000-00d0b000 r-xp 00000000 00:00 0          [vdso]
00d8f000-00dac000 r-xp 00000000 fc:01 14392      /lib/libselinux.so.1
00dac000-00dad000 r--p 0001d000 fc:01 14392      /lib/libselinux.so.1
00dad000-00dae000 rw-p 0001e000 fc:01 14392      /lib/libselinux.so.1
00db0000-00dc5000 r-xp 00000000 fc:01 1430       /lib/libresolv-2.12.so
00dc5000-00dc6000 ---p 00015000 fc:01 1430       /lib/libresolv-2.12.so
00dc6000-00dc7000 r--p 00015000 fc:01 1430       /lib/libresolv-2.12.so
00dc7000-00dc8000 rw-p 00016000 fc:01 1430       /lib/libresolv-2.12.so
00dc8000-00dca000 rw-p 00000000 00:00 0
00dcc000-00de9000 r-xp 00000000 fc:01 1312       /lib/libgcc_s-4.4.7-20120601.so.1
00de9000-00dea000 rw-p 0001d000 fc:01 1312       /lib/libgcc_s-4.4.7-20120601.so.1
05576000-055d8000 r-xp 00000000 fc:01 275065     /usr/lib/libssl.so.1.0.1e
055d8000-055db000 r--p 00061000 fc:01 275065     /usr/lib/libssl.so.1.0.1e
055db000-055df000 rw-p 00064000 fc:01 275065     /usr/lib/libssl.so.1.0.1e
08048000-0805f000 r-xp 00000000 fc:01 2099490    /home/centos/pg10_10mar/postgresql/edbpsql/bin/pg_basebackup
0805f000-08060000 rw-p 00016000 fc:01 2099490    /home/centos/pg10_10mar/postgresql/edbpsql/bin/pg_basebackup
08060000-08062000 rw-p 00000000 00:00 0
08d9f000-08dc0000 rw-p 00000000 00:00 0          [heap]
b7519000-b7719000 r--p 00000000 fc:01 269751     /usr/lib/locale/locale-archive
b7719000-b771e000 rw-p 00000000 00:00 0
b772a000-b772c000 rw-p 00000000 00:00 0
bfbf6000-bfc0b000 rw-p 00000000 00:00 0          [stack]
Aborted (core dumped)
[centos@tushar-centos bin]$

same scenario is working fine against HEAD (v10 ) on Linux32 [i.e no patch applied] 

[centos@tushar-centos bin]$ ./pg_basebackup --verbose -D /tmp/slave11
pg_basebackup: initiating base backup, waiting for checkpoint to complete
pg_basebackup: checkpoint completed
pg_basebackup: transaction log start point: 0/2800024 on timeline 1
pg_basebackup: starting background WAL receiver
pg_basebackup: transaction log end point: 0/28000E4
pg_basebackup: waiting for background process to finish streaming ...
pg_basebackup: base backup completed
[centos@tushar-centos bin]$

-- 
regards,tushar
EnterpriseDB  https://www.enterprisedb.com/
The Enterprise PostgreSQL Company