Re: New option for pg_basebackup, to specify a different directory for pg_xlog - Mailing list pgsql-hackers

From Gavin Flower
Subject Re: New option for pg_basebackup, to specify a different directory for pg_xlog
Date
Msg-id 528CFBCF.5050900@archidevsys.co.nz
Whole thread Raw
In response to Re: New option for pg_basebackup, to specify a different directory for pg_xlog  (Haribabu kommi <haribabu.kommi@huawei.com>)
Responses Re: New option for pg_basebackup, to specify a different directory for pg_xlog  (Haribabu kommi <haribabu.kommi@huawei.com>)
List pgsql-hackers
<div class="moz-cite-prefix">On 20/11/13 23:43, Haribabu kommi wrote:<br /></div><blockquote
cite="mid:8977CB36860C5843884E0A18D8747B0372BEECC6@szxeml558-mbs.china.huawei.com"type="cite"><pre wrap="">On 19
November2013 19:12 Fujii Masao wrote:
 
</pre><blockquote type="cite"><pre wrap="">On Tue, Nov 19, 2013 at 9:14 PM, Haribabu kommi
<a class="moz-txt-link-rfc2396E" href="mailto:haribabu.kommi@huawei.com"><haribabu.kommi@huawei.com></a> wrote:
</pre><blockquote type="cite"><pre wrap="">On 18 November 2013 23:30 Fujii Masao wrote:
</pre><blockquote type="cite"><pre wrap="">On Tue, Nov 19, 2013 at 12:01 AM, Haribabu kommi
<a class="moz-txt-link-rfc2396E" href="mailto:haribabu.kommi@huawei.com"><haribabu.kommi@huawei.com></a> wrote:

Thanks for newer version of the patch!

I found that the empty base directory is created and remains even
when the same directory is specified in both -D and --xlogdir and
</pre></blockquote></blockquote><pre wrap="">the
</pre><blockquote type="cite"><blockquote type="cite"><pre wrap="">error occurs.
I think that it's
better to throw an error in that case before creating any new
</pre></blockquote></blockquote><pre wrap="">directory.
</pre><blockquote type="cite"><blockquote type="cite"><pre wrap="">Thought?
</pre></blockquote><pre wrap="">
By creating the base directory only the patch finds whether provided
base and Xlog directories are same or not? To solve this problem
following options are possible

1. No problem as it is just an empty base directory, so it can be
</pre></blockquote><pre wrap="">reused in the next time
</pre><blockquote type="cite"><pre wrap="">   Leave it as it is.
2. Once the error is identified, the base directory can be deleted.
3. write a new function to remove the parent references from the
</pre></blockquote><pre wrap="">provided path to identify
</pre><blockquote type="cite"><pre wrap="">   the absolute path used for detecting base and Xlog directories are
</pre></blockquote><pre wrap="">same or not?
</pre><blockquote type="cite"><pre wrap="">
Please provide your suggestions.

</pre><blockquote type="cite"><pre wrap="">+    xlogdir = get_absolute_path(xlog_dir);

xlog_dir must be an absolute path. ISTM we don't do the above. No?
</pre></blockquote><pre wrap="">
It is required. As user can provide the path as
</pre></blockquote><pre wrap="">/home/installation/bin/../bin/data.
</pre><blockquote type="cite"><pre wrap="">The provided path is considered as absolute path only but while
comparing the same With data directory path it will not match.
</pre></blockquote><pre wrap="">
Okay, maybe I understand you. In order to know the real absolute path,
basically we need to create the directory specified in --xlogdir,
change the working directory to it and calculate the parent path.
You're worried about the cases as follows, for example.
Right?

* path containing ".." like /aaa/bbb/../ccc is specified in --xlogdir
* symbolic link is specified in --xlogdir

On the second thought, I'm thinking that it might be overkill to add
such not simple code for that small benefit.
</pre></blockquote><pre wrap="">
I tried using of stat'ing in two directories, which is having a problem in windows.
So modified old approach to detect limited errors. Updated patch attached. 
This will detect and throw an error in the following scenarios.
1. pg_basebackup -D /home/data --xlogdir=/home/data
2. pg_basebackup -D data --xlogdir=/home/data -- home is the CWD
3. pg_basebackup -D ../data --xlogdir=/data -- home is the CWD

Please let me know your suggestions.

Regards,
Hari babu.
</pre><br /><fieldset class="mimeAttachmentHeader"></fieldset><br /><pre wrap="">
</pre></blockquote> I don't think Postgres on other systems should be hobbled by the limitations of Microsoft
software!<br/><br /> If certain features of Postgres are either not available, or are available in a reduced form on
Microsoftplatforms, then this should be documented - might provide subtle hints to upgrade to Linux.<br /><br /><br />
Cheers,<br/> Gavin<br /> 

pgsql-hackers by date:

Previous
From: Josh Berkus
Date:
Subject: Re: Storage formats for JSON WAS: additional json functionality
Next
From: Josh Berkus
Date:
Subject: Re: Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1