Thread: pg_rewind fails with in-place tablespace

pg_rewind fails with in-place tablespace

From
"赵锐(惜元)"
Date:
Hello postgres hackers,
    Recently I encountered an issue: pg_rewind fails when dealing with in-place tablespace. The problem seems to be that pg_rewind is treating in-place tablespace as symbolic link, while in fact it should be treated as directory.

    Here is the output of pg_rewind:
    pg_rewind: error: file "pg_tblspc/16385" is of different type in source and target

    To help reproduce the failure, I have attached a tap test. And I am pleased to say that I have also identified a solution for this problem, which I have included in the patch.
    Thank you for your attention to this matter.

Best regards,
Rui Zhao
Attachment

Re: pg_rewind fails with in-place tablespace

From
Michael Paquier
Date:
On Wed, Jul 19, 2023 at 09:31:35PM +0800, 赵锐(惜元) wrote:
>  Recently I encountered an issue: pg_rewind fails when dealing with
>  in-place tablespace. The problem seems to be that pg_rewind is
>  treating in-place tablespace as symbolic link, while in fact it
>  should be treated as directory.
>  Here is the output of pg_rewind:
> pg_rewind: error: file "pg_tblspc/16385" is of different type in
> source and target
>  To help reproduce the failure, I have attached a tap test. And I am
>  pleased to say that I have also identified a solution for this
>  problem, which I have included in the patch.
>  Thank you for your attention to this matter.

Issue reproduced here, and agreed that we'd better do something about
that.  I am not sure if your patch is right for the job though, but
I'll try to study that a bit more.
--
Michael

Attachment

Re: pg_rewind fails with in-place tablespace

From
Michael Paquier
Date:
On Tue, Jul 25, 2023 at 04:36:42PM +0900, Michael Paquier wrote:
> On Wed, Jul 19, 2023 at 09:31:35PM +0800, 赵锐(惜元) wrote:
>>  To help reproduce the failure, I have attached a tap test. And I am
>>  pleased to say that I have also identified a solution for this
>>  problem, which I have included in the patch.
>>  Thank you for your attention to this matter.
>
> Issue reproduced here, and agreed that we'd better do something about
> that.  I am not sure if your patch is right for the job though, but
> I'll try to study that a bit more.

It took me some time to remember that for the case of a local source
we'd finish by using recurse_dir() and consider the in-place
tablespace as a regular directory, so a fix located in
libpq_traverse_files() sounds good to me.

+           if (strncmp(link_target, "pg_tblspc/", strlen("pg_tblspc/")) == 0)
+               type = FILE_TYPE_DIRECTORY;
+           else
+               type = FILE_TYPE_SYMLINK;

However this is not consistent with the other places where we detect
if an in-place tablespace is used, like pg_basebackup.c, where we rely
on the fact that the tablespace path is a relative path, using
is_absolute_path() to make the difference between a normal and
in-place tablespace.  I would choose consistency and do the same here,
checking if we have an absolute or relative path, depending on the
result of pg_tablespace_location().

Testing only for the creation of the tablespace is fine for the sake
of the report, but I would slightly more here and create a table on
this tablespace with some data, and a check_query() once pg_rewind is
done.

I am finishing with the attached.  Thoughts?
--
Michael

Attachment

Re: pg_rewind fails with in-place tablespace

From
Michael Paquier
Date:
On Fri, Jul 28, 2023 at 04:54:56PM +0900, Michael Paquier wrote:
> I am finishing with the attached.  Thoughts?

Applied this one as bf22792 on HEAD, without a backpatch as in-place
tablespaces are around for developers.  If there are opinions in favor
of a backpatch, feel free of course.
--
Michael

Attachment

回复:pg_rewind fails with in-place tablespace

From
"Rui Zhao"
Date:
Sorry for the delay in responding to this matter as I have been waiting for another similar subject to approved by a moderator.

Upon review, I am satisfied with the proposed solution and believe that checking absolute path is better than hard coding with "pg_tblspc/". I think we have successfully resolved this issue in the pg_rewind case.

However, I would like to bring your attention to another issue: pg_upgrade fails with in-place tablespace. Another issue is still waiting for approved. I have tested all the tools in src/bin with in-place tablespace, and I believe this is the final issue.

Thank you for your understanding and assistance.

Best regard,
Rui Zhao




------------------------------------------------------------------
发件人:Michael Paquier <michael@paquier.xyz>
发送时间:2023年7月31日(星期一) 06:49
收件人:赵锐(惜元) <xiyuan.zr@alibaba-inc.com>
抄 送:pgsql-hackers <pgsql-hackers@lists.postgresql.org>; Thomas Munro <thomas.munro@gmail.com>
主 题:Re: pg_rewind fails with in-place tablespace

On Fri, Jul 28, 2023 at 04:54:56PM +0900, Michael Paquier wrote:
> I am finishing with the attached.  Thoughts?

Applied this one as bf22792 on HEAD, without a backpatch as in-place
tablespaces are around for developers.  If there are opinions in favor
of a backpatch, feel free of course.
--
Michael

Re: 回复:pg_rewind fails with in-place tablespace

From
Michael Paquier
Date:
On Mon, Jul 31, 2023 at 10:07:44AM +0800, Rui Zhao wrote:
> However, I would like to bring your attention to another issue:
> pg_upgrade fails with in-place tablespace. Another issue is still
> waiting for approved. I have tested all the tools in src/bin with
> in-place tablespace, and I believe this is the final issue.

No problem.  Please feel free to start a new thread about that, I'm
okay to look at what you would like to propose.  Adding a test in
002_pg_upgrade.pl where the pg_upgrade runs happen would be a good
thing to have, I guess.
--
Michael

Attachment