Thread: error on online backup using pg_basebackup tool
Hi 4 all!!
--
As Ido not know if it is my mistake or actually a bug, sending the two topics.
When trying to perform online backup using the pg_basebackup tool, I get the message: The folder is not empty.
steps to reproduce the error:
1 - install postgres in the / usr / local / pgsql
2 - initialize the database: / usr / local / pgsql / bin / -D /data
3 - set up pg_hba.conf to allow replication (used by online backup with pg_basebackup)
4 - create the directories /data/newtblspc and /data/pg_tblspc/newtblspc2
5 - start with postgres -D /data
6 - create table spaces:
CREATE TABLESPACE newtblspc LOCATION '/data/newtblspc';
CREATE TABLESPACE newtblspc2 LOCATION '/data/pg_tblspc/newtblspc2'
7 - mkdir /data2
8 - Now perform a backup using pg_basebackup -D /data2 and get the error message.
--
T.'.A.'.F.'.,
Gerdan Rezende dos Santos
PostgreSQL & EnterpriseDB Specialist, Support, Training & Services
+55 (61) 9645-1525
Gerdan Rezende dos Santos
PostgreSQL & EnterpriseDB Specialist, Support, Training & Services
+55 (61) 9645-1525
On Thu, Sep 3, 2015 at 10:28 AM, Gerdan Rezende dos Santos <gerdan@gmail.com> wrote: > 6 - create table spaces: > CREATE TABLESPACE newtblspc LOCATION '/data/newtblspc'; > CREATE TABLESPACE newtblspc2 LOCATION '/data/pg_tblspc/newtblspc2' > 7 - mkdir /data2 > 8 - Now perform a backup using pg_basebackup -D /data2 and get the error > message. You are performing a base backup on a node that has tablespaces on the same hosts, so tablespaces are going to overlap. Isn't that your problem? Note that 9.4 you can use tablespace-mapping to address the problem, for older versions you are going to need some manual operations before being able to run the standby and the master on the same server. -- Michael
I use the tablespaces without problem but the backup online not complete... but when I attempt to online backup using the pg_basebackup displays the error message saying q pg_tblspc is not clean and the process is terminated without sucess
Em quarta-feira, 2 de setembro de 2015, Michael Paquier <michael.paquier@gmail.com> escreveu:
On Thu, Sep 3, 2015 at 10:28 AM, Gerdan Rezende dos Santos
<gerdan@gmail.com> wrote:
> 6 - create table spaces:
> CREATE TABLESPACE newtblspc LOCATION '/data/newtblspc';
> CREATE TABLESPACE newtblspc2 LOCATION '/data/pg_tblspc/newtblspc2'
> 7 - mkdir /data2
> 8 - Now perform a backup using pg_basebackup -D /data2 and get the error
> message.
You are performing a base backup on a node that has tablespaces on the
same hosts, so tablespaces are going to overlap. Isn't that your
problem? Note that 9.4 you can use tablespace-mapping to address the
problem, for older versions you are going to need some manual
operations before being able to run the standby and the master on the
same server.
--
Michael
--
T.'.A.'.F.'.,
Gerdan Rezende dos Santos
PostgreSQL & EnterpriseDB Specialist, Support, Training & Services
+55 (61) 9645-1525
Gerdan Rezende dos Santos
PostgreSQL & EnterpriseDB Specialist, Support, Training & Services
+55 (61) 9645-1525
On Thu, Sep 3, 2015 at 12:20 PM, Gerdan Rezende dos Santos <gerdan@gmail.com> wrote: >[...] Note 1: Please don't top-post. Note 2: Please don't post across multiple mailing lists. > I use the tablespaces without problem but the backup online not complete... > but when I attempt to online backup using the pg_basebackup displays the > error message saying q pg_tblspc is not clean and the process is terminated > without sucess. That's what I am saying. You will need to redirect manually the soft links of pg_tbspc into other paths if you want to run a standby on the same host as the master. That's not a bug, just an operation mistake. -- Michael
Em quinta-feira, 3 de setembro de 2015, Michael Paquier <michael.paquier@gmail.com> escreveu:
On Thu, Sep 3, 2015 at 12:20 PM, Gerdan Rezende dos Santos
<gerdan@gmail.com> wrote:
>[...]
Note 1: Please don't top-post.
Note 2: Please don't post across multiple mailing lists.
> I use the tablespaces without problem but the backup online not complete...
> but when I attempt to online backup using the pg_basebackup displays the
> error message saying q pg_tblspc is not clean and the process is terminated
> without sucess.
That's what I am saying. You will need to redirect manually the soft
links of pg_tbspc into other paths if you want to run a standby on the
same host as the master.
That's not a bug, just an operation mistake.
--
Michael
In other server with same configuration the proccess not complete too!
--
T.'.A.'.F.'.,
Gerdan Rezende dos Santos
PostgreSQL & EnterpriseDB Specialist, Support, Training & Services
+55 (61) 9645-1525
Gerdan Rezende dos Santos
PostgreSQL & EnterpriseDB Specialist, Support, Training & Services
+55 (61) 9645-1525
El 03/09/15 a las 00:54, Gerdan Rezende dos Santos escribió: > > > In other server with same configuration the proccess not complete too! Add --verbose to the pg_basebackup execution and send over the output error message. I find your procedure confusing, and I'd recommend you *not* to put tablespaces in $PGDTA/pg_tblspc/. That directory is for postgres to put the links to where the actual tablespace is (normally on another partition) -- Martín Marqués http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Thu, Sep 3, 2015 at 7:20 AM, Martín Marqués <martin@2ndquadrant.com> wrote:
El 03/09/15 a las 00:54, Gerdan Rezende dos Santos escribió:
>
>
> In other server with same configuration the proccess not complete too!
Add --verbose to the pg_basebackup execution and send over the output
error message.
I find your procedure confusing, and I'd recommend you *not* to put
tablespaces in $PGDTA/pg_tblspc/. That directory is for postgres to put
the links to where the actual tablespace is (normally on another partition)
--
Martín Marqués http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
This message on verbose mode: pg_basebackup: directory "/data/pg_tblspc/" exists but is not empty
From machine A to machine B
pb_basebackup started in machine B, connecting on machine A
On machine B
pb_basebackup -D /data/pg_tblspc/ -h machineA
T.'.A.'.F.'.,
Gerdan Rezende dos Santos
PostgreSQL & EnterpriseDB Specialist, Support, Training & Services
+55 (61) 9645-1525
Gerdan Rezende dos Santos
PostgreSQL & EnterpriseDB Specialist, Support, Training & Services
+55 (61) 9645-1525
2015-09-03 16:48 GMT-03:00 Gerdan Rezende dos Santos <gerdan@gmail.com>: > On Thu, Sep 3, 2015 at 7:20 AM, Martín Marqués <martin@2ndquadrant.com> > wrote: >> >> El 03/09/15 a las 00:54, Gerdan Rezende dos Santos escribió: >> > >> > >> > In other server with same configuration the proccess not complete too! >> >> Add --verbose to the pg_basebackup execution and send over the output >> error message. >> >> I find your procedure confusing, and I'd recommend you *not* to put >> tablespaces in $PGDTA/pg_tblspc/. That directory is for postgres to put >> the links to where the actual tablespace is (normally on another >> partition) > > This message on verbose mode: pg_basebackup: directory "/data/pg_tblspc/" > exists but is not empty The error message is quite clear, it has to use /data/pg_tblspc/ to put a tablespace, but that directory is not empty. Why are you putting tablespaces inside PGDATA? It really makes no sense. Just leave the objects you have there in the default tablespace. Regards, -- Martín Marqués http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Thu, Sep 3, 2015 at 5:01 PM, Martín Marqués <martin@2ndquadrant.com> wrote:
2015-09-03 16:48 GMT-03:00 Gerdan Rezende dos Santos <gerdan@gmail.com>:
> On Thu, Sep 3, 2015 at 7:20 AM, Martín Marqués <martin@2ndquadrant.com>
> wrote:
>>
>> El 03/09/15 a las 00:54, Gerdan Rezende dos Santos escribió:
>> >
>> >
>> > In other server with same configuration the proccess not complete too!
>>
>> Add --verbose to the pg_basebackup execution and send over the output
>> error message.
>>
>> I find your procedure confusing, and I'd recommend you *not* to put
>> tablespaces in $PGDTA/pg_tblspc/. That directory is for postgres to put
>> the links to where the actual tablespace is (normally on another
>> partition)
>
> This message on verbose mode: pg_basebackup: directory "/data/pg_tblspc/"
> exists but is not empty
The error message is quite clear, it has to use /data/pg_tblspc/ to
put a tablespace, but that directory is not empty.
Why are you putting tablespaces inside PGDATA? It really makes no
sense. Just leave the objects you have there in the default
tablespace.
Regards,
--
Martín Marqués http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
This problem, I found in my client ...
Why are you putting in PGDATA table ? He can not say to me
The solution then is to continue using the start_backup , rsync and stop_backup. ;(
Thanks!
Hi Gerdan, 2015-09-03 17:41 GMT-03:00 Gerdan Rezende dos Santos <gerdan@gmail.com>: > > This problem, I found in my client ... > Why are you putting in PGDATA table ? He can not say to me > The solution then is to continue using the start_backup , rsync and > stop_backup. ;( Or use repmgr. ;) If you intend to use repmgr3 (will only work with postgres 9.3 and above) you'll need to pass it the --rsync-only option so it doesn't use pg_basebackup to clone. Regards, -- Martín Marqués http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Thu, Sep 3, 2015 at 5:53 PM, Martín Marqués <martin@2ndquadrant.com> wrote:
Hi Gerdan,
2015-09-03 17:41 GMT-03:00 Gerdan Rezende dos Santos <gerdan@gmail.com>:
>
> This problem, I found in my client ...
> Why are you putting in PGDATA table ? He can not say to me
> The solution then is to continue using the start_backup , rsync and
> stop_backup. ;(
Or use repmgr. ;)
If you intend to use repmgr3 (will only work with postgres 9.3 and
above) you'll need to pass it the --rsync-only option so it doesn't
use pg_basebackup to clone.
Regards,
--
Martín Marqués http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Matín, the client have a BART from EDB.
EDB BART only use pg_basebackup.
Its my problem.
On Thu, Sep 3, 2015 at 4:41 PM, Gerdan Rezende dos Santos <gerdan@gmail.com> wrote:
On Thu, Sep 3, 2015 at 5:01 PM, Martín Marqués <martin@2ndquadrant.com> wrote:2015-09-03 16:48 GMT-03:00 Gerdan Rezende dos Santos <gerdan@gmail.com>:
> On Thu, Sep 3, 2015 at 7:20 AM, Martín Marqués <martin@2ndquadrant.com>
> wrote:
>>
>> El 03/09/15 a las 00:54, Gerdan Rezende dos Santos escribió:
>> >
>> >
>> > In other server with same configuration the proccess not complete too!
>>
>> Add --verbose to the pg_basebackup execution and send over the output
>> error message.
>>
>> I find your procedure confusing, and I'd recommend you *not* to put
>> tablespaces in $PGDTA/pg_tblspc/. That directory is for postgres to put
>> the links to where the actual tablespace is (normally on another
>> partition)
>
> This message on verbose mode: pg_basebackup: directory "/data/pg_tblspc/"
> exists but is not empty
The error message is quite clear, it has to use /data/pg_tblspc/ to
put a tablespace, but that directory is not empty.
Why are you putting tablespaces inside PGDATA? It really makes no
sense. Just leave the objects you have there in the default
tablespace.
Regards,
--
Martín Marqués http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & ServicesThis problem, I found in my client ...Why are you putting in PGDATA table ? He can not say to meThe solution then is to continue using the start_backup , rsync and stop_backup. ;(
You can always move the folder out of there. Depending on the version of PostgreSQL different steps need to be take. If I remember correctly for version 9.3 you can move the symlink to point to the new directory. Pre 9.3 you need to move the symlink and the pointer inside the pg tables.
-Joseph Kregloh
Thanks!
Hi Gerdan, 2015-09-03 18:06 GMT-03:00 Gerdan Rezende dos Santos <gerdan@gmail.com>: > > Matín, the client have a BART from EDB. > EDB BART only use pg_basebackup. > Its my problem. AFAIK BART is a backup & recovery tool, and I thought you were trying to spin new standbys, which is why I recommended repmgr. If you need to do backups then repmgr is not the way to go. There are other alternatives. Regards, -- Martín Marqués http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On 9/3/15 5:07 PM, Joseph Kregloh wrote: > You can always move the folder out of there. Depending on the version of > PostgreSQL different steps need to be take. If I remember correctly for > version 9.3 you can move the symlink to point to the new directory. Pre > 9.3 you need to move the symlink and the pointer inside the pg tables. You can move the directory and modify the symlink on 9.2 or later without any additional complications. However - and this is very important - PostgreSQL must be shut down during this operation. My recommendation would be to move both tablespaces out of $PGDATA. There has been a lot of discussion about disallowing it in future releases as it causes problems for various tools. -- -David david@pgmasters.net