Thread: Basebackup
How different is basebackup from new server with primary host and rsync from primary, move to new server and start cluster?
Hi Rajesh
Are you talking above backup taken with pg_basebackup with WAL Archiving enabled?
If yes then you can start a server on it on the new host.
Regards
Kashif Zeeshan
On Tue, Sep 3, 2024 at 5:31 PM Rajesh Kumar <rajeshkumar.dba09@gmail.com> wrote:
How different is basebackup from new server with primary host and rsync from primary, move to new server and start cluster?
The difference is that offloading resource consumption on primary while taking backup and copy it to new server.
So it’s better to run it from new server
On Tue, Sep 3, 2024 at 7:31 AM Rajesh Kumar <rajeshkumar.dba09@gmail.com> wrote:
How different is basebackup from new server with primary host and rsync from primary, move to new server and start cluster?
To overcome performance used Peer to Peer of both machines with CAT6A cable so that you get 10GBPS speed and this does not hamer primary server because separate network used for replication and also if possible used replication slot. This was already tested in 1.6 TB data. If you have problem in storage than used ( --format=tar --gzip) in pg_basebackup and then decompress it.
Regards,
Ajit W
On Tue, Sep 3, 2024 at 8:05 PM rams nalabolu <ramsveeru441@gmail.com> wrote:
The difference is that offloading resource consumption on primary while taking backup and copy it to new server.So it’s better to run it from new serverOn Tue, Sep 3, 2024 at 7:31 AM Rajesh Kumar <rajeshkumar.dba09@gmail.com> wrote:How different is basebackup from new server with primary host and rsync from primary, move to new server and start cluster?
On Tue, 2024-09-03 at 18:01 +0530, Rajesh Kumar wrote: > How different is basebackup from new server with primary host and rsync from primary, move to new server and start cluster? The main difference is that you have to stop the server if you want to use "rsync", otherwise the backup will be inconsistent. A backup taken with "pg_basebackup" will be recovered when you start it. You *can* perform an online file system backup with "rsync", but then you will have to use the low-level backup API: https://www.postgresql.org/docs/current/continuous-archiving.html#BACKUP-LOWLEVEL-BASE-BACKUP Yours, Laurenz Albe
Hi Rajesh,
pg_basebackup is easier, as it's a single command with PostgreSQL support. Data consistency is automatically ensured by postgresql. It transfers the entire database, potentially high network bandwidth is required. It is less flexible, due to mostly predefined options.
Here are a few differences between basebackup and rsync.
rsync is more complex, it requires multiple manual steps. This method requires stopping or ensuring the primary server's data directory is in a consistent state before starting the synchronization, Which can cause downtime or performance impact. It requires careful handling to ensure data consistency. It is more efficient with bandwidth, as it only transfers changes. It is more flexible with control over what gets copied.
You can use basebackup if you want a straightforward, reliable way to create a base backup. If you are looking for a method that integrates directly with PostgreSQL's replication capabilities.
Network bandwidth and performance impact on the primary server are not critical concerns. If you want an automated process that requires less manual intervention.
Network bandwidth and performance impact on the primary server are not critical concerns. If you want an automated process that requires less manual intervention.
You can use rsync if you have specific requirements for how the data should be copied (e.g., selective file copying). If you need more control over the synchronization process or want to optimize for minimal network usage. If you have a large dataset and want to minimize the data transfer by only copying changes after an initial sync. If you are comfortable with handling data consistency and can manage the additional complexity.
Regards,
Asad Ali
On Tue, Sep 3, 2024 at 5:31 PM Rajesh Kumar <rajeshkumar.dba09@gmail.com> wrote:
How different is basebackup from new server with primary host and rsync from primary, move to new server and start cluster?
Thanks Asad. I have two openshift environments in different data centers. I want to move one data from one DC to another Data center.
On Wed, 4 Sept 2024, 13:49 Asad Ali, <asadalinagri@gmail.com> wrote:
Hi Rajesh,Here are a few differences between basebackup and rsync.pg_basebackup is easier, as it's a single command with PostgreSQL support. Data consistency is automatically ensured by postgresql. It transfers the entire database, potentially high network bandwidth is required. It is less flexible, due to mostly predefined options.rsync is more complex, it requires multiple manual steps. This method requires stopping or ensuring the primary server's data directory is in a consistent state before starting the synchronization, Which can cause downtime or performance impact. It requires careful handling to ensure data consistency. It is more efficient with bandwidth, as it only transfers changes. It is more flexible with control over what gets copied.You can use basebackup if you want a straightforward, reliable way to create a base backup. If you are looking for a method that integrates directly with PostgreSQL's replication capabilities.
Network bandwidth and performance impact on the primary server are not critical concerns. If you want an automated process that requires less manual intervention.You can use rsync if you have specific requirements for how the data should be copied (e.g., selective file copying). If you need more control over the synchronization process or want to optimize for minimal network usage. If you have a large dataset and want to minimize the data transfer by only copying changes after an initial sync. If you are comfortable with handling data consistency and can manage the additional complexity.Regards,Asad AliOn Tue, Sep 3, 2024 at 5:31 PM Rajesh Kumar <rajeshkumar.dba09@gmail.com> wrote:How different is basebackup from new server with primary host and rsync from primary, move to new server and start cluster?
hi Rajesh,
But ... if you want to migrate your postgres from location A to location B (that is what it somehow sounds like) I guess ...
On 9/4/24 11:44 AM, Rajesh Kumar wrote:
it is not 100% clear what exactly you want to achieve.... I have two openshift environments in different data centers. I want to move one data from one DC to another Data center.
But ... if you want to migrate your postgres from location A to location B (that is what it somehow sounds like) I guess ...
- setting up a replica on the target (B) via pg_basebackup
- stopping your application and the old main (A)
- promoting the replica (B) to become the new main
- re-directing your application to the target (B)
On Wed, 4 Sept 2024, 13:49 Asad Ali, <asadalinagri@gmail.com> wrote:Hi Rajesh,Here are a few differences between basebackup and rsync.pg_basebackup is easier, as it's a single command with PostgreSQL support. Data consistency is automatically ensured by postgresql. It transfers the entire database, potentially high network bandwidth is required. It is less flexible, due to mostly predefined options.rsync is more complex, it requires multiple manual steps. This method requires stopping or ensuring the primary server's data directory is in a consistent state before starting the synchronization, Which can cause downtime or performance impact. It requires careful handling to ensure data consistency. It is more efficient with bandwidth, as it only transfers changes. It is more flexible with control over what gets copied.You can use basebackup if you want a straightforward, reliable way to create a base backup. If you are looking for a method that integrates directly with PostgreSQL's replication capabilities.
Network bandwidth and performance impact on the primary server are not critical concerns. If you want an automated process that requires less manual intervention.You can use rsync if you have specific requirements for how the data should be copied (e.g., selective file copying). If you need more control over the synchronization process or want to optimize for minimal network usage. If you have a large dataset and want to minimize the data transfer by only copying changes after an initial sync. If you are comfortable with handling data consistency and can manage the additional complexity.Regards,Asad AliOn Tue, Sep 3, 2024 at 5:31 PM Rajesh Kumar <rajeshkumar.dba09@gmail.com> wrote:How different is basebackup from new server with primary host and rsync from primary, move to new server and start cluster?
Physical replication using PgBackrest would probably be even faster, because parallelism will better use the network bandwidth.
On Mon, Sep 9, 2024 at 3:18 PM vrms <vrms@netcologne.de> wrote:
hi Rajesh,On 9/4/24 11:44 AM, Rajesh Kumar wrote:it is not 100% clear what exactly you want to achieve.... I have two openshift environments in different data centers. I want to move one data from one DC to another Data center.
But ... if you want to migrate your postgres from location A to location B (that is what it somehow sounds like) I guess ...could be quite efficient.
- setting up a replica on the target (B) via pg_basebackup
- stopping your application and the old main (A)
- promoting the replica (B) to become the new main
- re-directing your application to the target (B)
On Wed, 4 Sept 2024, 13:49 Asad Ali, <asadalinagri@gmail.com> wrote:Hi Rajesh,Here are a few differences between basebackup and rsync.pg_basebackup is easier, as it's a single command with PostgreSQL support. Data consistency is automatically ensured by postgresql. It transfers the entire database, potentially high network bandwidth is required. It is less flexible, due to mostly predefined options.rsync is more complex, it requires multiple manual steps. This method requires stopping or ensuring the primary server's data directory is in a consistent state before starting the synchronization, Which can cause downtime or performance impact. It requires careful handling to ensure data consistency. It is more efficient with bandwidth, as it only transfers changes. It is more flexible with control over what gets copied.You can use basebackup if you want a straightforward, reliable way to create a base backup. If you are looking for a method that integrates directly with PostgreSQL's replication capabilities.
Network bandwidth and performance impact on the primary server are not critical concerns. If you want an automated process that requires less manual intervention.You can use rsync if you have specific requirements for how the data should be copied (e.g., selective file copying). If you need more control over the synchronization process or want to optimize for minimal network usage. If you have a large dataset and want to minimize the data transfer by only copying changes after an initial sync. If you are comfortable with handling data consistency and can manage the additional complexity.Regards,Asad AliOn Tue, Sep 3, 2024 at 5:31 PM Rajesh Kumar <rajeshkumar.dba09@gmail.com> wrote:How different is basebackup from new server with primary host and rsync from primary, move to new server and start cluster?
Death to America, and butter sauce.
Iraq lobster!