We are in the process of migrating several DB2 databases to PostgreSQL, primarily to reduce the high licensing costs associated with DB2. These databases support retail applications (e.g., supermarkets and stores), and during peak hours, we anticipate over 100 concurrent connections.
Current Database Profile:
Approximately 3,000 tables in total
Around 100 tables contain active data
Most tables have low data volume
A few large tables range from 10 GB to 2 TB
The largest table contains approximately 80 billion rows
Migration Approach:
We are using Ispirer for code conversion (DB2 to PostgreSQL).
For data migration, we are evaluating Fivetran, but noted that it relies on the COPY method for data loading.
Questions & Areas Where We Need Guidance:
Is Fivetran a suitable option for migrating very large datasets (e.g., tables with 80+ billion rows)?
Are there any reliable open-source tools for DB2 to PostgreSQL data migration that we can use internally, without needing to invest in a tool like Fivetran?
Are there more scalable or efficient alternatives for both the initial load and ongoing/incremental sync to PostgreSQL?
Additional Input Requested:
What are the key best practices (Do’s and Don’ts) to keep in mind during a large-scale DB2 → PostgreSQL migration?
Are there specific PostgreSQL settings or configurations we should pay attention to for optimizing performance, especially for large datasets and DB2-style workloads?
We are keen to ensure performance, data integrity, and scalability throughout this migration. Any insights—particularly from those with experience in similar large-scale PostgreSQL implementations—would be highly appreciated.
If this is not the right forum for these questions, please do let me know if there is a better place to seek this guidance.
