Thread: partial data migration

On 3/7/19 1:54 AM, Julie Nishimura wrote:

P {margin-top:0;margin-bottom:0;}

Hello psql friends,

We need to migrate only 6 months worth of data from one instance to another. What would be the easiest way to do it? In Oracle, I would set up dblink. What about postgresql?

postgres_fdw

Thank you Ron! What if dev environment is on 9.6, but prod is on version 8.3? Will posgtres_fdw still be the right option? 

On 3/7/19 1:54 AM, Julie Nishimura wrote:

Hello psql friends,

We need to migrate only 6 months worth of data from one instance to another. What would be the easiest way to do it? In Oracle, I would set up dblink. What about postgresql?

postgres_fdw

--
Angular momentum makes the world go 'round.

(8.3?  That's even older than what we just migrated from!!!)

No.  Make some views (I'd probably make them "month-sized"), COPY each view from the source db to a file, and then COPY each file to it's relevant target table.

You should also think about a program named pg_bulkload.

https://www.postgresql.org/docs/9.6/sql-copy.html
https://www.postgresql.org/docs/8.3/sql-copy.html

Thank you Ron! What if dev environment is on 9.6, but prod is on version 8.3? Will posgtres_fdw still be the right option? 

Sent from my iPhone

On Mar 6, 2019, at 11:57 PM, Ron <ronljohnsonjr@gmail.com> wrote:

On 3/7/19 1:54 AM, Julie Nishimura wrote:

Hello psql friends,

We need to migrate only 6 months worth of data from one instance to another. What would be the easiest way to do it? In Oracle, I would set up dblink. What about postgresql?

postgres_fdw

Yes, for whole tables (even sets of tables) "pg_dump --table=" is good at that.  Even better, you can run the 9.6 pg_dump against the 8.3 database and get parallelism with "--jobs".

P {margin-top:0;margin-bottom:0;}

Ron, thanksagain. In case if I need to migrate the entire tables, I should be able to use pg_dump and pg_restore for certain tables, even between different versions, right? In case if I need to migrate from 8 to 9?

Thanks

---

From: Ron <ronljohnsonjr@gmail.com>
Sent: Thursday, March 7, 2019 5:59 PM
To: pgsql-general@lists.postgresql.org
Subject: Re: partial data migration

 

(8.3?  That's even older than what we just migrated from!!!)

No.  Make some views (I'd probably make them "month-sized"), COPY each view from the source db to a file, and then COPY each file to it's relevant target table.

You should also think about a program named pg_bulkload.

https://www.postgresql.org/docs/9.6/sql-copy.html
https://www.postgresql.org/docs/8.3/sql-copy.html

On 3/7/19 7:53 PM, Julie Nishimura wrote:

Thank you Ron! What if dev environment is on 9.6, but prod is on version 8.3? Will posgtres_fdw still be the right option? 

Sent from my iPhone

On Mar 6, 2019, at 11:57 PM, Ron <ronljohnsonjr@gmail.com> wrote:

On 3/7/19 1:54 AM, Julie Nishimura wrote:

Hello psql friends,

We need to migrate only 6 months worth of data from one instance to another. What would be the easiest way to do it? In Oracle, I would set up dblink. What about postgresql?

postgres_fdw

--
Angular momentum makes the world go 'round.

Great, thanks!

Yes, for whole tables (even sets of tables) "pg_dump --table=" is good at that.  Even better, you can run the 9.6 pg_dump against the 8.3 database and get parallelism with "--jobs".

On 3/7/19 8:11 PM, Julie Nishimura wrote:

Ron, thanksagain. In case if I need to migrate the entire tables, I should be able to use pg_dump and pg_restore for certain tables, even between different versions, right? In case if I need to migrate from 8 to 9?

Thanks

---

From: Ron <ronljohnsonjr@gmail.com>
Sent: Thursday, March 7, 2019 5:59 PM
To: pgsql-general@lists.postgresql.org
Subject: Re: partial data migration

 

(8.3?  That's even older than what we just migrated from!!!)

No.  Make some views (I'd probably make them "month-sized"), COPY each view from the source db to a file, and then COPY each file to it's relevant target table.

You should also think about a program named pg_bulkload.

https://www.postgresql.org/docs/9.6/sql-copy.html
https://www.postgresql.org/docs/8.3/sql-copy.html

On 3/7/19 7:53 PM, Julie Nishimura wrote:

Thank you Ron! What if dev environment is on 9.6, but prod is on version 8.3? Will posgtres_fdw still be the right option? 

Sent from my iPhone

On Mar 6, 2019, at 11:57 PM, Ron <ronljohnsonjr@gmail.com> wrote:

On 3/7/19 1:54 AM, Julie Nishimura wrote:

Hello psql friends,

We need to migrate only 6 months worth of data from one instance to another. What would be the easiest way to do it? In Oracle, I would set up dblink. What about postgresql?

postgres_fdw

--
Angular momentum makes the world go 'round.

--
Angular momentum makes the world go 'round.

On 3/7/19 5:53 PM, Julie Nishimura wrote:
> Thank you Ron! What if dev environment is on 9.6, but prod is on version 
> 8.3? Will posgtres_fdw still be the right option?

Thanks to the documentation writers:

https://www.postgresql.org/docs/11/postgres-fdw.html#id-1.11.7.42.15

> 
> Sent from my iPhone
> 
> On Mar 6, 2019, at 11:57 PM, Ron <ronljohnsonjr@gmail.com 
> <mailto:ronljohnsonjr@gmail.com>> wrote:
> 
>> On 3/7/19 1:54 AM, Julie Nishimura wrote:
>>> Hello psql friends,
>>> We need to migrate only 6 months worth of data from one instance to 
>>> another. What would be the easiest way to do it? In Oracle, I would 
>>> set up dblink. What about postgresql?
>>
>> postgres_fdw
>>
>> -- 
>> Angular momentum makes the world go 'round.


-- 
Adrian Klaver
adrian.klaver@aklaver.com