Re: Table copy - Mailing list pgsql-general

From Ron Johnson
Subject Re: Table copy
Date
Msg-id CANzqJaCutOig_e9eW_9tUeKxxs7sX5rUtBthfk9hKOEb5HcTPQ@mail.gmail.com
Whole thread Raw
In response to Re: Table copy  (Andy Hartman <hartman60home@gmail.com>)
List pgsql-general

Could there have been a network hiccup?  Or some sort of timeout?

If I needed to transfer 360GB of data, I'd probably do something old school like:

1. write a PowerShell script to export a set of rows into a csv file, 7zip compress it, then rsync or scp it to the target.
2. Write a bash script to decompress it then load the data into the PG table.
3. Repeat (1) with the next set of data, and (2) until complete.  Start the second (1) while the first (2) is running.

That's how I migrated 12GB of Oracle data to PG (except of course bash, not PowerShell).

On Wed, Feb 5, 2025 at 4:05 PM Andy Hartman <hartman60home@gmail.com> wrote:
nothing in log from mssql side and no  anti-virus 

On Wed, Feb 5, 2025 at 2:06 PM Adrian Klaver <adrian.klaver@aklaver.com> wrote:


On 2/5/25 9:46 AM, Andy Hartman wrote:
> [6992] ERROR:  unexpected EOF on client connection with an open transaction
> 2025-02-05 12:19:44.919 EST [6992] CONTEXT:  COPY sqlt_data_1_2022_03,
> line 24431524, column dataintegrity
> 2025-02-05 12:19:44.919 EST [6992] STATEMENT:  COPY sqlt_data_1_2022_03
> (tagid, intvalue, floatvalue, stringvalue, datevalue, dataintegrity,
> t_stamp) FROM STDIN (FORMAT BINARY)
> 2025-02-05 12:19:44.919 EST [6992] FATAL:  terminating connection
> because protocol synchronization was lost

You need to look at the other end of the connection also, in other words
what is happening on the MS SQL Server 2016/Windows Server 2019 side?

Also is there anti-virus software running on either end?



--
Adrian Klaver
adrian.klaver@aklaver.com


--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

pgsql-general by date:

Previous
From: Andy Hartman
Date:
Subject: Re: Table copy
Next
From: Michał Kłeczek
Date:
Subject: Re: Lookup tables