Re: pg_dump seems to be broken in regards to the "--exclude-table-data" option on Windows. - Mailing list pgsql-bugs

From tutiluren@tutanota.com
Subject Re: pg_dump seems to be broken in regards to the "--exclude-table-data" option on Windows.
Date
Msg-id MD2leHq--3-2@tutanota.com
Whole thread Raw
In response to Re: pg_dump seems to be broken in regards to the "--exclude-table-data" option on Windows.  (Alvaro Herrera <alvherre@2ndquadrant.com>)
Responses Re: pg_dump seems to be broken in regards to the "--exclude-table-data" option on Windows.
Re: pg_dump seems to be broken in regards to the "--exclude-table-data" option on Windows.
List pgsql-bugs
Alright. Sigh. I noticed a huge difference in the file sizes between my normal backups and my latest one. I started getting suspicious, so I made a new test WITHOUT using "--exclude-table-data" at all:

I dumped the same database:

1. With setting PGCLIENTENCODING=WIN1252, the weird "workaround".
2. Without setting that. (That is, UTF8.)

The first one is *MUCH* smaller. Opening it up in a visual diff viewer, I can see that HUGE amounts of my data has simply not been copied in the first case. Which is a nightmare scenario and thank God that I noticed this instead of just assuming that it was working now... my backups would've been worthless.

In other words: setting PGCLIENTENCODING=WIN1252 when the database is UTF8 makes pg_dump ignore massive amounts of the data in the database. For this reason, I cannot possibly use this as a "workaround" for my "--exclude-table-data" problem.

Yes, I have very carefully tried with this with "cmd.exe /U" as well as setting the Unicode codepage; it makes *no difference*. Nothing seems to make a difference; pg_dump doesn't seem to *want* to work. It's "all or nothing". I can't exclude any part of the database. The "PGCLIENTENCODING=WIN1252" workaround is sadly insanely dangerous and unusable.

Frankly, at this point I don't have the slightest idea what I could possibly do/try. I have tried so many things by now. If this is truly due to cmd.exe somehow, it must be doing something extraordinarily evil and broken even for Microsoft.

What do you suggest? Should I just pretend as if the --exclude* options don't exist as I've been forced to do many times with a lot of "crucial" features in a lot of software which just won't work correctly? It's just so incredibly inconvenient to have to either store all that temporary junk data or be forced to TRUNCATE it all for that specific table before each backup; what if I'm working right around midnight when the backup is made and I want the last few hours worth of data? I'd have just seconds worth of data instead since I couldn't exclude the table due to "some bug" which I don't even know where it lives: pg_dump or cmd.exe.

Is it entirely unthinkable that this is a pg_dump bug?

pgsql-bugs by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: pg_dump seems to be broken in regards to the "--exclude-table-data" option on Windows.
Next
From: Cy
Date:
Subject: subquery column error causes data destroying equality