Thread: Opening stdin/stdout in binary mode on Windows
I wrote:
>> pg_restore: [archiver] unsupported version (1.13) in file header
> Come to think of it, I'll bet that you cannot use "> DBFile" on Windows
> because it ends up opening the archive file in text instead of binary
> mode. SetOutput() in pg_backup_archiver.c tries to work around this by
> doing
> fn = fileno(stdout);
> AH->OF = fdopen(dup(fn), PG_BINARY_W);
> but it wouldn't surprise me in the least to learn that that doesn't work
> on Windows.
We now have confirmation that indeed this is the source of the pg_dump
misbehavior. Can anyone suggest a way that does work to do binary I/O
on stdout? What about reading from stdin?
If we can't fix this it will require some serious uglification of the
pg_dump/pg_restore docs for Windows ...
regards, tom lane
I wrote:
> Can anyone suggest a way that does work to do binary I/O
> on stdout? What about reading from stdin?
After a bit of googling I came up with the attached patch. I am not in
a position to test it, however. Can anyone try it out?
regards, tom lane
*** src/bin/pg_dump/pg_backup_archiver.c~ Tue Jan 25 17:44:31 2005
--- src/bin/pg_dump/pg_backup_archiver.c Wed Jan 26 13:34:33 2005
***************
*** 29,34 ****
--- 29,38 ----
#include <ctype.h>
#include <unistd.h>
+ #ifdef WIN32
+ #include <io.h>
+ #endif
+
#include "pqexpbuffer.h"
#include "libpq/libpq-fs.h"
***************
*** 1709,1714 ****
--- 1713,1731 ----
AH->gzOut = 0;
AH->OF = stdout;
+ /*
+ * On Windows, we need to use binary mode to read/write non-text archive
+ * formats. Force stdin/stdout into binary mode in case that is what
+ * we are using.
+ */
+ #ifdef WIN32
+ if (fmt != archNull)
+ {
+ setmode(fileno(stdout), O_BINARY);
+ setmode(fileno(stdin), O_BINARY);
+ }
+ #endif
+
#if 0
write_msg(modulename, "archive format is %d\n", fmt);
#endif
Tgl wrote: > I wrote: > > Can anyone suggest a way that does work to do binary I/O > > on stdout? What about reading from stdin? > > After a bit of googling I came up with the attached patch. I am not in > a position to test it, however. Can anyone try it out? I'll test...is there a quick way to reproduce the error? Merlin
"Merlin Moncure" <merlin.moncure@rcsonline.com> writes:
> I'll test...is there a quick way to reproduce the error?
Try
pg_dump -Fc mydb >mydb.dump
pg_restore -l mydb.dump
and see if pg_restore complains. I'd suggest checking
all four combinations of writing to stdout or a named file and
having pg_restore read from stdin or a named file.
regards, tom lane
Oh, one other thing: the patch is intended to still allow plain text
dumps to be written in text mode ... you might want to check that that
still works as intended.
regards, tom lane
> "Merlin Moncure" <merlin.moncure@rcsonline.com> writes: > > I'll test...is there a quick way to reproduce the error? > > Try > pg_dump -Fc mydb >mydb.dump > pg_restore -l mydb.dump > and see if pg_restore complains. I'd suggest checking > all four combinations of writing to stdout or a named file and > having pg_restore read from stdin or a named file. We are good to go. The old version of pg_dump gave numerous ftell errors and wouldn't restore and the new one gives no errors (and restore works properly). Also, text version runs fine (and should, by the code). Merlin
"Merlin Moncure" <merlin.moncure@rcsonline.com> writes:
> We are good to go. The old version of pg_dump gave numerous ftell errors
> and wouldn't restore and the new one gives no errors (and restore works
> properly).
Great. I applied the patch.
regards, tom lane
Well, that is a nifty trick. Great to have that solved with no
documentation changes required.
Without this fix we were going to have tons of headaches.
---------------------------------------------------------------------------
Tom Lane wrote:
> I wrote:
> > Can anyone suggest a way that does work to do binary I/O
> > on stdout? What about reading from stdin?
>
> After a bit of googling I came up with the attached patch. I am not in
> a position to test it, however. Can anyone try it out?
>
> regards, tom lane
>
> *** src/bin/pg_dump/pg_backup_archiver.c~ Tue Jan 25 17:44:31 2005
> --- src/bin/pg_dump/pg_backup_archiver.c Wed Jan 26 13:34:33 2005
> ***************
> *** 29,34 ****
> --- 29,38 ----
> #include <ctype.h>
> #include <unistd.h>
>
> + #ifdef WIN32
> + #include <io.h>
> + #endif
> +
> #include "pqexpbuffer.h"
> #include "libpq/libpq-fs.h"
>
> ***************
> *** 1709,1714 ****
> --- 1713,1731 ----
> AH->gzOut = 0;
> AH->OF = stdout;
>
> + /*
> + * On Windows, we need to use binary mode to read/write non-text archive
> + * formats. Force stdin/stdout into binary mode in case that is what
> + * we are using.
> + */
> + #ifdef WIN32
> + if (fmt != archNull)
> + {
> + setmode(fileno(stdout), O_BINARY);
> + setmode(fileno(stdin), O_BINARY);
> + }
> + #endif
> +
> #if 0
> write_msg(modulename, "archive format is %d\n", fmt);
> #endif
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postgresql.org so that your
> message can get through to the mailing list cleanly
>
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073
Bruce wrote: > Well, that is a nifty trick. Great to have that solved with no > documentation changes required. > > Without this fix we were going to have tons of headaches. This is pretty standard m.o. in porting unix applications to win32, or even in console mode win32 apps that need to support binary redirection. I almost suggested this (setmode), but stupidly assumed it must have been attempted and discarded for reasons unknown at an earlier time :-). Merlin
Merlin Moncure wrote: > Bruce wrote: > > Well, that is a nifty trick. Great to have that solved with no > > documentation changes required. > > > > Without this fix we were going to have tons of headaches. > > This is pretty standard m.o. in porting unix applications to win32, or > even in console mode win32 apps that need to support binary redirection. > I almost suggested this (setmode), but stupidly assumed it must have > been attempted and discarded for reasons unknown at an earlier time :-). I had never heard of it before. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073