Re: Warn when parallel restoring a custom dump without data offsets - Mailing list pgsql-hackers

From Justin Pryzby
Subject Re: Warn when parallel restoring a custom dump without data offsets
Date
Msg-id CAOaQA5xsa7KTsX3js5mt00Do+AiarQb5qd4ZYM3AR_NwXtFzeg@mail.gmail.com
Whole thread Raw
In response to Warn when parallel restoring a custom dump without data offsets  (David Gilman <davidgilman1@gmail.com>)
Responses Re: Warn when parallel restoring a custom dump without data offsets
List pgsql-hackers
I started fooling with this at home while our ISP is broke (pardon my brevity).

Maybe you also saw commit b779ea8a9a2dc3a089b3ac152b1ec4568bfeb26f
"Fix pg_restore so parallel restore doesn't fail when the input file
doesn't contain data offsets (which it won't, if pg_dump thought its
output wasn't seekable)..."

...which I guess should actually say "doesn't NECESSARILY fail", since
it also adds this comment:
"This could fail if we are asked to restore items out-of-order."

So this is a known issue and not a regression.  I think the PG11
commit you mentioned (548e5097) happens to make some databases fail in
parallel restore that previously worked (I didn't check).  Possibly
also some databases (or some pre-existing dumps) which used to fail
might possibly now succeed.

Your patch adds a warning if unseekable output might fail during
parallel restore.  I'm not opposed to that, but can we just make
pg_restore work in that case?  If the input is unseekable, then we can
never do a parallel restore at all.  If it *is* seekable, could we
make _PrintTocData rewind if it gets to EOF using ftello(SEEK_SET, 0)
and re-scan again from the beginning?  Would you want to try that ?



pgsql-hackers by date:

Previous
From: Thomas Munro
Date:
Subject: Re: Parallel Seq Scan vs kernel read ahead
Next
From: Jeff Davis
Date:
Subject: Re: Trouble with hashagg spill I/O pattern and costing