Hi,
On 2022-07-11 05:31:35 -0700, David G. Johnston wrote:
> On Saturday, July 9, 2022, PG Bug reporting form <noreply@postgresql.org>
> wrote:
>
> > The following bug has been logged on the website:
> >
> > Bug reference: 17543
> > Logged by: Nathan Ballance
> > Email address: nathanfballance@gmail.com
> > PostgreSQL version: 14.4
> > Operating system: Linux/UNIX
> > Description:
> >
> > Postgresql server with csvlog log_destination enabled will have malformed
> > CSV upon a disk space error. This causes any loading of the malformed *.csv
> > log file to error
> >
> >
> > ASK: Can the CSV file be written to in a safer way which ensures proper
> > format even upon disk error?
> >
> >
> This list is for reporting bugs, not feature requests.
>
> I’d have to say that there is little interest in sacrificing performance
> for safety here, which seems like an unavoidable proposition.
I agree in general, but this specific issue seems easy enough to address. We
could just track whether the last write failed, and if so, emit an additional
newline.
But that just fixes the simple case - if the last successful write contained
the start of an escaped string, the newline won't necessarily be recognized as
the end of a "row".
We can't generally solve this with reasonable cost, because if e.g. postgres
runs out of space and then is restarted, we'll not know how to "break out" of
whatever state we were in. We'd have to read the file to figure that out,
obviously not a realistic proposition.
It might be possible to devise a scheme in which we force rotation in cases
where corruption might have occurred?
Greetings,
Andres Freund
