Hi Andres
On Fri, 13 Dec 2024 at 08:38, Andres Freund <andres@anarazel.de> wrote:
> > Another interesting snippet: the application has a number of ETL
> > workers going at once. The actual number varies depending on a number
> > of factors but might be somewhere from 10 - 150. Each worker will have
> > a single postgres backend that they are feeding data to.
>
> Are they all inserting into distinct tables/partitions or into shared tables?
The set of tables they are writing into is the same, but we do take
some effort to randomize the order of the tables that we each worker
is writing into so as to reduce contention. Even so it is quite likely
that multiple processes will be writing into a table at a time.
Also worth noting that I have only seen this error triggered by COPY
statements (other than the upgrade case). There are some other cases
in our code that use INSERT but so far I have not seen that end in an
out of space error.
> When you say that they're not "all striking it at once", do you mean that some
> of them aren't interacting with the database at the time, or that they're not
> erroring out?
Sorry, I meant erroring out.
Thanks for the patch!
Cheers
Mike
