Hi Florents:
On Wed, 6 Jul 2022 at 12:35, Florents Tselai <florents.tselai@gmail.com> wrote:
> 50M+ rows and iirc pg_relation_size was north of 80GB or so.
Somebody already mentioned pg_table_size, but anyway this gives you
1.6Kb per row, which I would suspect is extremely low given your pdf
content and the 1M truncation you preventively applied.
When working with this kind of sizes it really pays to do some good
ole "back of the envelope" calculations and/or some tests with small
batches. Bear in mind if you have, say, 10k per row ( which I would
label as really low for pdf content ) you are going to have 500Gb of
data, if you manage to process this at 100Mb per second you will have
more than an hour ( and I think I'm estimating really low ).
FOS
