Hi,
On 2026-02-12 22:07:52 +0100, KAZAR Ayoub wrote:
> Currently optimizing COPY FROM using SIMD is still under review, but for
> the case of COPY TO using the same ideas, we found that the problem is
> trivial, the attached patch gives very nice speedups as confirmed by
> Manni's benchmarks.
I have a hard time believing that adding a strlen() to the handling of a short
column won't be a measurable overhead with lots of short attributes.
Particularly because the patch afaict will call it repeatedly if there are any
to-be-escaped characters.
Thanks for pointing that out, so here's what i did:
1) In the previous patch, strlen was called twice if a CSV attribute needed to add a quote, the attached patch gets the length in the beginning and uses it for both SIMD paths, so basically one call.
2) If an attribute needs encoding we need to recalculate string length because it can grow. (so 2 calls at maximum in all cases)
3) Supposing the very worse cases, i benchmarked this against master for tables that have 100, 500, 1000 columns : all integers only, so one would want to process the whole thing in just a pass rather than calculating length of such short attributes:
1000 columns:
TEXT: 17% regression
CSV: 3.4% regression
500 columns:
TEXT: 17.7% regression
CSV: 3.1% regression
100 columns:
TEXT: 17.3% regression
CSV: 3% regression
A bit unstable results, but yeah the overhead for worse cases like this is really significant, I can't argue whether this is worth it or not, so thoughts on this ?
I also don't think it's good how much code this repeats. I think you'd have to
start with preparatory moving the exiting code into static inline helper
functions and then introduce SIMD into those.
Done, yet i'm not too sure whether this is the right place to put it, let me know.
Regards,
Ayoub
