Re: Speed up COPY FROM text/CSV parsing using SIMD - Mailing list pgsql-hackers
| From | Manni Wood |
|---|---|
| Subject | Re: Speed up COPY FROM text/CSV parsing using SIMD |
| Date | |
| Msg-id | CAKWEB6oT5KbyF+uRRhjjJi7p2PmRdOzxp3T6vFcN04BCR-=B2w@mail.gmail.com Whole thread |
| In response to | Re: Speed up COPY FROM text/CSV parsing using SIMD (Nathan Bossart <nathandbossart@gmail.com>) |
| List | pgsql-hackers |
On Tue, Feb 24, 2026 at 11:48 AM Nathan Bossart <nathandbossart@gmail.com> wrote:
On Tue, Feb 24, 2026 at 04:57:21PM +0300, Nazir Bilal Yavuz wrote:
> I will investigate this. However, please note that the current master
> includes the inlining commit (dc592a4155), which makes the COPY FROM
> faster. In my case,
>
> 1: current master without dc592a4155: 14400ms
> 2: current master: 13960ms (%3 improvement against #1)
> 3: current master + SIMD: 15123ms (%5 regression against #1 and %8
> regression against #2)
>
> Is it possible for you to do a similar test? I mean dropping
> dc592a4155 from the current master and re-running the benchmark, that
> would be helpful.
IMHO as long as the difference from v18 looks reasonable, commit-by-commit
regressions and improvements that even out in the end are okay. That's
perhaps a bit of mental gymnastics (e.g., what if we had committed the
inlining patch for v18?), but I believe that's how we've dealt with similar
problems in the past. But maybe there are ways to avoid even these
in-development regressions, too...
--
nathan
Oh yes, I see now.
Commit 18bcdb75 is just before the v9 patch got applied, so I used that as "old master" and compared that with master (v9 applied) and then "master (v9 applied) + v10 applied".
arm NARROW old master 18bcdb75
TXT : 10997.568250 ms
CSV : 10797.549000 ms
TXT with 1/3 escapes: 10299.047000 ms
CSV with 1/3 quotes: 12559.385750 ms
arm NARROW master (v9 applied)
TXT : 10077.096250 ms 8.369778% improvement
CSV : 10310.671250 ms 4.509151% improvement
TXT with 1/3 escapes: 9893.155000 ms 3.941064% improvement
CSV with 1/3 quotes: 12133.064750 ms 3.394441% improvement
arm NARROW v10
TXT : 10467.816750 ms 4.816988% improvement
CSV : 9986.288000 ms 7.513381% improvement
TXT with 1/3 escapes: 10323.173750 ms -0.234262% regression
CSV with 1/3 quotes: 11843.611750 ms 5.699116% improvement
arm WIDE old master 18bcdb75
TXT : 11825.771250 ms
CSV : 13907.074000 ms
TXT with 1/3 escapes: 13430.691250 ms
CSV with 1/3 quotes: 17557.954500 ms
arm WIDE master (v9 applied)
TXT : 10568.344750 ms 10.632934% improvement
CSV : 13046.610500 ms 6.187236% improvement
TXT with 1/3 escapes: 12193.088500 ms 9.214736% improvement
CSV with 1/3 quotes: 16629.319000 ms 5.288973% improvement
arm WIDE v10
TXT : 9064.959000 ms 23.345727% improvement
CSV : 9019.553250 ms 35.144134% improvement
TXT with 1/3 escapes: 12344.497250 ms 8.087402% improvement
CSV with 1/3 quotes: 15495.863750 ms 11.744482% improvement
x86 NARROW old master 18bcdb75
TXT : 25909.060500 ms
CSV : 28137.591250 ms
TXT with 1/3 escapes: 27794.177000 ms
CSV with 1/3 quotes: 34541.704750 ms
x86 NARROW master
TXT : 26359.319000 ms -1.737842% regression
CSV : 25661.199750 ms 8.801007% improvement
TXT with 1/3 escapes: 28170.085250 ms -1.352471% regression
CSV with 1/3 quotes: 32638.147500 ms 5.510895% improvement
x86 NARROW v10
TXT : 26416.331500 ms -1.957890% regression
CSV : 25318.727500 ms 10.018142% improvement
TXT with 1/3 escapes: 28608.007500 ms -2.928061% regression
CSV with 1/3 quotes: 32805.627750 ms 5.026032% improvement
x86 WIDE old master 18bcdb75
TXT : 28778.426500 ms
CSV : 35671.908000 ms
TXT with 1/3 escapes: 32441.549750 ms
CSV with 1/3 quotes: 47024.416000 ms
x86 WIDE master
TXT : 26475.164250 ms 8.003434% improvement
CSV : 31963.478500 ms 10.395938% improvement
TXT with 1/3 escapes: 29671.120750 ms 8.539755% improvement
CSV with 1/3 quotes: 40391.616250 ms 14.105012% improvement
x86 WIDE v10
TXT : 23067.046750 ms 19.846046% improvement
CSV : 23259.092250 ms 34.797174% improvement
TXT with 1/3 escapes: 31796.098250 ms 1.989583% improvement
CSV with 1/3 quotes: 42925.792250 ms 8.715948% improvement
TXT : 10997.568250 ms
CSV : 10797.549000 ms
TXT with 1/3 escapes: 10299.047000 ms
CSV with 1/3 quotes: 12559.385750 ms
arm NARROW master (v9 applied)
TXT : 10077.096250 ms 8.369778% improvement
CSV : 10310.671250 ms 4.509151% improvement
TXT with 1/3 escapes: 9893.155000 ms 3.941064% improvement
CSV with 1/3 quotes: 12133.064750 ms 3.394441% improvement
arm NARROW v10
TXT : 10467.816750 ms 4.816988% improvement
CSV : 9986.288000 ms 7.513381% improvement
TXT with 1/3 escapes: 10323.173750 ms -0.234262% regression
CSV with 1/3 quotes: 11843.611750 ms 5.699116% improvement
arm WIDE old master 18bcdb75
TXT : 11825.771250 ms
CSV : 13907.074000 ms
TXT with 1/3 escapes: 13430.691250 ms
CSV with 1/3 quotes: 17557.954500 ms
arm WIDE master (v9 applied)
TXT : 10568.344750 ms 10.632934% improvement
CSV : 13046.610500 ms 6.187236% improvement
TXT with 1/3 escapes: 12193.088500 ms 9.214736% improvement
CSV with 1/3 quotes: 16629.319000 ms 5.288973% improvement
arm WIDE v10
TXT : 9064.959000 ms 23.345727% improvement
CSV : 9019.553250 ms 35.144134% improvement
TXT with 1/3 escapes: 12344.497250 ms 8.087402% improvement
CSV with 1/3 quotes: 15495.863750 ms 11.744482% improvement
x86 NARROW old master 18bcdb75
TXT : 25909.060500 ms
CSV : 28137.591250 ms
TXT with 1/3 escapes: 27794.177000 ms
CSV with 1/3 quotes: 34541.704750 ms
x86 NARROW master
TXT : 26359.319000 ms -1.737842% regression
CSV : 25661.199750 ms 8.801007% improvement
TXT with 1/3 escapes: 28170.085250 ms -1.352471% regression
CSV with 1/3 quotes: 32638.147500 ms 5.510895% improvement
x86 NARROW v10
TXT : 26416.331500 ms -1.957890% regression
CSV : 25318.727500 ms 10.018142% improvement
TXT with 1/3 escapes: 28608.007500 ms -2.928061% regression
CSV with 1/3 quotes: 32805.627750 ms 5.026032% improvement
x86 WIDE old master 18bcdb75
TXT : 28778.426500 ms
CSV : 35671.908000 ms
TXT with 1/3 escapes: 32441.549750 ms
CSV with 1/3 quotes: 47024.416000 ms
x86 WIDE master
TXT : 26475.164250 ms 8.003434% improvement
CSV : 31963.478500 ms 10.395938% improvement
TXT with 1/3 escapes: 29671.120750 ms 8.539755% improvement
CSV with 1/3 quotes: 40391.616250 ms 14.105012% improvement
x86 WIDE v10
TXT : 23067.046750 ms 19.846046% improvement
CSV : 23259.092250 ms 34.797174% improvement
TXT with 1/3 escapes: 31796.098250 ms 1.989583% improvement
CSV with 1/3 quotes: 42925.792250 ms 8.715948% improvement
-- Manni Wood EDB: https://www.enterprisedb.com
pgsql-hackers by date: