Home > mailing lists

Re: Speed up COPY FROM text/CSV parsing using SIMD - Mailing list pgsql-hackers

From	Manni Wood
Subject	Re: Speed up COPY FROM text/CSV parsing using SIMD
Date	March 7 00:25:46
Msg-id	CAKWEB6rnMKYGSt=t9=pL2kKUefsAdzKfjtwdqW_acv1+vMTKVA@mail.gmail.com Whole thread
In response to	Re: Speed up COPY FROM text/CSV parsing using SIMD (Nazir Bilal Yavuz <byavuz81@gmail.com>)
Responses	Re: Speed up COPY FROM text/CSV parsing using SIMD
List	pgsql-hackers

Tree view

On Fri, Mar 6, 2026 at 12:55 PM Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:

Hi Manni!

On Fri, 6 Mar 2026 at 21:13, Manni Wood <manni.wood@enterprisedb.com> wrote:
>
> When I ran `meson setup build --buildtype=debugoptimized` on both my x86 machine and my arm machine, the response on both was:
>
> "External libraries
> " lz4 : NO"
>
> However, I did not remove commit 34dfca2934 from any of my Postgres builds; I left that commit in place.
>
> Let me know if that helps!

That definitely helps, thanks! If you have a chance, could you also
run the benchmark with LZ4 enabled? I expect you may see significantly
better performance, similar to what I observed.

--
Regards,
Nazir Bilal Yavuz
Microsoft

Hi, Nazir.

Well, golly! Look at these numbers. Old master with no lz4, your v11 patch with no lz4, and then your v11 patch with lz4 compiled in.

(Aside: I assume everything is good for my lz4 build after installing the lz4 dev library and seeing this with my meson config:

" External libraries

" lz4 : YES 1.9.4"

and this from the db itself:

$ ./bin/psql -U mwood -d postgres -c 'show default_toast_compression'
default_toast_compression
---------------------------
lz4

)

x86 NARROW old master
TXT : 25909.060500 ms
CSV : 28137.591250 ms
TXT with 1/3 escapes: 27794.177000 ms
CSV with 1/3 quotes: 34541.704750 ms

x86 NARROW v11
TXT : 27212.945750 ms -5.032545% regression
CSV : 26985.971250 ms 4.092817% improvement
TXT with 1/3 escapes: 27216.510000 ms 2.078374% improvement
CSV with 1/3 quotes: 32817.267500 ms 4.992334% improvement

x86 NARROW v11 lz4
TXT : 26471.776500 ms -2.171889% regression
CSV : 25607.026250 ms 8.993538% improvement
TXT with 1/3 escapes: 28628.729750 ms -3.002617% regression
CSV with 1/3 quotes: 34729.006750 ms -0.542249% regression

x86 WIDE old master
TXT : 28778.426500 ms
CSV : 35671.908000 ms
TXT with 1/3 escapes: 32441.549750 ms
CSV with 1/3 quotes: 47024.416000 ms

x86 WIDE v11
TXT : 22571.305750 ms 21.568659% improvement
CSV : 22711.524750 ms 36.332184% improvement
TXT with 1/3 escapes: 29236.453000 ms 9.879604% improvement
CSV with 1/3 quotes: 40022.110750 ms 14.890786% improvement

x86 WIDE v11 lz4
TXT : 8032.912750 ms 72.087033% improvement
CSV : 8047.098000 ms 77.441358% improvement
TXT with 1/3 escapes: 15428.139500 ms 52.443272% improvement
CSV with 1/3 quotes: 27517.084500 ms 41.483410% improvement

arm NARROW old master
TXT : 10997.568250 ms
CSV : 10797.549000 ms
TXT with 1/3 escapes: 10299.047000 ms
CSV with 1/3 quotes: 12559.385750 ms

arm NARROW v11
TXT : 10340.966250 ms 5.970429% improvement
CSV : 10224.399500 ms 5.308144% improvement
TXT with 1/3 escapes: 10438.216750 ms -1.351288% regression
CSV with 1/3 quotes: 11865.934000 ms 5.521383% improvement

arm NARROW v11 lz4
TXT : 9783.737000 ms 11.037270% improvement
CSV : 10122.890750 ms 6.248254% improvement
TXT with 1/3 escapes: 10298.780250 ms 0.002590% improvement
CSV with 1/3 quotes: 11738.992250 ms 6.532115% improvement

arm WIDE old master
TXT : 11825.771250 ms
CSV : 13907.074000 ms
TXT with 1/3 escapes: 13430.691250 ms
CSV with 1/3 quotes: 17557.954500 ms

arm WIDE v11
TXT : 9001.442250 ms 23.882831% improvement
CSV : 8940.928750 ms 35.709490% improvement
TXT with 1/3 escapes: 12049.668500 ms 10.282589% improvement
CSV with 1/3 quotes: 15277.843250 ms 12.986201% improvement

arm WIDE v11 lz4
TXT : 3186.825500 ms 73.051859% improvement
CSV : 3142.526500 ms 77.403396% improvement
TXT with 1/3 escapes: 6180.176000 ms 53.984677% improvement
CSV with 1/3 quotes: 9460.505500 ms 46.118407% improvement

Cheers,

-Manni

-- Manni Wood EDB: https://www.enterprisedb.com

pgsql-hackers by date:

From: Robert Haas
Date: 06 March, 23:50:56
Subject: Re: Bypassing cursors in postgres_fdw to enable parallel plans

From: Daniel Gustafsson
Date: 07 March, 01:11:49
Subject: Re: Serverside SNI support in libpq

Re: Speed up COPY FROM text/CSV parsing using SIMD - Mailing list pgsql-hackers

Previous

Next