Re: zstd compression for pg_dump - Mailing list pgsql-hackers

From Tomas Vondra
Subject Re: zstd compression for pg_dump
Date
Msg-id 29a12485-944c-bc3b-23d0-2f684609b87f@enterprisedb.com
Whole thread Raw
In response to Re: zstd compression for pg_dump  (Kirk Wolak <wolakk@gmail.com>)
List pgsql-hackers
On 3/28/23 20:03, Kirk Wolak wrote:
> On Tue, Mar 28, 2023 at 12:23 PM Tomas Vondra
> <tomas.vondra@enterprisedb.com <mailto:tomas.vondra@enterprisedb.com>>
> wrote:
> 
>     On 3/27/23 19:28, Justin Pryzby wrote:
>     > On Fri, Mar 17, 2023 at 03:43:31AM +0100, Tomas Vondra wrote:
>     >> On 3/16/23 05:50, Justin Pryzby wrote:
>     >>> On Fri, Mar 10, 2023 at 12:48:13PM -0800, Jacob Champion wrote:
>     >>>> On Wed, Mar 8, 2023 at 10:59 AM Jacob Champion
>     <jchampion@timescale.com <mailto:jchampion@timescale.com>> wrote:
>     >>>>> I did some smoke testing against zstd's GitHub release on
>     Windows. To
>     ...
>     OK. I don't have access to a Windows machine so I can't test that. Is it
>     possible to disable the zstd threading, until we figure this out?
> 
> Thomas since I appear to be one of the few windows users (I use both),
> can I help?
> I can test pg_dump... for you, easy to do.  I do about 5-10 pg_dumps a
> day on windows while developing.
> 

Perhaps. But I'll leave the details up to Justin - it's his patch, and
I'm not sure how to verify the threading is OK.

I'd try applying this patch, build with --with-zstd and then run the
pg_dump TAP tests, and perhaps do some manual tests.

And perhaps do the same for --with-lz4 - there's a thread [1] suggesting
we don't detect lz4 stuff on Windows, so the TAP tests do nothing.

https://www.postgresql.org/message-id/ZAjL96N9ZW84U59p@msg.df7cb.de

> Also, I have an AWS instance I created to build PG/Win with readline
> back in November.
> I could give you access to that...  (you are not the only person who has
> made this statement here).
> I've made such instances available for other Open Source developers, to
> support them.
> 
>  Obvi I would share connection credentials privately.
> 

I'd rather leave the Windows stuff up to someone with more experience
with that platform. I have plenty of other stuff on my plate atm.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



pgsql-hackers by date:

Previous
From: "Gregory Stark (as CFM)"
Date:
Subject: Re: POC: Better infrastructure for automated testing of concurrency issues
Next
From: Justin Pryzby
Date:
Subject: Re: Add LZ4 compression in pg_dump