RE: libpq compression - Mailing list pgsql-hackers

From Iwata, Aya
Subject RE: libpq compression
Date
Msg-id 71E660EB361DF14299875B198D4CE5423DED67E5@g01jpexmbkw25
Whole thread Raw
In response to Re: libpq compression  (Konstantin Knizhnik <k.knizhnik@postgrespro.ru>)
Responses Re: libpq compression
Re: libpq compression
List pgsql-hackers
Hi, 

I am sorry for my late reply.

> I fixed all issues you have reported except using list of supported compression
> algorithms.
Sure. I confirmed that.

> It will require extra round of communication between client and server to
> make a decision about used compression algorithm.
In beginning of this thread, Robbie Harwood said  that no extra communication needed.
I think so, too.

> I still not sure whether it is good idea to make it possible to user to
> explicitly specify compression algorithm.
> Right now used streaming compression algorithm is hardcoded and depends on
> --use-zstd ort --use-zlib configuration options.
> If client and server were built with the same options, then they are able
> to use compression.
I understand that compression algorithm is hardcoded in your proposal.
However given the possibility of future implementation, I think 
it would be better for it to have a flexibility to choose compression library.

src/backend/libpq/pqcomm.c :
In current Postgres source code, pq_recvbuf() calls secure_read() 
and pq_getbyte_if_available() also calls secure_read().
It means these functions are on the same level. 
However in your change, pq_getbyte_if_available() calls pq_recvbuf(), 
and  pq_recvbuf() calls secure_read(). The level of these functions is different.

I think the purpose of pq_getbyte_if_available() is to get a character if it exists and 
the purpose of pq_recvbuf() is to acquire data up to the expected length.
In your change, pq_getbyte_if_available() may have to do unnecessary process waiting or something.

So how about changing your code like this?
The part that checks whether it is compressed is implemented as a #define macro(like fe_misc.c). And pq_recvbuf() and
pq_getbyte_if_available()modify little, like this;
 

-        r = secure_read(MyProcPort, PqRecvBuffer + PqRecvLength,
-                        PQ_RECV_BUFFER_SIZE - PqRecvLength);    
+    r = SOME_DEFINE_NAME_();

configure:
Adding following message to the top of zlib in configure
```
{$as_echo "$as_me:${as_lineno-$LINENO}:checking whethere to build with zstd support">&5
$as_echo_n "checking whether to build with zstd suppor... ">&6;}
```

Regards,
Aya Iwata

pgsql-hackers by date:

Previous
From: Amit Langote
Date:
Subject: Re: speeding up planning with partitions
Next
From: Andres Freund
Date:
Subject: Re: libpq compression