Re: libpq compression - Mailing list pgsql-hackers

From Matthias van de Meent
Subject Re: libpq compression
Date
Msg-id CAEze2WgFEOVwQZap+ZQFuA7A_uLDSAQRSNm3EqCVgrpeM=TGNQ@mail.gmail.com
Whole thread Raw
In response to Re: libpq compression  (Konstantin Knizhnik <k.knizhnik@postgrespro.ru>)
Responses Re: libpq compression  (Konstantin Knizhnik <k.knizhnik@postgrespro.ru>)
List pgsql-hackers
On Mon, 2 Nov 2020 at 20:20, Konstantin Knizhnik
<k.knizhnik@postgrespro.ru> wrote:
>
> On 02.11.2020 19:53, Matthias van de Meent wrote:
> > This is the result of network traffic of two backends one with enabled
> >> compression and another with disable compression
> >> after execution of "select * from pg_class" command:
> >>
> >> select * from pg_stat_network_traffic;
> >>     pid  | rx_raw_bytes | tx_raw_bytes | rx_compressed_bytes |
> >> tx_compressed_bytes
> >> -------+--------------+--------------+---------------------+---------------------
> >>    22276 |           29 |        86327 |                  38
> >> |               10656
> >>    22282 |           73 |        86327 |                   0
> >> |                   0
> > The current names and values of these columns are confusing me:
> > What column contains the amount of bytes sent to/received from the
> > client? Is the compression method of pid 22282 extremely efficient at
> > compressing, or does it void the data (compresses down to 0 bytes)?
> Names of the columns can be changed if you or somebody else will propose
> better alternatives.

How about Xx_logical_bytes for raw the pg command stream data, and
keeping Xx_compressed_bytes for the compressed data in/out?

> This view pg_stat_network_traffic reports traffic from server (backend)
> point of view, i.e.
> rx_bytes (received bytes) are commands sent from client to the server
> tx_bytes (transmitted bytes) are responses sent by server to the client.
>
> If compression is not used then rx_compressed_bytes =
> tx_compressed_bytes = 0
> It seems to be more natural then assigning them the same values as (raw
> bytes).
> Because it can really happen that for BLOBs with already compressed data
> (video images or sound)
> compressed data will be almost the same as raw data even if compression
> is enabled.
> So it seems to be important to distinguished situations when data can
> not be compressed and
> when it is not compressed at all.

Looking at it from that viewpoint, I agree. My primary reason for
suggesting this was that it would be useful to expose how much data
was transferred between the client and the server, which cannot be
constructed from that view for compression-enabled connections. That
is because the compression methods' counting only starts after some
bytes have already been transferred, and the raw/logical counter
starts deviating once compression is enabled.



pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: Move OpenSSL random under USE_OPENSSL_RANDOM
Next
From: Amit Langote
Date:
Subject: Re: ModifyTable overheads in generic plans