Re: PATCH: Batch/pipelining support for libpq - Mailing list pgsql-hackers

From Shay Rojansky
Subject Re: PATCH: Batch/pipelining support for libpq
Date
Msg-id CADT4RqBN-WkdgDLX3hAgsfGdJpqGv8BaZQknX7dyjmTT6jHgmg@mail.gmail.com
Whole thread Raw
In response to Re: PATCH: Batch/pipelining support for libpq  (Craig Ringer <craig@2ndquadrant.com>)
Responses Re: PATCH: Batch/pipelining support for libpq  (Craig Ringer <craig@2ndquadrant.com>)
List pgsql-hackers
>  Of course, this is a
> relatively minor performance issue (especially when compared to the overall
> performance benefits provided by batching), and providing an API distinction
> between adding a Sync and flushing the buffer may over-complicate the API. I
> just thought I'd mention it.

Unnecessary IMO. If we really want to add it later we'd probably do so
by setting a "no flush on endbatch" mode and adding an explicit flush
call. But I expect TCP_CORK will satisfy all realistic needs.

Unless I'm mistaken TCP_CORK is not necessarily going to work across all platforms (e.g. Windows), although SO_LINGER (which is more standard) also helps here.

> When it's about to send a batch, Npgsql knows whether it's in an (explicit)
> transaction or not (by examining the transaction indicator on the last
> ReadyForQuery message it received). If it's not in an (explicit)
> transaction, it automatically inserts a Sync message after every Execute. If
> some statement happens to be a BEGIN, it will be executed as a normal
> statement and so on. The only issue is that if an error occurs after a
> sneaked-in BEGIN, all subsequent statements will fail, and all have the Sync
> messages Npgsql inserted. The end result will be a series of errors that
> will be raised up to the user, but this isn't fundamentally different from
> the case of a simple auto-commit batch containing multiple failures (because
> of unique constraint violation or whatever) - multiple errors is something
> that will have to be handled in any case.

I'm a bit hesitant about how this will interact with multi-statements
containing embedded BEGIN or COMMIT, where a single protocol message
contains multiple ; delimited queries. But at least at this time of
night I can't give a concrete problematic example.

Unless I'm mistaken, in the extended protocol you can't combine multiple ; delimited queries in a single Parse - that's a feature supported only by the Query message of the simple protocol. But then, if you're in the simple protocol Sync doesn't play any role, does it? I mean, a Query message behaves as though it's implicitly followed by a Sync message - an error in a previous Query message doesn't cause later messages to be skipped...

Note that Npgsql only rarely uses the simple protocol for user messages. Npgsql is a binary-only driver, and the simple protocol doesn't support requesting binary results. So only the extended protocol is used, except for some edge cases where it's possible to use the simple protocol for efficiency - statements with no parameters and where the ExecuteNonQuery API is used (i.e. the user won't access any results).

pgsql-hackers by date:

Previous
From: Albe Laurenz
Date:
Subject: Re: Add PGDLLEXPORT to PG_FUNCTION_INFO_V1
Next
From: Craig Ringer
Date:
Subject: Re: PATCH: Batch/pipelining support for libpq