Re: About large objects asynchronous and non-blocking support - Mailing list pgsql-hackers

From Tatsuo Ishii
Subject Re: About large objects asynchronous and non-blocking support
Date
Msg-id 20130608.084609.1665307841244173409.t-ishii@sraoss.co.jp
Whole thread Raw
In response to Re: About large objects asynchronous and non-blocking support  (Dmitriy Igrishin <dmitigr@gmail.com>)
List pgsql-hackers
> 2013/6/6 Tatsuo Ishii <ishii@postgresql.org>
> 
>> > Hi.
>> >
>> > At the moment libpq doesn't seem to support asynchronous and
>> > non-blocking support for large objects, in the style of
>> > PQsendQuery/PQgetResult. This makes large objects hardly suited for
>> > single-threaded programs based on some variant of select().
>> >
>> > I would like to know whether this is a deliberate decision or it is
>> > considered a bug, and, in case, whether it is scheduled to be fixed.
>>
>> Certainly not bug, since the doc clearly stats that PQsendQuery can
>> only be used as a substituation of PQexec.  (see "Asynchronous Command
>> Processing" section" for more details). The large object API is
>> completely different from PQexec and its friends, so it cannot be used
>> with PQsendQuery.
>>
>> Talking about more details, PQexec and PQsendQuery is designed to
>> handle only "Q" messsage out of PostgreSQL frontend/backend protocol,
>> while to access large objects, you need to handle "V" message.
>>
> Really? I've specialized a C++ standard std::streambuf class by using
> only extended query protocol (by using prepared statements via
> PQsendPrepare,
> PQsendQueryPrepared) to call SQL functions like loread(), lowrite(),
> lo_tell(), etc.
> All these functions just needs to be called inside BEGIN block. And yes,
> it can be done asynchronously.

Thanks for reminding me. I totally forgot about them.

>> > Though I cannot guarantee anything, I may be interested into working out
>> > a patch, if no one is already doing the same (of course I understand
>> > that this patch wouldn't be for 9.3, which is already in its late
>> > release cycle).
>> >
>> > Do you think this may be of interest?
>>
>> Yes, I understand your pain, and I myself think we need new APIs for
>> large objects. Probably that would be not terribly hard. One idea
>> would be inventing an asynchronous version of PQfn and let
>> lo_read/lo_write allow to use the new API.
>>
> Yes, but according to
> http://www.postgresql.org/docs/9.2/static/protocol-flow.html#AEN95330
> and/or http://www.postgresql.org/docs/9.2/static/libpq-fastpath.html
> function call sub-protocol is obsolete. Thats why personally I decided to
> use prepared statements.

I'm not totally pleased with the comment in the doc.  For me the only
reason why those extended protocol functions are recommended is the
binary protocol can be used. The price is parsing, planning, and
preparing the query, all of them are essentially unnecessary for a
large object access use case.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp



pgsql-hackers by date:

Previous
From: Noah Misch
Date:
Subject: Re: ALTER DEFAULT PRIVILEGES FOR ROLE is broken
Next
From: Josh Berkus
Date:
Subject: Last log line for log_temp_files is disassociated with query