More thoughts about FE/BE protocol - Mailing list pgsql-interfaces
From | Tom Lane |
---|---|
Subject | More thoughts about FE/BE protocol |
Date | |
Msg-id | 10478.1049927451@sss.pgh.pa.us Whole thread Raw |
Responses |
Re: [HACKERS] More thoughts about FE/BE protocol
(Steve Crawford <scrawford@pinpointresearch.com>)
Re: [HACKERS] More thoughts about FE/BE protocol (Barry Lind <blind@xythos.com>) |
List | pgsql-interfaces |
I've been thinking some more about the FE/BE protocol redesign, specifically the desire to ensure that we can recover from error conditions without losing synchronization. The fact that the existing protocol doesn't do very well at this shows up in several places:* having to drop and restart the connection after a COPYerror* fastpath function calls also lose sync if there's an error* libpq gets terribly confused if it runs out of memoryfor a query result I'm coming around to the idea that the cleanest solution is to require *all* protocol messages, in both directions, to have an initial length word. That is, the general message format would look like<message type> 1 byte<payload length> number of followingbytes (4 bytes MSB-first)... message data as needed ... The advantage of doing it this way is that the recipient can absorb the whole message before starting to parse the contents; then, errors detected while processing the message contents don't cause us to lose protocol synchronization. Also, even if the message is so large as to run the recipient out of memory, it can still use the <payload length> to count off the number of bytes it has to drop before looking for another message. This would make it quite a bit easier for libpq to cope with out-of-memory, as an example. These advantages seem enough to me to justify adding three or four bytes to the length of each message. Does anyone have a problem with that? A related point is that I had been thinking of the new "extended query" facility (separate PARSE/BIND/DESCRIBE/EXECUTE steps) in terms of sending just one big message to the backend per interaction cycle, with the processing-step commands appearing as fields within that message. But putting a length word in front would effectively require the frontend to marshal the whole sequence before sending any of it. It seems better to send each of the processing-step commands as an independent message. To do that, we need to introduce an additional processing step, call it SYNC, that substitutes for the functionality associated with the overall message boundary in the other way. Specifically: * ReadyForQuery (Z) is sent in response to SYNC. * If no BEGIN has been issued, SYNC is the point at which an implicit COMMIT is done. * After an error occurs in an extended-query command, the backend reads and discards messages until it finds a SYNC, thenissues ReadyForQuery and resumes processing messages. This allows the frontend to be certain what has been processedand what hasn't. Comments? regards, tom lane
pgsql-interfaces by date: