Re: walprotocol.h vs frontends - Mailing list pgsql-hackers

From Simon Riggs
Subject Re: walprotocol.h vs frontends
Date
Msg-id CA+U5nM+mh4Frv3pjFzw2kamU3KUtWkgbbTGUwSA7HRQ74=Px2Q@mail.gmail.com
Whole thread Raw
In response to Re: walprotocol.h vs frontends  (Magnus Hagander <magnus@hagander.net>)
Responses Re: walprotocol.h vs frontends
List pgsql-hackers
On Mon, Aug 15, 2011 at 10:32 PM, Magnus Hagander <magnus@hagander.net> wrote:

>> At present the WALSender only sends from one file at a time, so
>> sending a message when we open a new file would be straightforward.
>
> Are you sure? We can receive a single message spanning multiple files...

You're right. That was the way the original code ran but I thought we
had stopped that when we introduced MAX_SEND_SIZE. The comment says
"don't cross a logfile boundary within one message". What that
actually does is prevent us incrementing a logid value, which happens
every 255 files. I read that as meaning "WAL file" which is not what
it means at all.

So right now what we do is allow a single packet to span multiple
files, but since MAX_SEND_SIZE is 128KB it will always be smaller than
a single file, so we can only ever span two files at most.

That is all just a little bizarre, especially since libpq sends data
in 8KB chunks anyway.

So I suggest we change XLogSend() so that it only ever sends up to the
end of a file. That way all "w" messages will relate to just one file,
and we can have another message to initiate a new file. And then, as
you say, give full metadata for the new file.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services


pgsql-hackers by date:

Previous
From: Jim Nasby
Date:
Subject: Re: index-only scans
Next
From: Jim Nasby
Date:
Subject: Re: synchronized snapshots