As long as sorting improvements are being considered, may I suggest an
experiment that uses a very simple model?
Assuming that you have K subfiles created by the initial sorting pass,
insert the top record of each file into a priority queue.
Then, emit records from the queue until the priority queue is empty.
Now, there will be the objection that we will be jumping willy-nilly all
over the disk because of reading one record at a time, but (depending on
how it is implemented) generally several records are buffered during a
read.
So (as a gentle suggestion) I suggest testing the model. It works great
for a single CPU or multiple CPU system for the work that *I* do. I
have no idea if it will be a benefit for PostgreSQL or not, but it
should be a very simple matter to try it. As long as someone is doing
the work right now, it would be a good time to give it a go.
I am not very familiar with PostgreSQL internals, but I would be willing
to give a hand with it (not really sure how much time I can guarantee,
though, since I would be doing it on my free time).
