I'm not sure if it's safe to cancel forcibly VACUUM's truncation during scaning shared_buffers. That scan happens after WAL-logging and before the actual truncation.
Ah ok. I misread your proposal. This is about the shared_buffers scan in DropRelFileNodeBuffers() and we can't cancel that operation.
What if we remember the buffers as seen by count_nondeletable_pages() and then just discard those specific buffers instead of scanning the entire shared_buffers again? Surely we revisit all to-be-truncated blocks before actual truncation. So we already know which buffers to discard. And we're holding exclusive lock at that point, so nothing can change underneath. Of course, we can't really remember a large number of buffers, so we can do this in small chunks. Scan last K blocks, remember those K buffers, discard those K buffers, truncate the relation and then try for next K blocks. If another backend requests lock on the table, we give up or retry after a while.
