By default temp files are created on hard drive as a result of low work_mem setting in postgres.conf. If your server runs on slow desktop hdd's it could be serious bottleneck in your performance so Yes, you should be concerned about it and tune work_mem parameter.