The backup failed with errors "error: could not connect to server: could not look up local user ID 1000: Too many open files" when the max_wal_senders was set to 2000.
The errors generated for the workers starting from backup worke=1017.
It wasn't the fact that you set max_wal_senders to 2000. It was the fact that you specified 1990 parallel workers. By so doing, you overloaded the machine, which is why everything failed. That's to be expected.
Thanks alot Robert,
In this case the backup folder was not being emptied as the backup was failed, the cleanup should be done in this case too.